REVIEW 4 cited by
Projection-Based Constrained Policy Optimization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
We consider the problem of learning control policies that optimize a reward function while satisfying constraints due to considerations of safety, fairness, or other costs. We propose a new algorithm, Projection-Based Constrained Policy Optimization (PCPO). This is an iterative method for optimizing policies in a two-step process: the first step performs a local reward improvement update, while the second step reconciles any constraint violation by projecting the policy back onto the constraint set. We theoretically analyze PCPO and provide a lower bound on reward improvement, and an upper bound on constraint violation, for each policy update. We further characterize the convergence of PCPO based on two different metrics: $\normltwo$ norm and Kullback-Leibler divergence. Our empirical results over several control tasks demonstrate that PCPO achieves superior performance, averaging more than 3.5 times less constraint violation and around 15\% higher reward compared to state-of-the-art methods.
Forward citations
Cited by 4 Pith papers
-
PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learning
PNAct trains a safe RL agent to take unsafe actions only when a hidden trigger is present, while keeping normal safe behavior and reward when the trigger is absent.
-
Towards Adaptive External Communication in Autonomous Vehicles: A Conceptual Design Framework
A three-layer framework (input, processing, output) for adaptive external human-machine interfaces in autonomous vehicles is introduced to systematize design and analysis.
-
Proactive Constrained Policy Optimization with Preemptive Penalty
A constrained policy optimization method that uses a preemptive log-barrier penalty and a constraint-aware intrinsic reward to reduce safety violations in reinforcement learning.
-
HAEPO: History-Aggregated Exploratory Policy Optimization
HAEPO weights each trajectory by its softmax-normalized cumulative log-likelihood, adds entropy and KL penalties, and matches or slightly surpasses PPO, GRPO, and DPO on small RL and summarization tasks.
Discussion (0). Sign in to comment.