REVIEW 2 cited by
Clipped Action Policy Gradient
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of actions being clipped to reduce the variance in estimation. We prove that our estimator, named clipped action policy gradient (CAPG), is unbiased and achieves lower variance than the conventional estimator that ignores action bounds. Experimental results demonstrate that CAPG generally outperforms the conventional estimator, indicating that it is a better policy gradient estimator for continuous control tasks. The source code is available at https://github.com/pfnet-research/capg.
Forward citations
Cited by 2 Pith papers
-
Generalization in Transfer Learning
Regularized PPO and adversarial RL variants (SC-PPO, ACC-RARL, ME-RARL) with early stopping extend reported transfer success in MuJoCo control tasks beyond RARL, but the gains depend on oracle selection of policy snapshots.
-
Health-Informed Policy Gradients for Multi-Agent Reinforcement Learning
A minimum-health counterfactual baseline for multi-agent policy gradients improves learning speed in continuous-control tasks with agent attrition.
Discussion (0). Continue with ORCID to comment.