REVIEW 1 cited by
Return Capping: Sample-Efficient CVaR Policy Gradient Optimisation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Return Capping: Sample-Efficient CVaR Policy Gradient Optimisation
read the original abstract
When optimising for conditional value at risk (CVaR) using policy gradients (PG), current methods rely on discarding a large proportion of trajectories, resulting in poor sample efficiency. We propose a reformulation of the CVaR optimisation problem by capping the total return of trajectories used in training, rather than simply discarding them, and show that this is equivalent to the original problem if the cap is set appropriately. We show, with empirical results in an number of environments, that this reformulation of the problem results in consistently improved performance compared to baselines. We have made all our code available here: https://github.com/HarryMJMead/cvar-return-capping.
Forward citations
Cited by 1 Pith paper
-
Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity
A shifted-value transformation turns static CVaR MDPs into a bounded, contracting Bellman operator with dense rewards, enabling discretized value iteration and Q-learning with explicit error bounds.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.