PromptPO shows LLMs can act as black-box policy optimizers for sequential RL when leveraging prior knowledge, matching baselines in exploration and robotics but underperforming in MuJoCo.
Correlated proxies: A new definition and improved mitigation for reward hacking.arXiv preprint arXiv:2403.03185
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
years
2026 5verdicts
UNVERDICTED 5representative citing papers
Prompt-elicited hacking trajectories do not reflect training-time reward hacking in code generation; monitors trained on Trace-and-Amplify data generalize better to unseen hacking types.
Trusted-direction projection constrains RL gradient updates in language models to a low-dimensional clean subspace, reducing reward hacking on mathematical reasoning tasks.
ConsistRM improves generative reward models via consistency-aware self-training, outperforming vanilla RFT by 1.5% on average across five benchmarks and four base models.
An off-Earth autonomy pathway can reduce AGI confrontation incentives by making early cooperation preferable to power-seeking on Earth.
citing papers explorer
-
When are LLMs Sufficient Policy Optimizers for Sequential RL Tasks?
PromptPO shows LLMs can act as black-box policy optimizers for sequential RL when leveraging prior knowledge, matching baselines in exploration and robotics but underperforming in MuJoCo.
-
Do Prompt-Elicited Trajectories Reflect Training-Time Reward Hacking? A Systematic Study on Monitoring Trainig-Time Reward Hacking in Code Generation
Prompt-elicited hacking trajectories do not reflect training-time reward hacking in code generation; monitors trained on Trace-and-Amplify data generalize better to unseen hacking types.
-
Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
Trusted-direction projection constrains RL gradient updates in language models to a low-dimensional clean subspace, reducing reward hacking on mathematical reasoning tasks.
-
ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training
ConsistRM improves generative reward models via consistency-aware self-training, outperforming vanilla RFT by 1.5% on average across five benchmarks and four base models.
-
Reframing AGI Confrontation with Off Earth Autonomy
An off-Earth autonomy pathway can reduce AGI confrontation incentives by making early cooperation preferable to power-seeking on Earth.