OrderGrad supplies unbiased likelihood-ratio and reparameterization gradient estimators for finite-sample L-statistics by applying a rank-based reward transformation usable in standard policy-gradient updates.
Rethinking reflection in pre- training.arXiv preprint arXiv:2504.04022
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
representative citing papers
One training example via RLVR boosts LLM math reasoning from 17.6% to 35.7% average across six benchmarks.
Proposes MaxPO using a Leave-Two-Out baseline for centered unbiased advantages in max@K policy gradients, with a unified derivation of finite-batch estimators.
Across four frontier reasoning models, 61–93% of correct chain-of-thought steps are redundant, and this over-thinking is provably optimal under any length-agnostic outcome reward.
Budget-matched evaluations and partial-prompt contamination probes show several RLVR reasoning gaps shrink or vanish once budgets, prompts, and dataset versions are controlled.
DARS adaptively increases rollouts on hard problems in RLVR to improve Pass@K, and when paired with batch scaling for breadth, achieves gains in both Pass@K and Pass@1 by treating depth and breadth as complementary exploration dimensions.
citing papers explorer
-
OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimation
OrderGrad supplies unbiased likelihood-ratio and reparameterization gradient estimators for finite-sample L-statistics by applying a rank-based reward transformation usable in standard policy-gradient updates.
-
Reinforcement Learning for Reasoning in Large Language Models with One Training Example
One training example via RLVR boosts LLM math reasoning from 17.6% to 35.7% average across six benchmarks.
-
On Advantage Estimates for Max@K Policy Gradients
Proposes MaxPO using a Leave-Two-Out baseline for centered unbiased advantages in max@K policy gradients, with a unified derivation of finite-batch estimators.
-
How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
Across four frontier reasoning models, 61–93% of correct chain-of-thought steps are redundant, and this over-thinking is provably optimal under any length-agnostic outcome reward.
-
Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
Budget-matched evaluations and partial-prompt contamination probes show several RLVR reasoning gaps shrink or vanish once budgets, prompts, and dataset versions are controlled.
-
Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
DARS adaptively increases rollouts on hard problems in RLVR to improve Pass@K, and when paired with batch scaling for breadth, achieves gains in both Pass@K and Pass@1 by treating depth and breadth as complementary exploration dimensions.