REVIEW 7 cited by
Train Long, Think Short: Curriculum Learning for Efficient Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent work on enhancing the reasoning abilities of large language models (LLMs) has introduced explicit length control as a means of constraining computational cost while preserving accuracy. However, existing approaches rely on fixed-length training budgets, which do not take advantage of the natural progression from exploration to compression during learning. In this work, we propose a curriculum learning strategy for length-controlled reasoning using Group Relative Policy Optimization (GRPO). Our method starts with generous token budgets and gradually tightens them over training, encouraging models to first discover effective solution strategies and then distill them into more concise reasoning traces. We augment GRPO with a reward function that balances three signals: task correctness (via verifier feedback), length efficiency, and formatting adherence (via structural tags). Experiments on GSM8K, MATH500, SVAMP, College Math, and GSM+ demonstrate that curriculum-based training consistently outperforms fixed-budget baselines at the same final budget, achieving higher accuracy and significantly improved token efficiency. We further ablate the impact of reward weighting and decay schedule design, showing that progressive constraint serves as a powerful inductive bias for training efficient reasoning models. Our code and checkpoints are released at: https://github.com/hammoudhasan/curriculum_grpo.
Forward citations
Cited by 7 Pith papers
-
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
This survey introduces the Generate-Filter-Control-Replay (GFCR) taxonomy to structure rollout pipelines for RL-based post-training of reasoning LLMs.
-
Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models
Spoken math models that emit a 40%-compressed reasoning trace between question and answer beat full-reasoning baselines by ~3 accuracy points while using roughly one third of the text tokens.
-
GraphDancer: Training LLMs to Explore and Reason over Graphs via Two-Stage Curriculum Post-Training
Curriculum-based reinforcement learning teaches a small LLM to explore knowledge graphs with function calls and generalize across domains.
-
Training-Trajectory-Aware Token Selection
Training-Trajectory-Aware Token Selection (T3S) reconstructs the token-level training objective to overcome a performance bottleneck in continual distillation of reasoning capabilities from large to small language models.
-
Learning to Reason Efficiently with Discounted Reinforcement Learning
Discounted correctness rewards in GRPO post-training shorten LLM reasoning traces by roughly 13-40% while matching pass@1 accuracy, supported by Blackwell-optimality analysis for restricted policy classes.
-
Recursive Vision Language Models for General Symbolic Reasoning
R-Qwen, a LoRA-adapted Qwen model that iteratively refines explicit candidate solutions under constraint projection, outperforms prior recursive models and zero-shot frontier LLMs on eight symbolic reasoning benchmarks.
-
CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning
CLPO: an online curriculum that rewrites each problem according to the model's own current accuracy and scales the KL penalty by difficulty, reporting strong but internally inconsistent gains over GRPO and DAPO.
Discussion (0). Sign in to comment.