REVIEW 6 cited by
Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
In this paper, we propose R$^3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL), a novel method that employs only outcome supervision to achieve the benefits of process supervision for large language models. The core challenge in applying RL to complex reasoning is to identify a sequence of actions that result in positive rewards and provide appropriate supervision for optimization. Outcome supervision provides sparse rewards for final results without identifying error locations, whereas process supervision offers step-wise rewards but requires extensive manual annotation. R$^3$ overcomes these limitations by learning from correct demonstrations. Specifically, R$^3$ progressively slides the start state of reasoning from a demonstration's end to its beginning, facilitating easier model exploration at all stages. Thus, R$^3$ establishes a step-wise curriculum, allowing outcome supervision to offer step-level signals and precisely pinpoint errors. Using Llama2-7B, our method surpasses RL baseline on eight reasoning tasks by $4.1$ points on average. Notebaly, in program-based reasoning on GSM8K, it exceeds the baseline by $4.2$ points across three backbone models, and without any extra data, Codellama-7B + R$^3$ performs comparable to larger models or closed-source models.
Forward citations
Cited by 6 Pith papers
-
SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Information
A new human-curated benchmark for LLM mobile assistants shows models struggle to locate scattered personal information, with the best model at 57.3% accuracy.
-
Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems
AdaPrefix-GRPO treats solution-prefix length as a feedback controller targeting 50% rollout success rate during GRPO training, then anneals to zero prefix, yielding 1.6–2.1× accuracy gains over vanilla GRPO at matched...
-
From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
The Diligent Learner, a reverse-curriculum algorithm with explicit backtracking and a validator, is proven to learn chain-of-thought reasoning efficiently under two learnability assumptions, while standard methods fai...
-
Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
A trained critique model providing step-level feedback improves an LLM actor's mathematical reasoning at test time and during self-improvement, with gains concentrated on hard problems.
-
Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning
CCL orders LLM training data by the model's own measured accuracy and converts the hardest problems into hinted completion tasks, reporting higher average benchmark scores than uniform training.
-
From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications
This paper is a broad tutorial on applying LAMs and agentic AI to 6G, largely restating existing research rather than introducing new results.
Discussion (0). Continue with ORCID to comment.