REVIEW 16 cited by
Rethinking Reflection in Pre-Training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
A language model's ability to reflect on its own reasoning provides a key advantage for solving complex problems. While most recent research has focused on how this ability develops during reinforcement learning, we show that it actually begins to emerge much earlier - during the model's pre-training. To study this, we introduce deliberate errors into chains-of-thought and test whether the model can still arrive at the correct answer by recognizing and correcting these mistakes. By tracking performance across different stages of pre-training, we observe that this self-correcting ability appears early and improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4 trillion tokens displays self-correction on our six self-reflection tasks.
Forward citations
Cited by 16 Pith papers
-
Few-Shot Vision-Language Reasoning for Satellite Imagery via Verifiable Rewards
A few-shot RLVR method using only rule-based rewards lifts a 2B vision-language model's remote sensing accuracy by double digits, with 128 examples rivaling thousands.
-
Text Detoxification: Data Efficiency, Semantic Preservation and Model Generalization
A two-stage SFT plus GRPO framework with a composite toxicity and semantic-similarity reward achieves state-of-the-art detoxification performance on ParaDetox using only 20% of annotated data.
-
GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking
A cue-tagging and rethinking training recipe lifts a 7B multimodal model to 81.5% on M3CoT, though the gain may reflect training on the same benchmark.
-
J1: Exploring Simple Test-Time Scaling for LLM-as-a-Judge
J1-7B, a judge LLM trained with supervised fine-tuning and reinforcement learning, improves when forced to reflect with 'wait' tokens, and the scaling ability emerges during the RL phase.
-
Natural Language Reinforcement Learning
NLRL replaces scalar RL values with LLM-generated language narratives, trains language critics with language MC/TD, and improves policies via LLM-based policy iteration, outperforming PPO on four small agentic tasks.
-
Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoning
A plan-then-reason SFT plus a plan-quality reward in GRPO improves math-reasoning accuracy by small but consistent margins over GRPO and DAPO.
-
Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
The paper identifies confounds in RLVR evaluations that inflate apparent gains and proposes a minimum standard for budget-matched, contamination-aware assessment with calibration tracking.
-
Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entities
Giving an LLM a partial fact it already knows can trigger correct answers to questions it could not answer alone.
-
REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once
Asking a reasoning model several problems at once reveals large accuracy drops and exposes differences that single-question benchmarks miss.
-
Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
Suppressing low-probability 'wait' tokens that trigger self-affirmation reflections shortens reasoning-model output by 8-18% with roughly unchanged accuracy.
-
From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
Self-reflection in LLMs can be steered up or down by a single activation-space vector, improving accuracy when amplified and cutting output length when suppressed.
-
GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior
GUI-Reflection trains an 8B multimodal GUI agent to recognize mistakes, undo incorrect actions, and retry, improving AndroidWorld success rate from 14.58% (filtered BC baseline) to 34.72% with reflection data and onli...
-
Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start
Using a two-stage SFT-plus-GRPO pipeline, the authors train 3B and 7B multimodal models that beat prior open-source models on four math reasoning benchmarks, while showing that reflective 'aha moment' text is not a re...
-
Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective
Reasoning trajectories are formalized as pseudo-gradient descent on LLM parameters, making LLM reasoning training a MAML-style meta-learning problem.
-
Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them
GRPO produces modest math gains with small knowledge loss, while SFT gives larger math gains but degrades knowledge benchmarks more, with parameter analyses suggesting the loss comes from larger mid-layer MLP updates.
-
Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.
Discussion (0). Continue with ORCID to comment.