Pith. sign in

REVIEW 20 cited by

Self-rewarding correction for mathematical reasoning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.19613 v1 pith:X3EKF6HQ submitted 2025-02-26 cs.AI cs.LG

Wei Xiong , Hanning Zhang , Chenlu Ye , Lichang Chen , Nan Jiang , Tong Zhang This is my paper
classification cs.AIcs.LG
keywords modelsreasoningself-rewardingself-correctionoutputsallowsapproachdata
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We study self-rewarding reasoning large language models (LLMs), which can simultaneously generate step-by-step reasoning and evaluate the correctness of their outputs during the inference time-without external feedback. This integrated approach allows a single model to independently guide its reasoning process, offering computational advantages for model deployment. We particularly focus on the representative task of self-correction, where models autonomously detect errors in their responses, revise outputs, and decide when to terminate iterative refinement loops. To enable this, we propose a two-staged algorithmic framework for constructing self-rewarding reasoning models using only self-generated data. In the first stage, we employ sequential rejection sampling to synthesize long chain-of-thought trajectories that incorporate both self-rewarding and self-correction mechanisms. Fine-tuning models on these curated data allows them to learn the patterns of self-rewarding and self-correction. In the second stage, we further enhance the models' ability to assess response accuracy and refine outputs through reinforcement learning with rule-based signals. Experiments with Llama-3 and Qwen-2.5 demonstrate that our approach surpasses intrinsic self-correction capabilities and achieves performance comparable to systems that rely on external reward models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning

    cs.AI 2026-06 unverdicted novelty 7.0 of 10

    ReSum trains LLMs via RLVR to self-summarize reasoning trajectories, yielding 4% average performance gains and 18.6% shorter rollouts through contrastive rollout branches.

  2. ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization

    cs.SE 2026-04 unverdicted novelty 7.0 of 10

    ACE uses a solver-adversary loop with adversarial unit test generation and execution-based preference optimization to enable self-evolving LLM code generation, reporting 3-7% pass@1 gains over solver-verifier baseline...

  3. Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction

    cs.CL 2026-06 conditional novelty 6.5 of 10

    Structured cognitive-episode features from LRM reasoning traces, combined with item semantics, improve human item-difficulty prediction and show harder items drive more implementation-centered, iterative solving.

  4. Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    Epi2Diff extracts cognitive episode sequences from LRM reasoning traces and combines them with semantic features to predict human item difficulty, outperforming baselines on four educational datasets.

  5. ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning

    cs.AI 2026-06 conditional novelty 6.0 of 10

    ReSum's contrastive RL branching on self-summarization points improves LLM math reasoning accuracy by about 4% and shortens rollouts by about 18.6% across tested backbones.

  6. Guarded Repair for Harm-Aware Post-hoc Replacement of LLM Mathematical Reasoning

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    GuardedRepair uses guarded best-of-N repair with symbolic checks, semantic diagnostics, and conservative policies to selectively replace LLM reasoning traces, raising GSM8K accuracy from 95.60% to 96.89% and ASDiv fro...

  7. Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    A new RL paradigm for reasoning where models generate their own internal process supervision from outcome feedback by recycling failed trajectories.

  8. Can LLMs Learn to Reason Robustly under Noisy Supervision?

    cs.LG 2026-04 conditional novelty 6.0 of 10

    Online Label Refinement lets LLMs learn robust reasoning from noisy supervision by correcting labels when majority answers show rising rollout success and stable history, delivering 3-4% gains on math and reasoning be...

  9. rePIRL: Learn PRM with Inverse RL for LLM Reasoning

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    rePIRL learns effective process reward models for LLM reasoning via a dual policy-PRM update process inspired by inverse RL, unifying online and offline methods with reported gains over prior approaches on math and co...

  10. CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning

    cs.CL 2026-02 conditional novelty 6.0 of 10

    CPMobius uses iterative coach-player reinforcement learning to improve mathematical reasoning in LLMs without external training data, yielding +4.9 average accuracy gains on Qwen2.5-Math-7B-Instruct.

  11. CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning

    cs.CL 2026-02 conditional novelty 6.0 of 10

    A cooperative Coach–Player RL loop, rewarded by Player validation-progress, lifts math reasoning by 2–5 points while the Player trains only on self-generated pseudo-labeled tasks.

  12. Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RL

    cs.LG 2025-10 conditional novelty 6.0 of 10

    Self-rewarding RL can be stabilized by ensembling multiple policy models' majority-vote rewards, reaching within 3.6% of verifiable-reward RL on math benchmarks.

  13. LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?

    cs.CL 2025-10 unverdicted novelty 6.0 of 10

    LightReasoner distills supervision signals from SLM-LLM behavioral divergence to improve LLM reasoning on math benchmarks with up to 28.1% accuracy gains and 90-99% reductions in resources.

  14. Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training

    cs.LG 2025-09 unverdicted novelty 6.0 of 10

    PROF curates RL training data via PRM-ORM consistency to improve both final-answer accuracy and intermediate reasoning quality while reducing reliance on strong process reward models.

  15. Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning

    cs.LG 2026-06 unverdicted novelty 5.0 of 10

    Survey mapping RL techniques onto LLM training and highlighting gaps in value-based, off-policy, and bootstrapping methods.

  16. Trust Region On-Policy Distillation

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    TrOPD stabilizes on-policy distillation for LLMs with trust-region learning, outlier estimation, and off-policy guidance, outperforming prior OPD methods on reasoning and code benchmarks.

  17. Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning

    cs.LG 2026-04 unverdicted novelty 5.0 of 10

    A method internalizes outcome supervision into process supervision by extracting step-level learning signals from failed reasoning trajectories during reinforcement learning.

  18. ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization

    cs.SE 2026-04 unverdicted novelty 5.0 of 10

    ACE introduces a solver-adversary loop where an LLM generates both candidate programs and adversarial tests, using execution outcomes for preference optimization to achieve 3-7% pass@1 gains on code benchmarks without...

  19. The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes

    cs.CL 2026-06 unverdicted novelty 4.0 of 10

    A literature survey that introduces a taxonomy for LLM reasoning paradigms, analyzes methodological trends, and synthesizes failure modes from over 300 papers.

  20. rePIRL: Learn PRM with Inverse RL for LLM Reasoning

    cs.LG 2026-02 conditional novelty 4.0 of 10

    rePIRL learns token-level process rewards for LLM reasoning via a guided-cost-learning-style IRL objective, and shows these rewards improve reasoning policies on math/coding benchmarks.

Pith tools