Pith. sign in

REVIEW 26 cited by

Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.06395 v3 pith:ZTCDUZCN submitted 2025-06-05 cs.CL cs.LG

Pengyi Li , Matvey Skripkin , Alexander Zubrey , Andrey Kuznetsov , Ivan Oseledets This is my paper
classification cs.CLcs.LG
keywords modelsrewardrlscconfidencelanguagelearningneedonly
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) excel at reasoning, yet post-training remains critical for aligning their behavior with task goals. Existing reinforcement learning (RL) methods often depend on costly human annotations or external reward models. We propose Reinforcement Learning via Self-Confidence (RLSC), which uses the model's own confidence as reward signals-eliminating the need for labels, preference models, or reward engineering. Applied to Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps, RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a simple, scalable post-training method for inference models, requiring only a small number of samples and unlabelled supervision.

Discussion (0). Sign in to comment.

Forward citations

Cited by 26 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. When Do Intrinsic Rewards Work for Code Reasoning? A Comprehensive Study

    cs.AI 2026-06 unverdicted novelty 7.0 of 10

    Empirical evaluation on LiveCodeBench shows certainty-based RLIF yields early gains followed by output shortening and reasoning collapse, providing no advantage for RLVR initialization on code tasks.

  2. Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Voting

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    TTRL gains are reinterpreted as mostly sharpening rather than learning, with an identified extinction window causing net corruption; TTRL-Guard mitigates via FRS, MPS, and RCSU for improved pass@1.

  3. Relationship-Centered Care: Relatedness and Responsible Design for Human Connections in Mental-Health Care

    cs.HC 2026-03 accept novelty 7.0 of 10

    Length-aware GFlowNet matching of α-power base distributions elicits either stronger LLM reasoning (α>1) or restored creativity (α<1) without external supervision, matching or beating RLIF and GRPO baselines.

  4. Architecture-Aware Reinforcement Learning Makes Sliding-Window Attention Competitive in Math Reasoning

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    Reinforcement learning after SFT conversion narrows the performance gap between sliding-window attention and full self-attention on math reasoning benchmarks while preserving linear complexity.

  5. Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Voting

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    TTRL-Guard mitigates the Correct-Answer Extinction Window in test-time RL via flip-rate-aware reward scaling, minority-preserving sampling, and risk-conditioned sparse updates, yielding best average pass@1 on Qwen mod...

  6. PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    PDCR improves vision-language reasoning by computing separate normalized confidence advantages for perception steps and reasoning steps after unsupervised decomposition.

  7. Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    Mutual Reinforcement Learning allows heterogeneous LLMs to exchange experience through mechanisms like Peer Rollout Pooling, Cross-Policy GRPO Advantage Sharing, and Success-Gated Transfer, with outcome-level sharing ...

  8. Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    Large reasoning models show measurable hidden-state dynamics that a new statistic can use to distinguish correct reasoning trajectories without labels.

  9. Hallucinations Undermine Trust; Metacognition is a Way Forward

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    LLMs need metacognition to align expressed uncertainty with their actual knowledge boundaries, moving beyond knowledge expansion to reduce confident errors.

  10. HEALing Entropy Collapse: Enhancing Exploration in Few-Shot RLVR via Hybrid-Domain Entropy Dynamics Alignment

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    HEAL mitigates entropy collapse in few-shot RLVR by selectively adding general-domain data and aligning trajectory-level entropy dynamics, matching full-shot performance with 32 target samples.

  11. Can LLMs Learn to Reason Robustly under Noisy Supervision?

    cs.LG 2026-04 conditional novelty 6.0 of 10

    Online Label Refinement lets LLMs learn robust reasoning from noisy supervision by correcting labels when majority answers show rising rollout success and stable history, delivering 3-4% gains on math and reasoning be...

  12. Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards

    cs.LG 2026-03 unverdicted novelty 6.0 of 10

    DCPO decouples reasoning optimization from calibration in RLVR to fix overconfidence in LLMs without losing accuracy.

  13. Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards

    cs.LG 2026-03 unverdicted novelty 6.0 of 10

    DCPO decouples reasoning and calibration in RLVR, matching GRPO accuracy while reducing overconfidence via a claimed fundamental gradient conflict.

  14. CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning

    cs.CL 2026-02 conditional novelty 6.0 of 10

    CPMobius uses iterative coach-player reinforcement learning to improve mathematical reasoning in LLMs without external training data, yielding +4.9 average accuracy gains on Qwen2.5-Math-7B-Instruct.

  15. CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning

    cs.CL 2026-02 conditional novelty 6.0 of 10

    A cooperative Coach–Player RL loop, rewarded by Player validation-progress, lifts math reasoning by 2–5 points while the Player trains only on self-generated pseudo-labeled tasks.

  16. Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RL

    cs.LG 2025-10 conditional novelty 6.0 of 10

    Self-rewarding RL can be stabilized by ensembling multiple policy models' majority-vote rewards, reaching within 3.6% of verifiable-reward RL on math benchmarks.

  17. Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking

    cs.CV 2025-09 conditional novelty 6.0 of 10

    EvoQuality lets a VLM self-train for image quality assessment using majority-voted pairwise preferences as pseudo-labels, improving PLCC by about 32% and rivaling supervised models.

  18. Compute as Teacher: Turning Inference Compute Into Reference-Free Supervision

    cs.LG 2025-09 unverdicted novelty 6.0 of 10

    Parallel inference rollouts aggregated into pseudo-references enable reference-free RL supervision that matches expert-annotated performance on health tasks while using 9x less test-time compute.

  19. Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Selecting the most confident 32-token prefix and completing only it gives better accuracy per compute than majority voting on five math reasoning datasets, using only the model's own confidence as a selector.

  20. GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling

    cs.LG 2026-06 unverdicted novelty 5.0 of 10

    GeoMin uses geometric distribution modeling on labeled data to assess self-reward reliability, enabling better performance in semi-supervised RLVR with only 10% of typical annotations.

  21. Trust Region On-Policy Distillation

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    TrOPD stabilizes on-policy distillation for LLMs with trust-region learning, outlier estimation, and off-policy guidance, outperforming prior OPD methods on reasoning and code benchmarks.

  22. Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    FEST improves RLVR sample efficiency on math and coding benchmarks by combining supervised signals, on-policy signals, and decaying weights on just 128 randomly chosen demonstrations, matching full-dataset baselines.

  23. Free Energy-Driven Reinforcement Learning with Adaptive Advantage Shaping for Unsupervised Reasoning in LLMs

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    FREIA applies free energy principles and adaptive advantage shaping to unsupervised RL, outperforming baselines by 0.5-3.5 Pass@1 points on math reasoning with a 1.5B model.

  24. Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots

    cs.LG 2026-06 unverdicted novelty 4.0 of 10

    PivotTrace selects unlabeled data for RLVR by quantifying uncertainty via pivot density from attention dynamics, outperforming full supervision using only 29.3% annotations and converging 2.75 times faster.

  25. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

  26. A Survey of Reinforcement Learning for Large Reasoning Models

    cs.CL 2025-09 accept novelty 3.0 of 10

    A survey compiling RL methods, challenges, data resources, and applications for enhancing reasoning in large language models and large reasoning models since DeepSeek-R1.

Pith tools