Pith. sign in

REVIEW 8 cited by

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2507.21931 v1 pith:PRXD77GH submitted 2025-07-29 cs.CL cs.AI

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

Carel van Niekerk , Renato Vukovic , Benjamin Matthias Ruppik , Hsien-chin Lin , Milica Ga\v{s}i\'c This is my paper
classification cs.CL cs.AI
keywords modelpost-trainingintrinsiclearningreinforcementrlsfself-feedbackanswers
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks. We present Reinforcement Learning from Self-Feedback (RLSF), a post-training stage that uses the model's own confidence as an intrinsic reward, mimicking how humans learn in the absence of external feedback. After a frozen LLM generates several chain-of-thought solutions, we define and compute the confidence of each final answer span and rank the traces accordingly. These synthetic preferences are then used to fine-tune the policy with standard preference optimization, similar to RLHF yet requiring no human labels, gold answers, or externally curated rewards. RLSF simultaneously (i) refines the model's probability estimates -- restoring well-behaved calibration -- and (ii) strengthens step-by-step reasoning, yielding improved performance on arithmetic reasoning and multiple-choice question answering. By turning a model's own uncertainty into useful self-feedback, RLSF affirms reinforcement learning on intrinsic model behaviour as a principled and data-efficient component of the LLM post-training pipeline and warrents further research in intrinsic rewards for LLM post-training.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. When Do Intrinsic Rewards Work for Code Reasoning? A Comprehensive Study

    cs.AI 2026-06 unverdicted novelty 7.0

    Empirical evaluation on LiveCodeBench shows certainty-based RLIF yields early gains followed by output shortening and reasoning collapse, providing no advantage for RLVR initialization on code tasks.

  2. Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs

    cs.CL 2026-06 unverdicted novelty 6.0

    RLMF uses quality of model self-judgments to refine RL rankings and select training data, achieving SOTA faithful calibration while preserving accuracy and outperforming standard RL by up to 63%.

  3. GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling

    cs.LG 2026-06 unverdicted novelty 5.0

    GeoMin uses geometric distribution modeling on labeled data to assess self-reward reliability, enabling better performance in semi-supervised RLVR with only 10% of typical annotations.

  4. Trust Region On-Policy Distillation

    cs.LG 2026-05 unverdicted novelty 5.0

    TrOPD stabilizes on-policy distillation for LLMs with trust-region learning, outlier estimation, and off-policy guidance, outperforming prior OPD methods on reasoning and code benchmarks.

  5. Towards AI epidemiology: a measurement standardisation framework for prospective risk detection

    cs.AI 2025-12 conditional novelty 5.0

    A Logia grammar is introduced that turns expert–AI interactions into eight standardised fields, enabling population-level surveillance of AI output failures without access to model internals.

  6. Self-Rewarding Vision-Language Model via Reasoning Decomposition

    cs.CV 2025-08 unverdicted novelty 5.0

    Vision SR1 decomposes VLM reasoning into visual and language components and uses internal self-rewards to improve visual reasoning and reduce hallucinations more efficiently than external-supervision methods.

  7. From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoning

    cs.AI 2026-01 conditional novelty 4.0

    Post-training LLMs first on abstract number-free reasoning plans (CoMT), then with confidence-weighted rewards (CCRL), raises math accuracy by ~2-5 points over standard CoT-SFT+RL across four models.

  8. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.