Pith. sign in

REVIEW 34 cited by

ReFT: Reasoning with Reinforced Fine-Tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.08967 v3 pith:FKUZSAER submitted 2024-01-17 cs.CL

Trung Quoc Luong , Xinbo Zhang , Zhanming Jie , Peng Sun , Xiaoran Jin , Hang Li This is my paper · ORCID
classification cs.CL
keywords reasoningrefttrainingfine-tuninggivenlearningquestionability
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

One way to enhance the reasoning capability of Large Language Models (LLMs) is to conduct Supervised Fine-Tuning (SFT) using Chain-of-Thought (CoT) annotations. This approach does not show sufficiently strong generalization ability, however, because the training only relies on the given CoT data. In math problem-solving, for example, there is usually only one annotated reasoning path for each question in the training data. Intuitively, it would be better for the algorithm to learn from multiple annotated reasoning paths given a question. To address this issue, we propose a simple yet effective approach called Reinforced Fine-Tuning (ReFT) to enhance the generalizability of learning LLMs for reasoning, with math problem-solving as an example. ReFT first warmups the model with SFT, and then employs on-line reinforcement learning, specifically the PPO algorithm in this paper, to further fine-tune the model, where an abundance of reasoning paths are automatically sampled given the question and the rewards are naturally derived from the ground-truth answers. Extensive experiments on GSM8K, MathQA, and SVAMP datasets show that ReFT significantly outperforms SFT, and the performance can be potentially further boosted by combining inference-time strategies such as majority voting and re-ranking. Note that ReFT obtains the improvement by learning from the same training questions as SFT, without relying on extra or augmented training questions. This indicates a superior generalization ability for ReFT.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 34 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Self-Improving Large Language Models via Progressive Experience Evolution

    cs.CL 2026-08 conditional novelty 7.0 of 10

    SPEE improves LLM math reasoning by explicitly evolving a global pool of transferable experience and distilling it into the policy before applying GRPO reinforcement learning.

  2. Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

    cs.LG 2026-08 conditional novelty 6.0 of 10

    For mid-tier LLMs, generating paraphrased versions of an input and majority voting over answers converts inference budget into accuracy more efficiently than self-consistency sampling.

  3. MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation

    cs.AI 2026-07 conditional novelty 6.0 of 10

    MAGA distills three domain-specific GUI agents into one student by amplifying supervision on wrongly generated action tokens and feeding the teacher a training-only hint of the correct action type, improving mean succ...

  4. LeAct: Learning to Reason from Expert Actions

    cs.LG 2026-07 conditional novelty 6.0 of 10

    An AI can learn to reason by sampling explanations for an expert's actions and keeping only the ones that help it predict those actions.

  5. SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text

    cs.AI 2026-05 conditional novelty 6.0 of 10

    A 24-dataset benchmark for inducing schema graphs from raw text, plus an auditable LLM-based pipeline that reports the highest scores on the benchmark's four schema-similarity metrics.

  6. rePIRL: Learn PRM with Inverse RL for LLM Reasoning

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    rePIRL learns token-level process rewards for LLM reasoning via a guided-cost-learning-style IRL objective, and shows these rewards improve reasoning policies on math/coding benchmarks.

  7. AdsQA: Towards Advertisement Video Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    AdsQA adds an ad-video question-answering benchmark and ReAd-R, a GRPO-trained model that beats 7B baselines but not larger closed models.

  8. Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice

    cs.CL 2025-07 conditional novelty 6.0 of 10

    An end-to-end simultaneous speech-to-speech translation model with voice cloning, trained with a two-stage reinforcement learning reward scheme, reports high accuracy and low latency on the authors' RealSI benchmark.

  9. One Token to Fool LLM-as-a-Judge

    cs.LG 2025-07 conditional novelty 6.0 of 10

    LLM reward models falsely accept empty 'master key' responses such as ':' or 'Thought process:' across many models, and a fine-tuning defense reduces these false positives to near zero.

  10. BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset

    cs.CL 2025-07 conditional novelty 6.0 of 10

    BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.

  11. Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A 3B model trained with a small SFT warm-up followed by verifiable-reward RL matches or exceeds far larger models on EHR-based medical calculation, trial matching, and diagnosis tasks.

  12. ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making

    cs.RO 2025-05 conditional novelty 6.0 of 10

    ManiTaskGen automatically generates diverse, feasible mobile manipulation tasks from any input scene, and uses them to benchmark and improve vision-language robot agents.

  13. Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Guided by Gut lets an LLM guide its own search using token-level confidence and step novelty, with a RL fine-tuning step to calibrate that confidence, achieving PRM-level accuracy at far lower compute and memory.

  14. Tina: Tiny Reasoning Models via LoRA

    cs.CL 2025-04 conditional novelty 6.0 of 10

    Applying LoRA during reinforcement learning to a 1.5B reasoning model yields competitive math and scientific reasoning at roughly $9 of training and evaluation cost.

  15. Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance

    cs.CL 2025-02 conditional novelty 6.0 of 10

    Fin-o1 shows that an 8B or 14B model trained on a finance-specific chain-of-thought corpus with GRPO can outperform general-purpose reasoning models like GPT-o1 and DeepSeek-R1 on financial reasoning benchmarks.

  16. RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception

    cs.CV 2025-01 conditional novelty 6.0 of 10

    An RL agent generates hard synthetic spatial-reasoning examples to fine-tune VLMs, improving performance on simulated test scenes.

  17. Towards Sparse Video Understanding and Reasoning

    cs.CV 2026-02 conditional novelty 5.0 of 10

    A video-QA agent that carries only a structured text summary between rounds beats dense-frame baselines on accuracy while using a handful of frames per video.

  18. Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards

    cs.LG 2025-09 conditional novelty 5.0 of 10

    The paper identifies confounds in RLVR evaluations that inflate apparent gains and proposes a minimum standard for budget-matched, contamination-aware assessment with calibration tracking.

  19. GPO: Learning from Critical Steps to Improve LLM Reasoning

    cs.AI 2025-09 conditional novelty 5.0 of 10

    Guided Pivotal Optimization improves LLM reasoning post-training by resetting rollouts at an advantage-estimated critical step, beating each of five base optimizers on seven benchmarks in 34 of 35 comparisons.

  20. Beyond Isolated Capabilities: Bridging Long CoT Reasoning and Long-Context Understanding

    cs.CL 2025-07 conditional novelty 5.0 of 10

    DeepSeek-R1-distilled models show higher multi-document QA accuracy than their base counterparts and flatter position-bias curves, especially with 50-80 documents.

  21. Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training

    cs.LG 2025-07 conditional novelty 5.0 of 10

    Reinforcement fine-tuning largely prevents catastrophic forgetting during continual post-training of a multimodal LLM, while supervised fine-tuning degrades both task and general performance.

  22. SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning

    cs.LG 2025-06 conditional novelty 5.0 of 10

    SwS uses failures during RL training to synthesize targeted math problems, improving reasoning accuracy on eight benchmarks.

  23. Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs

    cs.LG 2025-05 conditional novelty 5.0 of 10

    SASR dynamically mixes SFT and GRPO training steps using a gradient-norm ratio, and reports improved average accuracy over static SFT-then-RL schedules on GSM8K, MATH, and Knight-and-Knives.

  24. IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment

    cs.AI 2025-05 reject novelty 5.0 of 10

    IDEAL tunes SFT data mixture proportions per domain with influence-function gradients, claiming about 7% average benchmark improvement over uniform mixing.

  25. Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimization

    cs.CL 2025-04 conditional novelty 5.0 of 10

    Combining continued pretraining with reasoning preference optimization yields a 72B Japanese medical model that keeps 0.868 accuracy on IgakuQA with and without explanation prompting, while a model without RPO drops to 0.834.

  26. RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?

    cs.LG 2025-01 reject novelty 5.0 of 10

    Long-CoT fine-tuning with QwQ-generated traces improves math and geometry benchmarks, and RedStar-DPO surpasses QwQ on AIME24, but several reported numbers conflict with the paper's own tables.

  27. ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding

    cs.CL 2025-01 conditional novelty 5.0 of 10

    ReARTeR improves RAG multi-step reasoning by combining a process reward model, an aligned explanation model, temporal-difference lookahead, and MCTS-based iterative preference training.

  28. Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handling

    cs.SE 2024-12 conditional novelty 5.0 of 10

    Tree-of-Code improves code-based LLM agents by generating complete end-to-end programs as tree nodes, using execution success to self-filter and majority voting to pick the final answer.

  29. Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning

    cs.CL 2025-06 conditional novelty 4.0 of 10

    CCL orders LLM training data by the model's own measured accuracy and converts the hardest problems into hinted completion tasks, reporting higher average benchmark scores than uniform training.

  30. AI Agent Behavioral Science

    q-bio.NC 2025-06 conditional novelty 4.0 of 10

    AI agents should be studied as behavioral entities shaped by context and interaction, not only as trained models.

  31. Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertise

    cs.AI 2025-05 reject novelty 4.0 of 10

    A multi-agent router that selects culturally specialized LLM personas reports a jump in self-scored cultural alignment from 0.208 to 0.820, but the metric and the claimed method are not independently validated.

  32. Natural Language Fine-Tuning

    cs.CL 2024-12 reject novelty 4.0 of 10

    NLFT weights each token by how much its probability shifts under different natural language prompts and claims to beat supervised fine-tuning with 50 examples, but the paper's loss equation and reported gains are inte...

  33. Reward Models in Deep Reinforcement Learning: A Survey

    cs.LG 2025-06 conditional novelty 3.0 of 10

    A structured survey of reward modeling in deep RL, proposing a three-axis taxonomy and reviewing applications and evaluation methods.

  34. Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models

    cs.CL 2025-06 reject novelty 2.0 of 10

    RLSC, a method that uses a language model's self-confidence as reward, is shown to improve math benchmark accuracy, but the results are undermined by training on the AIME test set and the method reduces to known self-...

Pith tools