Pith. sign in

REVIEW 22 cited by

LIMR: Less is More for RL Scaling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.11886 v1 pith:PFSNMZPS submitted 2025-02-17 cs.LG cs.AIcs.CL

Xuefeng Li , Haoyang Zou , Pengfei Liu This is my paper
classification cs.LGcs.AIcs.CL
keywords trainingdatalimrmodelssamplesscalingachievesb-scale
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this paper, we ask: what truly determines the effectiveness of RL training data for enhancing language models' reasoning capabilities? While recent advances like o1, Deepseek R1, and Kimi1.5 demonstrate RL's potential, the lack of transparency about training data requirements has hindered systematic progress. Starting directly from base models without distillation, we challenge the assumption that scaling up RL training data inherently improves performance. we demonstrate that a strategically selected subset of just 1,389 samples can outperform the full 8,523-sample dataset. We introduce Learning Impact Measurement (LIM), an automated method to evaluate and prioritize training samples based on their alignment with model learning trajectories, enabling efficient resource utilization and scalable implementation. Our method achieves comparable or even superior performance using only 1,389 samples versus the full 8,523 samples dataset. Notably, while recent data-efficient approaches (e.g., LIMO and s1) show promise with 32B-scale models, we find it significantly underperforms at 7B-scale through supervised fine-tuning (SFT). In contrast, our RL-based LIMR achieves 16.7% higher accuracy on AIME24 and outperforms LIMO and s1 by 13.0% and 22.2% on MATH500. These results fundamentally reshape our understanding of RL scaling in LLMs, demonstrating that precise sample selection, rather than data scale, may be the key to unlocking enhanced reasoning capabilities. For reproducible research and future innovation, we are open-sourcing LIMR, including implementation of LIM, training and evaluation code, curated datasets, and trained models at https://github.com/GAIR-NLP/LIMR.

Discussion (0). Sign in to comment.

Forward citations

Cited by 22 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A Kalman filter with policy-update-coupled process noise tracks non-stationary prompt difficulty and selects intermediate-difficulty batches for RL finetuning without extra rollouts.

  2. Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVR

    cs.LG 2026-07 conditional novelty 6.0 of 10

    SARA uses a Bayesian stopping rule on a prompt's first few responses to abandon saturated response groups early and reallocate rollouts, matching accuracy with up to 67% fewer rollouts than dynamic sampling.

  3. Learning as Reasoning Unfolds: Progressive Rollout Allocation for Efficient Reinforcement Learning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Allocating extra rollout budget to prompts with the highest reward variance makes GRPO training reach the same accuracy with up to 2.3x fewer rollouts.

  4. TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning

    eess.SP 2026-04 unverdicted novelty 6.0 of 10

    TimeRFT fine-tunes time-series foundation models with step-wise reward signals and difficulty-filtered data, beating supervised fine-tuning on eight benchmarks across data regimes.

  5. CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts

    cs.CL 2025-10 conditional novelty 6.0 of 10

    A consistency reward parsed by a 7B LLM, plus a two-stage refine-then-monitor pipeline and reformulated easy questions, improves MCQ-RL accuracy-with-consistency in law and medicine (58.9 vs 51.4 average Acc+).

  6. Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning

    cs.CL 2025-09 conditional novelty 6.0 of 10

    BRIDGE couples SFT and RL via bilevel optimization plus a cooperative-gain LoRA objective and reports consistent math-reasoning gains over cold-start and mixing baselines.

  7. FormaRL: Enhancing Autoformalization with no Labeled Data

    cs.AI 2025-08 conditional novelty 6.0 of 10

    A reinforcement learning framework improves autoformalization without labeled data by rewarding outputs that pass Lean syntax and LLM consistency checks.

  8. PrefixAgent: An LLM-Powered Design Framework for Efficient Prefix Adder Optimization

    cs.AR 2025-07 conditional novelty 6.0 of 10

    PrefixAgent fine-tunes an LLM on E-graph generated optimization traces to synthesize prefix adders with smaller area than prior methods and commercial tools at up to 64 bits.

  9. Test-Time Scaling with Reflective Generative Model

    cs.LG 2025-07 conditional novelty 6.0 of 10

    MetaStone-S1 combines a shared policy and self-supervised process reward head to select high-quality reasoning traces, reaching o3-mini-level scores at 32B parameters.

  10. How Far Are We from Optimal Reasoning Efficiency?

    cs.CL 2025-06 conditional novelty 6.0 of 10

    The authors define a reasoning efficiency frontier and a gap metric (REG), then train models with REO-RL to shrink the gap by at least 50% with only small accuracy losses.

  11. Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts

    cs.AI 2025-06 conditional novelty 6.0 of 10

    An online pre-rollout filter that predicts and skips zero-variance prompts from reward dynamics reduces GRPO rollout time by up to 2.4x with comparable math reasoning accuracy.

  12. SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis

    cs.LG 2025-06 conditional novelty 6.0 of 10

    SynthRL synthesizes harder, answer-preserving visual math questions from easy seed questions and reports small but mixed out-of-domain RLVR gains for Qwen2.5-VL-7B.

  13. Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs

    cs.AI 2025-05 conditional novelty 6.0 of 10

    Post-training an LLM on economic reasoning problems improves accuracy on economic benchmarks and, without game-specific training, raises its Nash equilibrium frequency and win rates in strategic games.

  14. HardTests: Synthesizing High-Quality Test Cases for LLM Coding

    cs.CL 2025-05 conditional novelty 6.0 of 10

    HardTestGen generates higher-precision, higher-recall test suites for 47,136 competitive programming problems, improving test precision by 11.3 points and recall by 17.5 points over TACO and CodeContests when judging ...

  15. SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning

    cs.RO 2025-05 conditional novelty 6.0 of 10

    SCIZOR filters suboptimal and redundant state-action pairs from robot demonstrations without human labels, improving imitation-learning policy success rates by about 15% on average.

  16. Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A 920-example distillation from DeepSeek R1 outperforms zero-RL models on the same Qwen2.5-32B base across several math and science benchmarks.

  17. Towards Revealing the Effectiveness of Small-Scale Fine-tuning in R1-style Reinforcement Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Re-distilling a model's own RL-trained policy into 1K SFT samples reproduces RL accuracy at a fraction of the compute.

  18. Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Tool-Star combines cold-start supervised fine-tuning with a multi-tool self-critic reinforcement learning algorithm and hierarchical rewards to improve LLM tool-use reasoning.

  19. The Hallucination Tax of Reinforcement Finetuning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Standard RFT sharply reduces LLM refusal on unanswerable questions, and adding 10% synthetic unanswerable math during RFT restores refusal with small accuracy losses.

  20. A Constitution-Grid Instrument for Data-Efficient RL Alignment (C-Guard)

    cs.CL 2026-07 conditional novelty 5.0 of 10

    C-Guard uses a constitution grid and a per-cell learnability score to aim RL training data, cutting over-refusal by 9.6 points while exposing a hidden 0.06 rise in adversarial under-refusal.

  21. From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A sample-centric RLVR method (LPPO), combining hint prefixes and learning-progress weighting, improves math reasoning pass@1 by 2 to 4 points over a GRPO baseline.

  22. SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning

    cs.LG 2025-06 conditional novelty 5.0 of 10

    SwS uses failures during RL training to synthesize targeted math problems, improving reasoning accuracy on eight benchmarks.

Pith tools