Pith. sign in

REVIEW 19 cited by

An Empirical Study on Eliciting and Improving R1-like Reasoning Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.04548 v1 pith:HERP3XJW submitted 2025-03-06 cs.CL

Zhipeng Chen , Yingqian Min , Beichen Zhang , Jie Chen , Jinhao Jiang , Daixuan Cheng , Wayne Xin Zhao , Zheng Liu
show 5 more authors
Xu Miao Yang Lu Lei Fang Zhongyuan Wang Ji-Rong Wen
This is my paper
classification cs.CL
keywords modelstrainingreasoningaccuracyaimeapproachbaseenhancing
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this report, we present the third technical report on the development of slow-thinking models as part of the STILL project. As the technical pathway becomes clearer, scaling RL training has become a central technique for implementing such reasoning models. We systematically experiment with and document the effects of various factors influencing RL training, conducting experiments on both base models and fine-tuned models. Specifically, we demonstrate that our RL training approach consistently improves the Qwen2.5-32B base models, enhancing both response length and test accuracy. Furthermore, we show that even when a model like DeepSeek-R1-Distill-Qwen-1.5B has already achieved a high performance level, it can be further refined through RL training, reaching an accuracy of 39.33% on AIME 2024. Beyond RL training, we also explore the use of tool manipulation, finding that it significantly boosts the reasoning performance of large reasoning models. This approach achieves a remarkable accuracy of 86.67% with greedy search on AIME 2024, underscoring its effectiveness in enhancing model capabilities. We release our resources at the STILL project website: https://github.com/RUCAIBox/Slow_Thinking_with_LLMs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RLVR Datasets and Where to Find Them: Tracing Data Lineage for Better Training Data

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    ATLAS traces RLVR data to 20 atomic sources, most datasets are variants, and DAPO++ curated with SCA improves RLVR performance while Q predicts training effectiveness.

  2. GUI-AC: Enhancing Continual Learning in GUI Agents

    cs.CV 2026-06 conditional novelty 6.0 of 10

    Grounding-certainty-guided Adaptive Advantage and Dynamic Clipping improve continual GUI grounding over RFT baselines on ScreenSpot-V1/V2/Pro.

  3. SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    SAGE reshapes the reverse-KL anchor via guide function q(x,y) for controllable empirical support expansion, yielding gains in both pass@1 and pass@k on math reasoning benchmarks.

  4. PruneTIR: Inference-Time Tool Call Pruning for Effective yet Efficient Tool-Integrated Reasoning

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    PruneTIR prunes erroneous tool-call trajectories during LLM inference via three trigger-based components to raise Pass@1 accuracy and efficiency while shortening context.

  5. TimelineReasoner: Advancing Timeline Summarization with Large Reasoning Models

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    TimelineReasoner applies large reasoning models in a Global Cognition plus Detail Exploration loop to produce more accurate, complete, and coherent timelines from news than prior LLM-based methods.

  6. Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework

    cs.AI 2025-09 conditional novelty 6.0 of 10

    Sticker-TTS uses three collaborating models to distill and reuse compact summaries (stickers) of past reasoning attempts, improving math benchmark accuracy at a claimed equivalent inference cost.

  7. WebThinker: Empowering Large Reasoning Models with Deep Research Capability

    cs.CL 2025-04 unverdicted novelty 6.0 of 10

    WebThinker equips large reasoning models with autonomous web exploration and interleaved reasoning-drafting via a Deep Web Explorer and RL-based DPO training, yielding gains on GPQA, GAIA, and report-generation benchmarks.

  8. ReTool: Reinforcement Learning for Strategic Tool Use in LLMs

    cs.CL 2025-04 unverdicted novelty 6.0 of 10

    ReTool uses outcome-driven RL to train 32B LLMs to dynamically use code tools during reasoning, reaching 72.5% accuracy on AIME and surpassing o1-preview.

  9. OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles

    cs.CV 2025-03 conditional novelty 6.0 of 10

    Iterative SFT-RL cycles enable a 7B LVLM to develop sophisticated visual chain-of-thought reasoning and improve performance on math and general reasoning benchmarks.

  10. DAPO: An Open-Source LLM Reinforcement Learning System at Scale

    cs.LG 2025-03 conditional novelty 6.0 of 10

    DAPO introduces decoupled clipping and dynamic sampling for LLM RL, achieving 50 on AIME 2024 with Qwen2.5-32B while fully open-sourcing code, data, and the verl-based training system.

  11. Trust Region On-Policy Distillation

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    TrOPD stabilizes on-policy distillation for LLMs with trust-region learning, outlier estimation, and off-policy guidance, outperforming prior OPD methods on reasoning and code benchmarks.

  12. How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors

    cs.AI 2026-05 unverdicted novelty 5.0 of 10

    IMAX trains soft prefixes with an InfoMax reward to drive diverse exploration in RLVR, yielding up to 11.60% gains in Pass@4 over standard RLVR across model scales.

  13. EEPO: Exploration-Enhanced Policy Optimization via Sample-Then-Forget

    cs.CL 2025-10 unverdicted novelty 5.0 of 10

    EEPO uses sample-then-forget rollouts with adaptive unlearning to boost exploration in RLVR, delivering relative gains of 24.3% on Qwen2.5-3B, 33.0% on Llama3.2-3B-Instruct, and 10.4% on Qwen3-8B-Base over GRPO across...

  14. Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards

    cs.LG 2025-09 conditional novelty 5.0 of 10

    The paper identifies confounds in RLVR evaluations that inflate apparent gains and proposes a minimum standard for budget-matched, contamination-aware assessment with calibration tracking.

  15. Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models

    cs.LG 2025-08 conditional novelty 5.0 of 10

    Using Pass@k as an RLVR reward, with bootstrap sampling and an analytical advantage formula, improves exploration and later Pass@1 performance of reasoning LLMs.

  16. Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning

    cs.LG 2025-04 unverdicted novelty 5.0 of 10

    PODS applies max-variance down-sampling to GRPO rollouts in LLM RLVR, delivering at least 1.7x faster training to peak test accuracy on reasoning benchmarks.

  17. GUI-AC: Enhancing Continual Learning in GUI Agents

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    GUI-AC stabilizes RFT for non-stationary GUI data by down-weighting noisy advantages and relaxing clipping bounds via a grounding certainty term.

  18. Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thought

    cs.AI 2025-09 reject novelty 4.0 of 10

    LLM chain-of-thought length converges to an optimum because of an underfitting-overfitting tradeoff, formalized in a continuous reasoning-space framework and tested with RL.

  19. A Survey of Reinforcement Learning for Large Reasoning Models

    cs.CL 2025-09 accept novelty 3.0 of 10

    A survey compiling RL methods, challenges, data resources, and applications for enhancing reasoning in large language models and large reasoning models since DeepSeek-R1.

Pith tools