Pith. sign in

REVIEW 4 major objections 7 minor 25 cited by

Thinkless: LLM Learns When to Think

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A reinforcement-learning policy can teach a reasoning language model when to think, cutting long-chain usage by 50–90%.

desk verdict A real RL training fix for hybrid reasoning—worth refereeing, but the headline Think% metric needs per-mode token stats and a few missing hyperparameters. read the letter →

arxiv 2505.13379 v2 pith:TK3MBQIO submitted 2025-05-19 cs.CL cs.AI

Gongfan Fang , Xinyin Ma , Xinchao Wang This is my paper · ORCID
classification cs.CLcs.AI
keywords reasoninglanguagemodelshybridreinforcementlearningGRPOcontroltokensinferenceefficiencymodeselectionchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a reasoning language model can decide, for each input, whether to think at length or answer immediately, instead of always emitting a long chain of thought. It proposes Thinkless, a two-stage scheme: first distill paired long and short responses so the model can produce either style, then optimize a decoupled reinforcement-learning objective, DeGRPO, that treats choosing the reasoning mode as a separate learning signal from improving answer accuracy. The central claim is that this lets a small reasoning model route easy questions to short answers, cutting long-chain thinking by 50–90% on standard math benchmarks while keeping the single-answer correctness (Pass@1) within a few points of the always-think baseline. If correct, this matters because the dominant cost of reasoning models is token generation; adaptive thinking makes that cost depend on the query instead of being fixed for every query.

What carries the argument

The load-bearing object is the DeGRPO objective, a decoupled variant of group-relative policy optimization. The paper splits the sequence loss into a control-token term $\alpha \mathcal{L}_{i,0}$ and a response term $\frac{1}{T_i}\sum_{t=1}^{T_i}\mathcal{L}_{i,t}$, removing the length normalization that makes vanilla GRPO under-update the <think> token. The reward is deliberately simple: a correct short answer gets $1.0$, a correct long answer gets $1-\gamma$, and any wrong answer gets $-1$; advantage is computed against the group mean, which is robust to mixed-difficulty batches. This decomposition is what lets the same model improve its short-form answer quality while learning which queries deserve long reasoning.

What would settle it

Retrain the same pipeline with the short-correct reward margin $\gamma$ set to two extremes, for example $0.01$ and $0.99$, and measure long-thinking usage and Pass@1 on MATH-500 and AIME 2024. If the 50–90% reduction and the near-baseline accuracy do not both survive across this sweep, the headline efficiency gain is tied to the unreported reward margin rather than to DeGRPO itself.

Watch

Extended reading notes

Core claim

The paper's central claim is that a reasoning LLM can learn when to think from reward feedback alone, without a hand-built router or difficulty heuristic. On MATH-500, Minerva Algebra, and GSM8K, the trained model reduces long-chain thinking usage by roughly 50–90% while keeping Pass@1 within a few points of the original reasoning model; on AIME 2024, where nearly every problem is hard, it uses thinking mode on all queries and retains essentially the same accuracy. The decision is carried by a control token emitted as the first output token—<short> or <think>—so switching modes is a single-token decision. DeGRPO normalizes the control-token and response-token losses separately, which prevents the mode signal from being drowned out by hundreds of response tokens and stops the collapse to a single mode that the paper observes in vanilla GRPO.

Load-bearing premise

The reported savings rest on a reward that pays a correct short answer more than a correct long answer by a margin the paper never reports; if that margin is too large, the model may stop thinking too often and lose accuracy, and if it is too small, it may keep thinking and save little.

Editorial extensions

If this is right

  • Simple queries dominate math benchmarks such as GSM8K, so the same model can serve most requests with a fraction of its original tokens while accuracy stays close to the always-think policy.
  • DeGRPO removes the early collapse to the short mode that vanilla GRPO shows, so both response styles remain trainable enough for the policy to improve short answers instead of merely avoiding them.
  • The probability the model assigns to <think> acts as a learned difficulty signal, rising on multi-condition problems and falling on routine arithmetic, which could be reused as a soft routability score.
  • On hard benchmarks like AIME, the model chooses thinking mode on all tested queries, so the efficiency savings do not come at the expense of the hardest questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct follow-up is to sweep the unreported reward margin $\gamma$ between correct short and correct long answers; the claimed 50–90% saving is one operating point on an accuracy-versus-token frontier, not a fixed property of the algorithm.
  • The same control-token decoupling should transfer to other binary output-mode choices beyond math—for example, whether to call a tool, cite sources, or answer in a formal style—wherever one mode is longer and more expensive than the other.
  • If the warm-up distillation were replaced by a weaker short-response teacher, the RL stage would have to improve response quality and mode selection simultaneously, which would likely move the reported trade-off; this makes the quality of the paired distillation a hidden precondition for the headline numbers.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Thinkless, a two-stage framework for training an LLM to decide between a concise <short> response and a detailed <think> response. The method first distills paired long/short responses into a base reasoning model via SFT, then applies a proposed Decoupled Group Relative Policy Optimization (DeGRPO) that separates the loss on the control token from the loss on response tokens. Experiments on AIME, Minerva Algebra, MATH-500, and GSM8K report Pass@1 close to the base reasoning model while reducing the fraction of <think> queries by roughly 50–90% on the easier benchmarks. The paper also analyzes training dynamics, showing that vanilla GRPO collapses to a single mode while DeGRPO exhibits a U-shaped learning curve.

Significance. If the central claim is verified, the work is a practical and timely contribution to efficient reasoning: it shows that a small reasoning model can learn a difficulty- and capability-dependent routing policy without an external router, and the decoupled objective addresses a real mode-collapse problem in applying GRPO to hybrid responses. The release of code is a strength, and the U-shape finding plus the calibration case study provide falsifiable qualitative predictions. However, because the reward is defined over the control token rather than actual output length, the headline efficiency figures need additional verification before the significance can be fully assessed.

major comments (4)
  1. [§3.2, Eq. (4) and reward definition] The reward function r(a,y*,c) depends on the control token c and answer correctness only, not on the actual number of tokens generated. Consequently, after RL the policy can emit <short> and still generate a long chain-of-thought, receiving the higher reward 1.0 whenever the extracted answer is correct. The headline "Think%" therefore measures the self-reported mode, not actual computation. While the total #Tokens reported in Table 1 do decrease, per-mode token statistics (e.g., average token count for <short> and <think> trajectories) are not reported, so we cannot verify that short-mode responses remain concise after RL. If many <short> trajectories remain verbose, the 50–90% reduction in long-chain thinking is overstated and the accuracy/efficiency trade-off is misrepresented. Please report per-mode lengths and consider rewarding actual brevity (e.g., a length term) to enforce conciseness.
  2. [§3.2, reward design] The preference margin γ in the reward r(a,y*,c) = 1.0 (short, correct), 1−γ (think, correct), −1 (wrong) is never specified; the paper only states 1>γ>0. This parameter determines the relative pressure toward short responses and directly controls the trade-off between mode reduction and accuracy. Without its value and a sensitivity analysis, the reported results are not reproducible, and the stability of the method under different γ is unknown. Please state γ and include an ablation over γ.
  3. [§4.2, Table 1] All results in Table 1 appear to be from a single run with no standard deviations, confidence intervals, or significance tests. Several comparisons are close (e.g., Minerva Algebra Pass@1: 0.9459 for Thinkless vs 0.9577 for the base; GSM8K: 0.8418 vs 0.8347), so without error bars we cannot be confident that the "within a few points" accuracy claim holds robustly. Additionally, evaluation is restricted to mathematics benchmarks; the central claim of "learning when to think" would be strengthened by results on other domains (e.g., science, code, agentic tasks). This is a moderate but important gap for the reported performance figures.
  4. [§4.2, paragraph after Table 1] The statement "we found that the RL will also compress the length of long responses" is not supported by the described training objective. Since the reward is identical for any correct <think> response regardless of its length, the policy gradient provides no direct incentive to shorten responses within the think mode; it only incentivizes selecting <short> mode. If the authors intend this claim, they should provide evidence (e.g., per-mode token counts) and explain the mechanism.
minor comments (7)
  1. [§4.1] Typo: "direcytly" should be "directly".
  2. [§2, last paragraph] The phrase "the its difficulty" should be "its difficulty".
  3. [§1, second paragraph] The reference to "as illustrated in Figure 3" appears to be a mistake; the relevant figure in the introduction seems to be Figure 1.
  4. [§4.2, Finding 1] The stated range "50%–90%" is slightly inconsistent with Table 1: on MATH-500 the Think% is 51.56%, corresponding to a reduction of about 48.4% from a hypothetical 100% baseline, which is marginally below 50%. Please clarify the exact calculation or adjust the range.
  5. [§3.2, first paragraph after Eq. (3)] The phrase "mode-mode imbalance" is confusing; it likely should be "mode-short imbalance" or "think-short imbalance" to match the later terminology.
  6. [Figure 4] The legend entries "= 0.001" and "= 0.5" are ambiguous; they should explicitly refer to the control-token weight α and the axis should be labeled accordingly.
  7. [References] Reference [37] "Aime problem set 1983-2024" is not a standard citable source; a proper dataset reference or URL would be more appropriate.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: DeGRPO's reward preference for correct <short> answers is the training objective, not a hidden equivalence, and the accuracy/routing results are independent empirical outcomes.

full rationale

Thinkless is an empirical RL paper: the central claims are evaluation outcomes of optimizing DeGRPO on DeepScaleR and measuring on external benchmarks (MATH-500, Minerva Algebra, GSM8K, AIME), not quantities derived from the inputs. The reward design in Section 3.2 does explicitly assign 1.0 to a correct <short> response and 1−γ to a correct <think> response, so an increase in short-mode usage is aligned with the training signal; however, the reward encodes only a preference, not a guarantee, and the magnitude of the Think% reduction, the preservation of Pass@1, the per-difficulty routing behavior, and the U-shaped learning dynamics are not contained in the reward definition. Those outcomes depend on optimization and generalization and are therefore independent evidence. The unreported value of γ and the fact that the reward is keyed to the control token rather than actual response length are robustness/measurement concerns, not circularity. The paper's self-citations ([11] survey, [26] CoT-Valve baseline) are contextual or comparative and are not load-bearing for the central claim; no uniqueness theorem or ansatz is imported from prior author work. Overall, the derivation chain is self-contained: an objective is specified, a policy is trained, and external benchmarks are reported.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The ledger contains two hand-picked scalars (gamma and alpha) plus three background assumptions about reward extraction, distillation quality, and GRPO. No new physical or architectural entities are introduced.

free parameters (2)
  • gamma = not reported
    Reward margin preferring correct short answers over correct long answers; set by hand but value never stated in the paper.
  • alpha = 1/1000
    Weight for control token loss in DeGRPO; chosen by hand for stable training.
assumptions (3)
  • domain assumption The Extract-Answer function reliably identifies the final answer in both short and long generations.
    Reward is based on exact match between extracted answer and ground truth; incorrect extraction adds noise.
  • domain assumption The paired distillation data (long from R1-671B, short from Qwen2.5-Math-1.5B-Instruct) gives a balanced initialization for both control tokens.
    SFT on this paired corpus must link each token to its style with comparable quality, otherwise RL starts from a biased policy.
  • standard math GRPO with relative advantage r - mean(r) is a valid and stable optimization target for this RL problem.
    The paper uses the standard GRPO objective from prior work, so it relies on that framework's validity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Thinkless: LLM Learns When to Think." pith.science (2026). https://pith.science/paper/TK3MBQIO

@misc{pith2026250513379,
  author       = {Pith},
  title        = {Pith review of: Thinkless: LLM Learns When to Think},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TK3MBQIO}},
  note         = {Machine review of arXiv:2505.13379}
}
read the original abstract

Reasoning Language Models, capable of extended chain-of-thought reasoning, have demonstrated remarkable performance on tasks requiring complex logical inference. However, applying elaborate reasoning for all queries often results in substantial computational inefficiencies, particularly when many problems admit straightforward solutions. This motivates an open question: Can LLMs learn when to think? To answer this, we propose Thinkless, a learnable framework that empowers an LLM to adaptively select between short-form and long-form reasoning, based on both task complexity and the model's ability. Thinkless is trained under a reinforcement learning paradigm and employs two control tokens, <short> for concise responses and <think> for detailed reasoning. At the core of our method is a Decoupled Group Relative Policy Optimization (DeGRPO) algorithm, which decomposes the learning objective of hybrid reasoning into two components: (1) a control token loss that governs the selection of the reasoning mode, and (2) a response loss that improves the accuracy of the generated answers. This decoupled formulation enables fine-grained control over the contributions of each objective, stabilizing training and effectively preventing collapse observed in vanilla GRPO. Empirically, on several benchmarks such as Minerva Algebra, MATH-500, and GSM8K, Thinkless is able to reduce the usage of long-chain thinking by 50% - 90%, significantly improving the efficiency of Reasoning Language Models. The code is available at https://github.com/VainF/Thinkless

Figures

Figures reproduced from arXiv: 2505.13379 by the authors.

Figure 1
Figure 1. Thinkless learns a hybrid LLM capable of adaptively selecting between thinking and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. ThinkLess trains a hybrid model that adaptively selects reasoning modes based on task [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Policy–training comparison between vanilla GRPO and decoupled GRPO. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: A large token loss co￾efficient α accelerates the shift in reasoning behavior, leading to the rapid emergence of all-correct short-mode samples. The Influence of Decoupling. Building upon the above anal￾ysis, we further visualize the effect of decoupling on model behav…
Figure 5
Figure 5. Figure 5: Distribution of the model’s probability of emitting [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 25 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning

    cs.CL 2026-08 conditional novelty 7.0 of 10

    Skill entropy, a reference-model-based measure of skill-switching difficulty, calibrates a new cross-skill benchmark and serves as an RL reward, more than doubling small models' scores.

  2. AdaThinkV: Adaptive Thinking for Token-Efficient Video Reasoning

    cs.CV 2026-08 conditional novelty 7.0 of 10

    A video reasoning model learns per question whether to reason aloud or answer directly, improving accuracy by about 3 points over the best adaptive baseline while using about 23% fewer output tokens.

  3. ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Per-step retrieval of solved exemplars injected into the reasoning trace improves test-time scaling accuracy, with up to 13.4 absolute points gained on AIME 2025.

  4. When to Plan: Learning to Select Between Reactive Control and Deliberative Planning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    An RL-trained meta-policy that uses ensemble uncertainty to choose between a cheap reactive policy and costly planning reaches goals faster than fixed baselines and adapts as the reactive policy improves.

  5. Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM Reasoning

    cs.LG 2026-02 conditional novelty 6.0 of 10

    CEEH selectively applies entropy regularization to hard questions and a shortest-correct-length penalty to easy ones, reducing reasoning length while preserving accuracy across six math benchmarks.

  6. Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    Extra-CoT trains a semantic compressor on math CoT data, applies mixed-ratio SFT, and uses CHRPO reinforcement learning to achieve over 73% token reduction on MATH-500 with 0.6% accuracy gain on Qwen3-1.7B.

  7. ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Multi-question prompts elicit shorter chain-of-thought traces, and fine-tuning on those traces transfers the compression to single-question reasoning.

  8. Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers

    cs.CL 2026-01 conditional novelty 6.0 of 10

    A prompt combining the 'Okay' reasoning cue with the '</think>\n\n' no-think cue gives LLMs an intermediate reasoning budget without training, and also speeds up RL fine-tuning.

  9. Probing the Difficulty Perception Mechanism of Large Language Models

    cs.CL 2025-10 conditional novelty 6.0 of 10

    LLMs linearly encode math-problem difficulty in their final-token representations, and specific final-layer attention heads are specialized for easy vs hard problems.

  10. AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification

    cs.CL 2025-09 conditional novelty 6.0 of 10

    An entropy-guided reward that shortens answers to easy, confident questions and extends reasoning on hard ones reduces medical QA tokens by 4.7x to 6.4x with minimal accuracy loss.

  11. Verifying Meta-Awareness via Predictive Rewards in Reasoning Models

    cs.LG 2025-09 conditional novelty 6.0 of 10

    Rewarding reasoning models for accurately predicting their own rollout length, pass-rate, and math notions improves math benchmark accuracy and speeds up GRPO training.

  12. Hierarchical Budget Policy Optimization for Adaptive Reasoning

    cs.AI 2025-07 conditional novelty 6.0 of 10

    Training reasoning models with hierarchical token budgets and budget-aware rewards produces up to 60.6% token reduction with no accuracy loss, and benchmark-level evidence of length adaptation.

  13. LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization

    cs.AI 2025-07 conditional novelty 6.0 of 10

    LAPO is a two-stage RL method that reduces reasoning token usage by up to 40.9% while slightly improving accuracy, by learning per-problem length targets from the model's own correct answers and prompting the model to...

  14. SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control

    cs.AI 2025-07 conditional novelty 6.0 of 10

    SmartThinker shortens AI reasoning chains by estimating the importance of each step and penalizing the unimportant ones, preserving accuracy while using far fewer tokens.

  15. Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Suppressing 'thinking tokens' in a 1.5B reasoning model preserves accuracy while cutting tokens, and the proposed DuP-PO RL method improves both accuracy and efficiency over GRPO.

  16. How Far Are We from Optimal Reasoning Efficiency?

    cs.CL 2025-06 conditional novelty 6.0 of 10

    The authors define a reasoning efficiency frontier and a gap metric (REG), then train models with REO-RL to shrink the gap by at least 50% with only small accuracy losses.

  17. VeriThinker: Learning to Verify Makes Reasoning Model Efficient

    cs.LG 2025-05 conditional novelty 6.0 of 10

    VeriThinker shows that fine-tuning a reasoning model only on a solution-verification task reduces chain-of-thought length on MATH500 and AIME by 20-45% while preserving or slightly improving accuracy.

  18. MixReasoning: Switching Modes to Think

    cs.AI 2025-10 conditional novelty 5.0 of 10

    A reasoning model can cut token usage by roughly a third to a half without losing accuracy if it switches to brief output on low-uncertainty steps and expands only at high-uncertainty forks.

  19. Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards

    cs.LG 2025-09 conditional novelty 5.0 of 10

    The paper identifies confounds in RLVR evaluations that inflate apparent gains and proposes a minimum standard for budget-matched, contamination-aware assessment with calibration tracking.

  20. Implicit Reasoning in Large Language Models: A Comprehensive Survey

    cs.CL 2025-09 conditional novelty 5.0 of 10

    A survey organizing implicit (silent) reasoning in LLMs into three execution paradigms, plus evidence, benchmarks, and challenges.

  21. Think in Blocks: Adaptive Reasoning from Direct Response to Deep Reasoning

    cs.AI 2025-08 unverdicted novelty 5.0 of 10

    A training framework that makes LLMs predict an integer reasoning budget as a number of blocks and adapt reasoning depth to task difficulty.

  22. KAT-V1: Kwai-AutoThink Technical Report

    cs.CL 2025-07 conditional novelty 5.0 of 10

    KAT-V1-40B is a 40B language model that switches between deep reasoning and direct answering per query, reporting matching-or-better benchmark scores with lower token use.

  23. Schema-R1: A reasoning training approach for schema linking in Text-to-SQL Task

    cs.AI 2025-06 conditional novelty 4.0 of 10

    Schema-R1 combines cold-start SFT on 200 CoT samples with GRPO rule-based RL, reporting table and column filter accuracy gains of 10 percentage points or more over a fine-tuned baseline on Spider-dev.

  24. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

  25. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

44 extracted references · 5 canonical work pages · cited by 25 Pith papers

  1. [1]

    L1: Controlling how long a reasoning model thinks with reinforcement learning

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697, 2025

    arXiv 2025

  2. [2]

    Claude 3.7 Sonnet

    Anthropic. Claude 3.7 Sonnet. https://www.anthropic.com/claude/sonnet, 2025. Accessed: 2025-05-10

    work page 2025

  3. [3]

    Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching

    Simon A Aytes, Jinheon Baek, and Sung Ju Hwang. Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching. arXiv preprint arXiv:2503.05179, 2025

    arXiv 2025

  4. [4]

    Llama-nemotron: Efficient reasoning models, 2025

    Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp, Ehud Karpas, Ran Zilberstein, Jiaqi Zeng, Soumye Singhal, Alexander Bukharin, Yian Zhang, Tugrul Konuk, Gerald Shen, Ameya Sunil Mahabaleshwarkar, Bilal Kartal, Yoshi Suhara, Olivier Delalleau, Ziji...

    work page 2025

  5. [5]

    Research: Learning to reason with search for llms via reinforcement learning

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z Pan, Wen Zhang, Huajun Chen, Fan Yang, et al. Research: Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470, 2025

    arXiv 2025

  6. [6]

    Distilling reasoning ability from large language models with adaptive thinking

    Xiaoshu Chen, Sihang Zhou, Ke Liang, and Xinwang Liu. Distilling reasoning ability from large language models with adaptive thinking. arXiv preprint arXiv:2404.09170, 2024

    arXiv 2024

  7. [7]

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187, 2024

    arXiv 2024

  8. [8]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

Show all 44 references
  1. [9]

    The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks

    Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, et al. The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks. arXiv preprint arXiv:2502.08235, 2025

    2025 arXiv

  2. [10]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

  3. [11]

    Efficient reasoning models: A survey

    Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. Efficient reasoning models: A survey. arXiv preprint arXiv:2504.10903, 2025

    2025

  4. [12]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  5. [13]

    Token-budget- aware llm reasoning

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget- aware llm reasoning. arXiv preprint arXiv:2412.18547, 2024

    2024 arXiv

  6. [14]

    Dna bench: When silence is smarter–benchmarking over-reasoning in reasoning llms

    Masoud Hashemi, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudhan, Jishnu Sethumadhavan Nair, Aman Tiwari, and Vikas Yadav. Dna bench: When silence is smarter–benchmarking over-reasoning in reasoning llms. arXiv preprint arXiv:2503.15793, 2025

    2025 arXiv

  7. [15]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

    2021

  8. [16]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 10

    2015 arXiv

  9. [17]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Hel- yar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  10. [18]

    C3ot: Generating shorter chain-of-thought without compromising effectiveness

    Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter chain-of-thought without compromising effectiveness. arXiv preprint arXiv:2412.11664, 2024

    2024 arXiv

  11. [19]

    Mixed distillation helps smaller language model better reasoning

    Chenglin Li, Qianglong Chen, Liangyue Li, Caiyu Wang, Yicheng Li, Zulong Chen, and Yin Zhang. Mixed distillation helps smaller language model better reasoning. arXiv preprint arXiv:2312.10730, 2023

    2023 arXiv

  12. [20]

    Small models struggle to learn from strong reasoners

    Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubra- manian, and Radha Poovendran. Small models struggle to learn from strong reasoners. arXiv preprint arXiv:2502.12143, 2025

    2025

  13. [21]

    Reward-guided speculative decoding for efficient llm reasoning

    Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Reward-guided speculative decoding for efficient llm reasoning. arXiv preprint arXiv:2501.19324, 2025

    2025 arXiv

  14. [22]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023

    2023 arXiv

  15. [23]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  16. [24]

    O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570, 2025

    2025 arXiv

  17. [25]

    Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, et al. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog, 2025

    2025

  18. [26]

    Cot-valve: Length- compressible chain-of-thought tuning

    Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length- compressible chain-of-thought tuning. arXiv preprint arXiv:2502.09601, 2025

    2025 arXiv

  19. [27]

    Teaching small language models to reason

    Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. Teaching small language models to reason. arXiv preprint arXiv:2212.08410, 2022

    2022 arXiv

  20. [28]

    Routellm: Learning to route llms with preference data

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route llms with preference data. arXiv preprint arXiv:2406.18665, 2024

    2024 arXiv

  21. [29]

    Reasoning with latent thoughts: On the power of looped transformers

    Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J Reddi. Reasoning with latent thoughts: On the power of looped transformers. 2025

    2025

  22. [30]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  23. [31]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  24. [32]

    Megatron-lm: Training multi-billion parameter language models using model parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019

    1909 arXiv

  25. [33]

    Towards reasoning ability of small language models

    Gaurav Srivastava, Shuxiang Cao, and Xuan Wang. Towards reasoning ability of small language models. arXiv preprint arXiv:2502.11569, 2025

    2025

  26. [34]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  27. [35]

    Open thoughts, January 2025

    Open Thoughts Team. Open thoughts, January 2025

    2025

  28. [36]

    Qwen3, April 2025

    Qwen Team. Qwen3, April 2025

    2025

  29. [37]

    Aime problem set 1983-2024, 2023

    Hemish Veeraboina. Aime problem set 1983-2024, 2023

    1983

  30. [38]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  31. [39]

    Agentic reasoning: Reasoning llms with tools for the deep research

    Junde Wu, Jiayuan Zhu, and Yuyuan Liu. Agentic reasoning: Reasoning llms with tools for the deep research. arXiv preprint arXiv:2502.04644, 2025. 11

    2025 arXiv

  32. [40]

    Chain of draft: Thinking faster by writing less

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600, 2025

    2025 arXiv

  33. [41]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  34. [42]

    Distilling system 2 into system 1

    Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023, 2024

    2024 arXiv

  35. [43]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503.18892, 2025

    2025 arXiv

  36. [44]

    Improving mathematical reasoning capabilities of small language models via feedback-driven distillation

    Xunyu Zhu, Jian Li, Can Ma, and Weiping Wang. Improving mathematical reasoning capabilities of small language models via feedback-driven distillation. arXiv preprint arXiv:2411.14698, 2024. 12

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.