Pith. sign in

REVIEW 4 major objections 6 minor 80 references

What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large reasoning models put their 'thinking cues' at sentence starts; a small model can finish the sentence, cutting FLOPs by 30-55% while keeping most accuracy.

desk verdict A genuinely useful sentence-level misalignment finding and a simple decoding method, but the TFLOPs numbers omit the draft model's scoring passes and need correction. read the letter →

arxiv 2506.06998 v1 pith:OCEN2IRA submitted 2025-06-08 cs.CL cs.AIcs.LG

Ming Li , Zhengyuan Yang , Xiyao Wang , Dianqi Li , Kevin Lin , Tianyi Zhou , Lijuan Wang This is my paper
classification cs.CLcs.AIcs.LG
keywords largereasoningmodelschain-of-thoughtefficientdecodingspeculativetokenmisalignmentoverthinkingcost-qualitytrade-offFoReaL-Decoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that large reasoning models differ from ordinary instruction-tuned models at the token level in a way that can be exploited for efficient decoding: the divergence between them does not fade with longer responses, yet within each sentence the divergence is concentrated in the first few 'thinking-cue' tokens and then drops quickly. If that sentence-level front-loading is real, a large reasoning model only needs to lead the opening of each sentence while a small, cheap model completes the rest. The paper proposes FoReaL-Decoding, a training-free decoding algorithm with a stochastic gate that interpolates between a strong Leading model and a weaker Draft model. On AIME24, GPQA-Diamond, MATH500, and AMC23, it reports 30-55% lower theoretical FLOPs and up to 40% shorter chains of thought while keeping 86-100% of the large model's accuracy. The payoff would be a controllable, plug-and-play cost-quality trade-off for reasoning-heavy tasks.

What carries the argument

The load-bearing object is the Local Misalignment Diminish pattern, the sentence-level observation that misalignment between a reasoning leader and a draft model is high only at the first few token positions of each sentence and then rapidly decays. FoReaL-Decoding turns that pattern into a decoding policy: a Bernoulli gate $g_s \sim \mathrm{Bernoulli}(p)$ decides whether sentence $s$ is led; if led, the Leading model emits the first $n$ tokens; after that the Draft model runs alongside it, and the handoff is made permanent only when the two models' top-1 predictions agree for $k$ consecutive steps (the hit threshold, $k=5$ in the experiments); otherwise the leader finishes the sentence. This sentence-level likelihood rule $P_{\mathrm{CoL}}(Y_s \mid g_s)$ is what lets the method interpolate smoothly between draft-only and leader-only decoding.

What would settle it

Compute per-sentence-position misalignment curves for a new leader/draft pair on a held-out hard benchmark: if the misalignment rate does not drop sharply within the first few tokens of each sentence, the front-loading premise that FoReaL-Decoding exploits is absent. Alternatively, sweep the handoff threshold $k$ from 1 to 10 on AIME24: a sharp accuracy collapse at small $k$ would show that the method's savings depend on the top-1 agreement proxy rather than on genuine sentence-initial reasoning cues.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the token-level misalignment between a large reasoning model and a smaller or non-reasoning model is not uniform: it rebounds at the response level but diminishes sharply at the sentence level, because reasoning models put their distinctive thinking cues at sentence openings and then fall back into tokens that a weaker model would produce anyway. From that observation, the paper claims that a strong reasoning model can act as a reasoning leader that writes the first n tokens of each led sentence while a cheap draft model finishes the sentence, using k consecutive top-1 agreements as the handoff signal. In experiments across four math benchmarks, the paper reports that this FoReaL-Decoding setting reduces theoretical FLOPs by 30-55% and chain-of-thought length by up to 40% while retaining 86-100% of the leader's accuracy, and that the two control knobs produce a smooth accuracy-cost frontier on which the leader-only operating point is Pareto-dominated.

Load-bearing premise

The load-bearing premise is that the Local Misalignment Diminish pattern holds for the specific leader/draft pairs and tasks tested: the tokens carrying a reasoning model's distinctive decisions are concentrated in the first few positions of each sentence, so a weaker model can complete the rest without meaningful accuracy loss.

Editorial extensions

If this is right

  • On AIME24 and AMC23, the paper reports that every tested standalone large-reasoning-model operating point is Pareto-dominated by some FoReaL-Decoding configuration, meaning higher or equal accuracy at lower theoretical cost.
  • The method's two knobs, lead count $n$ and lead probability $p$, sweep a smooth accuracy-cost curve; the paper identifies $(n,p)=(15,0.6)$ as a sweet spot and finds that lead counts beyond 25 add little accuracy.
  • The paper reports transfer to the Qwen3 family: the 32B/1.7B pairing reaches 73.3% on AIME24 versus 76.6% for the 32B model alone at roughly half the TFLOPs.
  • In the high-efficiency configuration with a small reasoning leader and a larger instruct draft, response lengths drop substantially, and on AIME24 and AMC23 the method matches the leader's accuracy at similar or lower computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the sentence-initial front-loading pattern generalizes to other reasoning-model families and domains, the same leader-then-draft recipe could be adapted to code generation or agentic planning by retuning $(n, p, k)$.
  • Because the paper measures theoretical FLOPs, actual wall-clock latency gains on real serving stacks remain an open question; a latency benchmark with interleaved leader/draft inference would test the practical speedup.
  • FoReaL deliberately mixes two model distributions rather than preserving the leader's distribution, so it is a cost-quality dial rather than a drop-in replacement wherever exact distributional fidelity is required.
  • An adaptive version that sets the lead probability per problem based on predicted difficulty would likely extend the reported Pareto frontier beyond what fixed $(n,p)$ sweeps achieve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper performs a token-level analysis of misalignment between a large reasoning model (DeepSeek-R1-Distill-Qwen-32B) and smaller instruction-tuned, base, or reasoning models. It identifies two phenomena: a Global Misalignment Rebound, where divergence from non-reasoning models persists or grows with response length, and a Local Misalignment Diminish, where divergence is concentrated at sentence-initial tokens. Motivated by these findings, the authors propose FoReaL-Decoding, a training-free collaborative decoding method in which a strong 'Leading' model generates the first n tokens of each led sentence and a weaker 'Draft' model completes the sentence, with a stochastic gate controlling the probability p of leading a sentence and a hit-based handoff mechanism (k=5). Experiments on AIME24, GPQA-Diamond, MATH500, and AMC23 report theoretical TFLOPs reductions of 30-55% and chain-of-thought length reductions of up to 40% while retaining 86-100% of the large model's accuracy.

Significance. The main empirical finding that reasoning-critical token divergence is front-loaded at sentence beginnings is a useful and non-obvious observation that could inform future efficient decoding methods for large reasoning models. FoReaL-Decoding itself is simple, training-free, and provides a continuous cost-quality trade-off via the (n,p) controls, which is a practical advantage over methods that require post-training or auxiliary reward models. The paper also provides a clearly specified algorithm and explicit FLOPs formulas, and the Pareto analysis in Figure 4 is informative. If the efficiency accounting is corrected and the accuracy claims are made statistically robust, this could be a valuable contribution to the increasingly important area of efficient reasoning-model inference.

major comments (4)
  1. [Appendix A, Algorithm 1; Eqs. (4)-(5); Appendix B] The TFLOPs reported in Tables 1, 2, and 4 omit the Draft model's forward passes required for the hit checks. In Algorithm 1, for every led sentence with λ > n−k, the algorithm computes top-1(PD(·|c)) at each position until k consecutive matches occur, but these scoring passes are never included in the single-model FLOPs formulas of Appendix B or in the ratio-based totals of Table 2. In the high-efficiency setting (R1-Distill-Qwen-1.5B leader + Qwen2.5-7B-Instruct draft), this omission is especially serious because the 7B draft is roughly 4.7x more expensive per forward pass than the 1.5B leader; for the n=15,p=1.0 AIME24 row, including the scoring passes would move the 6.47 TFLOPs figure further away from the leader-only 2.86 TFLOPs, not closer to the claimed 30-55% reduction. The paper should either include these scoring FLOPs in the totals or explicitly state that TFLOPs counts only emitting-model compute, and then provide a separate estimate of the overhead.
  2. [Section 4.2, Tables 1 and 4] The headline claim that FoReaL-Decoding 'cuts inference cost by 30-55%' while preserving accuracy is not supported across the reported configurations. In the R1-Distill-Qwen-1.5B + Qwen2.5-7B-Instruct setting, the n=15,p=1.0 row on AIME24 increases TFLOPs from 2.86 (leader-only) to 6.47, and several other rows on GPQA-D also show increases. Even in the 32B+1.5B setting, the MATH500 n=25,p=1.0 row reduces TFLOPs by only 15% (from 4.13 to 3.50), which is below the stated 30-55% range. The abstract and conclusion present this range without specifying the model combination or operating point; the authors should report the range across the full sweep or clearly state the configurations to which the efficiency claim applies.
  3. [Section 4.2, Table 1] Accuracy values are from single greedy runs per configuration on benchmarks with small problem counts (AIME24 has 30 problems, AMC23 has 50). Differences of 3.3 to 10 accuracy points between configurations (e.g., AIME24 rows with 33.3% vs 50.0% for the 32B+1.5B setting) are within one or two standard errors, so the claims of 'same or slightly higher' accuracy than the Leader and the 86-100% retention range are not statistically established. The paper should provide multiple runs with standard deviations or confidence intervals, and adjust the accuracy-retention claims accordingly.
  4. [Section 2, Figures 1-2] The Local Misalignment Diminish phenomenon is demonstrated only for DeepSeek-R1-Distill-Qwen-32B paired with Qwen2.5-1.5B variants (base, instruct, or small reasoning). The method is then applied to R1-Distill-Qwen-1.5B as the Leader with Qwen2.5-7B-Instruct as the Draft, and to Qwen3-32B with Qwen3-1.7B/Qwen3-0.6B, without verifying that the sentence-front-loading pattern holds for those model combinations. If the pattern weakens, the hit-based handoff in Eqs. (4)-(5) may not transfer, and the accuracy retention could degrade. Please add a verification of the phenomenon for each model pair used in the experiments, or at least a discussion of robustness.
minor comments (6)
  1. [Section 4.2] The text describing the R1-Distill-Qwen-1.5B + Qwen2.5-7B-Instruct setting states that efficiencies 'are reduced to an extremely low level, even faster than directly utilizing the small reasoning models'; Table 1 shows the n=15,p=1.0 AIME24 row at 6.47 TFLOPs vs 2.86 for leader-only, so this statement holds only for low p and should be qualified.
  2. [Section 4.1] The hit threshold k is fixed at 5 for all experiments, but no sensitivity analysis is provided for this free parameter; a short ablation on k would strengthen the method's robustness.
  3. [Appendix B, Eq. (9)] The decode FLOPs formula contains an attention term 4sh that grows linearly with context length; the paper should clarify whether the reported TFLOPs already account for the growing KV-cache cost over long chain-of-thought responses, since this term can become substantial for responses of 10,000+ tokens.
  4. [Tables 1 and 4] TFLOPs values are reported to two decimal places, which implies a precision not supported by the approximate FLOPs formulas and single-run length measurements; rounding to one decimal or reporting with error bars would be more appropriate.
  5. [Table 1] The comparison with Speculative Thinking cites 'reported results' without specifying the experimental conditions (e.g., model pair, decoding configuration, or source); please include this information for reproducibility.
  6. [Section 2] The text refers to 'Figure 1 (lower)' when describing the sentence-level analysis, but the lower panel is in Figure 2; please correct this cross-reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the phenomena are empirical, the decoding method is a design response to them, and the headline accuracy/FLOPs results are measured against external benchmarks rather than derived from the method's definitions.

full rationale

The paper's central chain is: (1) measure token-level top-1 agreement between a reasoning leader and a weaker draft on math data; (2) observe that misalignment is concentrated at sentence beginnings and diminishes afterwards; (3) design FoReaL-Decoding to let the leader emit the first n tokens of a sentence and hand off after k consecutive top-1 hits; (4) measure accuracy and TFLOPs on AIME24, GPQA-Diamond, MATH500, and AMC23. Steps (1)-(2) are empirical observations made by the authors on the data, not assumptions smuggled in through citations. Step (3) uses the same top-1 agreement statistic that defined the phenomenon (Eq. (1) vs. Eq. (5)), but this is a consistent design choice, not a logical reduction: matching k=5 tokens at the handoff does not by itself force the draft to preserve full-sentence quality, so the reported 86-100% accuracy retention is an empirical result. The self-citations (e.g., [18], [28]-[35], [72]) appear in related-work and motivation contexts and are not load-bearing for the main claim. A reviewer concern that the TFLOPs estimates omit the draft model's scoring passes during the handoff check (Algorithm 1, when λ>n-k, and Appendix B's single-model formulas) would be a correctness/accounting issue, not circularity, because the efficiency numbers are empirical measurements under the paper's metric rather than consequences of the metric's definition. The presence of some post-hoc operating-point selection (e.g., the (15,0.6) sweet spot) is a reporting concern, not a circular derivation. Therefore no circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or algorithmic entities beyond the decoding policy itself. The main free parameters are the two control knobs and the fixed hit threshold. The load-bearing domain assumptions are the validity of top-1 agreement as an alignment measure and the transfer of the Local Misalignment Diminish pattern across model pairs.

free parameters (3)
  • lead count n = swept over {5,15,25,∞}; sweet spot 15
    Controls how many tokens the leading model writes at the start of each sentence. The paper's headline performance claims use specific n values selected after seeing benchmark results.
  • lead probability p = swept over {0,0.2,0.4,0.6,0.8,1.0}; sweet spot 0.6
    Stochastic gate probability. The reported '86-100% accuracy' and '30-55% FLOPs' ranges are selected from this sweep.
  • hit threshold k = fixed to 5
    Number of consecutive top-1 agreements required before handing off to the draft model. Chosen by hand; no sensitivity analysis.
assumptions (4)
  • domain assumption Top-1 greedy agreement between draft and leader models is a valid measure of token-level misalignment (from [40]).
    Section 2's analysis and Eq. (1) define alignment via argmax agreement; this inherits the assumptions of [40].
  • domain assumption The Local Misalignment Diminish pattern, measured on R1-Distill-Qwen-32B vs. small Qwen2.5 models, holds for the other model pairs and benchmarks used in the method experiments.
    The method in Section 3 is built on this pattern, but the analysis in Section 2 only demonstrates it on one leader model and three draft models.
  • domain assumption Sentence boundaries can be reliably detected with periods, question marks, exclamation marks, and newlines.
    Section 2, sentence-level analysis; sentence splitting affects the computed misalignment rates and the per-sentence gating in the method.
  • domain assumption The FLOPs formulas in Appendix B (from [13,20]) accurately account for the compute of both models in the collaborative decoding process.
    The efficiency claims in Table 1 depend on this; the appendix only gives single-model formulas and does not specify how the two models' compute is combined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding." pith.science (2026). https://pith.science/paper/OCEN2IRA

@misc{pith2026250606998,
  author       = {Pith},
  title        = {Pith review of: What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OCEN2IRA}},
  note         = {Machine review of arXiv:2506.06998}
}
read the original abstract

Large reasoning models (LRMs) achieve strong reasoning performance by emitting long chains of thought. Yet, these verbose traces slow down inference and often drift into unnecessary detail, known as the overthinking phenomenon. To better understand LRMs' behavior, we systematically analyze the token-level misalignment between reasoning and non-reasoning models. While it is expected that their primary difference lies in the stylistic "thinking cues", LRMs uniquely exhibit two pivotal, previously under-explored phenomena: a Global Misalignment Rebound, where their divergence from non-reasoning models persists or even grows as response length increases, and more critically, a Local Misalignment Diminish, where the misalignment concentrates at the "thinking cues" each sentence starts with but rapidly declines in the remaining of the sentence. Motivated by the Local Misalignment Diminish, we propose FoReaL-Decoding, a collaborative fast-slow thinking decoding method for cost-quality trade-off. In FoReaL-Decoding, a Leading model leads the first few tokens for each sentence, and then a weaker draft model completes the following tokens to the end of each sentence. FoReaL-Decoding adopts a stochastic gate to smoothly interpolate between the small and the large model. On four popular math-reasoning benchmarks (AIME24, GPQA-Diamond, MATH500, AMC23), FoReaL-Decoding reduces theoretical FLOPs by 30 to 50% and trims CoT length by up to 40%, while preserving 86 to 100% of model performance. These results establish FoReaL-Decoding as a simple, plug-and-play route to controllable cost-quality trade-offs in reasoning-centric tasks.

Figures

Figures reproduced from arXiv: 2506.06998 by the authors.

Figure 1
Figure 1. Left: An example comparing the token distribution alignment between DeepSeek-R1-Distill￾Qwen-32B and Qwen2.5-1.5B-Instruct, qualitatively showing that the misaligned tokens (colored in red) are related to thinking patterns, and probably appear at the start of sentences. Right: The WordCloud of the misaligned tokens calculated on a mix of math datasets, quantitatively showing the high-frequency misaligned tokens like… view at source ↗
Figure 2
Figure 2. Top: Response-level misalignment changes with response length. Bottom: sentence￾level misalignment changes with response length. The y-axis represents the average misalignment rate at each token position, the x-axis represents the token position within the whole response or sentence. We reveal the novel Global Misalign￾ment Rebound and Local Misalignment Diminish phenomenon that only occurs on current LRMs, shown as… view at source ↗
Figure 3
Figure 3. Effects of lead count and lead probability on AIME24 and AMC23 datasets, based on 2 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The trade-off curves between accuracy and TFLOPs. Blue markers correspond to FoReaL￾Decoding variants, red circles denote the corre￾sponding LRMs, and the dashed line is the empir￾ically computed Pareto frontier. On both bench￾marks, every LRM point is Pareto dominated…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

80 extracted references · 29 canonical work pages

  1. [1]

    L1: Controlling how long a reasoning model thinks with reinforcement learning, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning, 2025

    2025

  2. [2]

    Large language models for mathematical reasoning: Progresses and challenges

    Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. In Neele Falk, Sara Papi, and Mike Zhang, editors,Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 225–237, St. Jul...

    work page 2024

  3. [3]

    AIME 2022–2024 Validation Set, 2024

    AI-MO. AIME 2022–2024 Validation Set, 2024

    work page 2022

  4. [4]

    AMC 12 2023 Integer-Answer Validation Set, 2024

    AI-MO. AMC 12 2023 Integer-Answer Validation Set, 2024

    work page 2023

  5. [5]

    Reasoning language models: A blueprint, 2025

    Maciej Besta, Julia Barth, Eric Schreiber, Ales Kubicek, Afonso Catarino, Robert Gerstenberger, Piotr Nyczyk, Patrick Iff, Yueling Li, Sam Houliston, Tomasz Sternal, Marcin Copik, Grzegorz Kwa´sniewski, Jürgen Müller, Łukasz Flis, Hannes Eberhard, Hubert Niewiadomski, and Torsten Hoefler. Reasoning language models: A blueprint, 2025

    work page 2025

  6. [6]

    Data diversity matters for robust instruction tuning, 2023

    Alexander Bukharin and Tuo Zhao. Data diversity matters for robust instruction tuning, 2023

    work page 2023

  7. [7]

    Medusa: Simple llm inference acceleration framework with multiple decoding heads

    Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. Medusa: Simple llm inference acceleration framework with multiple decoding heads. arXiv preprint arXiv:2401.10774, 2024

    arXiv 2024

  8. [8]

    Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023

    arXiv 2023

Show all 80 references
  1. [9]

    Alpagasus: Training a better alpaca with fewer data, 2023

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training a better alpaca with fewer data, 2023

    2023

  2. [10]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models, 2025

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models, 2025

    2025

  3. [11]

    Extracting and understanding the superficial knowledge in alignment.arXiv preprint arXiv:2502.04602, 2025

    Runjin Chen, Gabriel Jacob Perin, Xuxi Chen, Xilun Chen, Yan Han, Nina ST Hirata, Junyuan Hong, and Bhavya Kailkhura. Extracting and understanding the superficial knowledge in alignment.arXiv preprint arXiv:2502.04602, 2025

    2025 arXiv

  4. [12]

    Do not think that much for 2+3=? on the overthinking of o1-like llms, 2025

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking of o1-like llms, 2025

    2025

  5. [13]

    Towards coarse-to-fine evaluation of inference efficiency for large language models.arXiv preprint arXiv:2404.11502, 2024

    Yushuo Chen, Tianyi Tang, Erge Xiang, Linjiang Li, Wayne Xin Zhao, Jing Wang, Yunpeng Chai, and Ji-Rong Wen. Towards coarse-to-fine evaluation of inference efficiency for large language models.arXiv preprint arXiv:2404.11502, 2024

    2024 arXiv

  6. [14]

    Gonzalez

    Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, Ion Stoica, Ana Klimovic, Graham Neubig, and Joseph E. Gonzalez. The danger of overthinking: Examining t...

    2025

  7. [15]

    Process reinforcement through implicit rewards, 2025

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, and Ning Ding. Process...

    2025

  8. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, and etc. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. 11

    2025

  9. [17]

    Mods: Model-oriented data selection for instruction tuning, 2023

    Qianlong Du, Chengqing Zong, and Jiajun Zhang. Mods: Model-oriented data selection for instruction tuning, 2023

    2023

  10. [18]

    Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill?arXiv preprint arXiv:2504.06514, 2025

    Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill?arXiv preprint arXiv:2504.06514, 2025

    2025 arXiv

  11. [19]

    Token-budget-aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024

    2024 arXiv

  12. [20]

    Reproduce the inference-time scaling experiment, 2024

    Xiaotian Han. Reproduce the inference-time scaling experiment, 2024

    2024

  13. [21]

    Training large language models to reason in a continuous latent space.arXiv preprint arXiv:2412.06769, 2024

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space.arXiv preprint arXiv:2412.06769, 2024

    2024 arXiv

  14. [22]

    Towards reasoning in large language models: A survey, 2023

    Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey, 2023

    2023

  15. [23]

    Collaborative decoding of critical tokens for boosting factuality of large language models.arXiv preprint arXiv:2402.17982, 2024

    Lifeng Jin, Baolin Peng, Linfeng Song, Haitao Mi, Ye Tian, and Dong Yu. Collaborative decoding of critical tokens for boosting factuality of large language models.arXiv preprint arXiv:2402.17982, 2024

    2024 arXiv

  16. [24]

    macmillan, 2011

    Daniel Kahneman.Thinking, fast and slow. macmillan, 2011

    2011

  17. [25]

    Overthink: Slowdown attacks on reasoning llms, 2025

    Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, and Eugene Bagdasarian. Overthink: Slowdown attacks on reasoning llms, 2025

    2025

  18. [26]

    Fast inference from transformers via speculative decoding

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. InInternational Conference on Machine Learning, pages 19274–19286. PMLR, 2023

    2023

  19. [27]

    Superficial safety alignment hypothesis.arXiv preprint arXiv:2410.10862, 2024

    Jianwei Li and Jung-Eun Kim. Superficial safety alignment hypothesis.arXiv preprint arXiv:2410.10862, 2024

    2024

  20. [28]

    Ruler: Improving llm controllability by rule-based data recycling.arXiv preprint arXiv:2406.15938, 2024

    Ming Li, Han Chen, Chenguang Wang, Dang Nguyen, Dianqi Li, and Tianyi Zhou. Ruler: Improving llm controllability by rule-based data recycling.arXiv preprint arXiv:2406.15938, 2024

    2024 arXiv

  21. [29]

    Selective reflection-tuning: Student-selected data recycling for LLM instruction-tuning

    Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, and Tianyi Zhou. Selective reflection-tuning: Student-selected data recycling for LLM instruction-tuning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguist...

    2024

  22. [30]

    Reflection-tuning: Recycling data for better instruction-tuning

    Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, and Tianyi Zhou. Reflection-tuning: Recycling data for better instruction-tuning. InNeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023

    2023

  23. [31]

    Mosaic-it: Free compositional data augmentation improves instruction tuning

    Ming Li, Pei Chen, Chenguang Wang, Hongyu Zhao, Yijun Liang, Yupeng Hou, Fuxiao Liu, and Tianyi Zhou. Mosaic-it: Free compositional data augmentation improves instruction tuning. arXiv preprint arXiv:2405.13326, 2024

    2024 arXiv

  24. [32]

    How instruction and reasoning data shape post- training: Data quality through the lens of layer-wise gradients.arXiv preprint arXiv:2504.10766, 2025

    Ming Li, Yanhong Li, Ziyue Li, and Tianyi Zhou. How instruction and reasoning data shape post- training: Data quality through the lens of layer-wise gradients.arXiv preprint arXiv:2504.10766, 2025

    2025 arXiv

  25. [33]

    What happened in llms layers when trained for fast vs

    Ming Li, Yanhong Li, and Tianyi Zhou. What happened in llms layers when trained for fast vs. slow thinking: A gradient perspective.arXiv preprint arXiv:2410.23743, 2024

    2024 arXiv

  26. [34]

    Superfiltering: Weak-to-strong data filtering for fast instruction-tuning

    Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the...

    2024

  27. [35]

    From quantity to quality: Boosting LLM performance with self- guided data selection for instruction tuning

    Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting LLM performance with self- guided data selection for instruction tuning. In Kevin Duh, Helena Gomez, and Steven Bethard, editors...

    2024

  28. [36]

    Contrastive decoding: Open-ended text generation as optimization.arXiv preprint arXiv:2210.15097, 2022

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization.arXiv preprint arXiv:2210.15097, 2022

    arXiv

  29. [37]

    From system 1 to system 2: A survey of reasoning large language models, 2025

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Zhijiang Guo, Le Song, and Cheng-Lin Liu. From sys...

    2025

  30. [38]

    Reward-guided speculative decoding for efficient llm reasoning.arXiv preprint arXiv:2501.19324, 2025

    Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Reward-guided speculative decoding for efficient llm reasoning.arXiv preprint arXiv:2501.19324, 2025

    2025 arXiv

  31. [39]

    Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    2023 arXiv

  32. [40]

    The unlocking spell on base llms: Rethinking alignment via in-context learning.arXiv preprint arXiv:2312.01552, 2023

    Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. The unlocking spell on base llms: Rethinking alignment via in-context learning.arXiv preprint arXiv:2312.01552, 2023

    2023 arXiv

  33. [41]

    Code- mind: A framework to challenge large language models for code reasoning, 2024

    Changshu Liu, Shizhuo Dylan Zhang, Ali Reza Ibrahimzada, and Reyhaneh Jabbarvand. Code- mind: A framework to challenge large language models for code reasoning, 2024

    2024

  34. [42]

    What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning.arXiv preprint arXiv:2312.15685, 2023

    Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning.arXiv preprint arXiv:2312.15685, 2023

    2023 arXiv

  35. [43]

    Optimizing speculative decoding for serving large language models using goodput.arXiv preprint arXiv:2406.14066, 2024

    Xiaoxuan Liu, Cade Daniel, Langxiang Hu, Woosuk Kwon, Zhuohan Li, Xiangxi Mo, Alvin Cheung, Zhijie Deng, Ion Stoica, and Hao Zhang. Optimizing speculative decoding for serving large language models using goodput.arXiv preprint arXiv:2406.14066, 2024

    2024 arXiv

  36. [44]

    Efficient inference for large reasoning models: A survey, 2025

    Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. Efficient inference for large reasoning models: A survey, 2025

    2025

  37. [45]

    O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning, 2025

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning, 2025

    2025

  38. [46]

    s1: Simple test-time scaling, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025

    2025

  39. [47]

    OpenAI o1 System Card, December 2024

    OpenAI. OpenAI o1 System Card, December 2024

    2024

  40. [48]

    A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond, 2025

    Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, Peng Li, Wei Wei, Jing Shao, Chaochao Lu, Yue Zhang, Xian- Sheng Hua, Bowen Zhou, and Yu Cheng. A survey of efficient reasoning for large reasoning models:...

    2025

  41. [49]

    Revisiting the superficial alignment hypothesis.arXiv preprint arXiv:2410.03717, 2024

    Mohit Raghavendra, Vaskar Nath, and Sean Hendryx. Revisiting the superficial alignment hypothesis.arXiv preprint arXiv:2410.03717, 2024

    2024 arXiv

  42. [50]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling, 2024. 13

    2024

  43. [51]

    The benefits of a concise chain of thought on problem- solving in large language models

    Matthew Renze and Erhan Guven. The benefits of a concise chain of thought on problem- solving in large language models. In2024 2nd International Conference on Foundation and Large Language Models (FLLM), pages 476–483. IEEE, 2024

    2024

  44. [52]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    2024

  45. [53]

    Learning to decode collaboratively with multiple language models.arXiv preprint arXiv:2403.03870, 2024

    Shannon Zejiang Shen, Hunter Lang, Bailin Wang, Yoon Kim, and David Sontag. Learning to decode collaboratively with multiple language models.arXiv preprint arXiv:2403.03870, 2024

    2024 arXiv

  46. [54]

    Efficient reasoning with hidden thinking.arXiv preprint arXiv:2501.19201, 2025

    Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. Efficient reasoning with hidden thinking.arXiv preprint arXiv:2501.19201, 2025

    2025 arXiv

  47. [55]

    Codi: Compressing chain-of-thought into continuous space via self-distillation.arXiv preprint arXiv:2502.21074, 2025

    Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thought into continuous space via self-distillation.arXiv preprint arXiv:2502.21074, 2025

    2025 arXiv

  48. [56]

    Fast best-of-n decoding via speculative rejection.arXiv preprint arXiv:2410.20290, 2024

    Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection.arXiv preprint arXiv:2410.20290, 2024

    2024 arXiv

  49. [57]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  50. [58]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024

    2024

  51. [59]

    Qwen3, April 2025

    Qwen Team. Qwen3, April 2025

    2025

  52. [60]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and etc. Llama 2: Open foundation and fine-tuned chat models, 2023

    2023

  53. [61]

    Dynamic self-consistency: Leveraging reasoning paths for efficient llm sampling.arXiv preprint arXiv:2408.17017, 2024

    Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Dynamic self-consistency: Leveraging reasoning paths for efficient llm sampling.arXiv preprint arXiv:2408.17017, 2024

    2024 arXiv

  54. [62]

    Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement.arXiv preprint arXiv:2504.07934, 2025

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement.arXiv preprint arXiv:2504.07934, 2025

    2025 arXiv

  55. [63]

    Scaling inference-time search with vision value model for improved visual comprehension.arXiv preprint arXiv:2412.03704, 2024

    Xiyao Wang, Zhengyuan Yang, Linjie Li, Hongjin Lu, Yuancheng Xu, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Scaling inference-time search with vision value model for improved visual comprehension.arXiv preprint arXiv:2412.03704, 2024

    2024 arXiv

  56. [64]

    Mementos: A comprehensive bench- mark for multimodal large language model reasoning over image sequences.arXiv preprint arXiv:2401.10529, 2024

    Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, et al. Mementos: A comprehensive bench- mark for multimodal large language model reasoning over image sequences.arXiv preprint arXiv:2401.10529, 2024

    2024 arXiv

  57. [65]

    Multimodal chain-of-thought reasoning: A comprehensive survey, 2025

    Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: A comprehensive survey, 2025

    2025

  58. [66]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023

    2023

  59. [67]

    When more is less: Understanding chain-of-thought length in llms.arXiv preprint arXiv:2502.07266, 2025

    Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in llms.arXiv preprint arXiv:2502.07266, 2025

    2025 arXiv

  60. [68]

    Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025

    2025

  61. [69]

    Evaluating mathematical reasoning beyond accuracy, 2025

    Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. Evaluating mathematical reasoning beyond accuracy, 2025. 14

    2025

  62. [70]

    Self- rewarding correction for mathematical reasoning, 2025

    Wei Xiong, Hanning Zhang, Chenlu Ye, Lichang Chen, Nan Jiang, and Tong Zhang. Self- rewarding correction for mathematical reasoning, 2025

    2025

  63. [71]

    Chain of draft: Thinking faster by writing less.arXiv preprint arXiv:2502.18600, 2025

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less.arXiv preprint arXiv:2502.18600, 2025

    2025 arXiv

  64. [72]

    A survey on knowledge distillation of large language models.ArXiv, abs/2402.13116, 2024

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models.ArXiv, abs/2402.13116, 2024

    2024 arXiv

  65. [73]

    Speculative thinking: Enhancing small-model reasoning with large model guidance at inference time, 2025

    Wang Yang, Xiang Yue, Vipin Chaudhary, and Xiaotian Han. Speculative thinking: Enhancing small-model reasoning with large model guidance at inference time, 2025

    2025

  66. [74]

    Limo: Less is more for reasoning, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025

    2025

  67. [75]

    Distilling system 2 into system 1.arXiv preprint arXiv:2407.06023, 2024

    Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1.arXiv preprint arXiv:2407.06023, 2024

    2024 arXiv

  68. [76]

    Lightthinker: Thinking step-by-step compression.arXiv preprint arXiv:2502.15589, 2025

    Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression.arXiv preprint arXiv:2502.15589, 2025

    2025

  69. [77]

    overthinking

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment.Advances in Neural Information Processing Systems, 36:55006–55021, 2023. 15 Table of Contents for Appendix A Pseudo Cod...

    2023

  70. [79]

    superficial

    provides a comprehensive token-level evidence by comparing the top-k token distributions of base models and their chat-tuned counterparts. The authors show that almost all divergence concentrates on discourse markers, politeness phrases, and safety disclaimers, while core cont...

  71. [80]

    amateur” model alongside a strong “expert

    runs a weak “amateur” model alongside a strong “expert” and selects tokens that maximize 19 their likelihood gap, sharply reducing repetition and incoherence without retraining. [23] introduces a critical-token strategy that switches to the pretrained base model whenever factu...

  72. [2024]

    Association for Computational Linguistics

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.