Pith. sign in

REVIEW 4 major objections 6 minor 37 references

This paper argues that large reasoning models, which look excellent on static benchmarks, can lose up to 60% accuracy when interrupted or when the problem changes while they are thinking.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:03 UTC pith:MDV73Y2D

load-bearing objection A genuinely new dynamic-evaluation suite for reasoning models, with named failure modes that replicate the right worries; the headline drop is real but the update-driven magnitudes lean on unvalidated data and an unvalidated classifier. the 4 major comments →

arxiv 2510.11713 v4 pith:MDV73Y2D submitted 2025-10-13 cs.CL cs.LG

Are Large Reasoning Models Interruptible?

Tsung-Han Wu , Mihran Miroyan , David M. Chan , Trevor Darrell , Narges Norouzi , Joseph E. Gonzalez This is my paper
classification cs.CL cs.LG
keywords large reasoning modelsinterruptibilitydynamic contextreasoning leakageself-doubtpanicfrozen world assumptionLLM evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large reasoning models are usually tested in a 'frozen world': the problem is handed over once, the model thinks for as long as it wants, and the environment does not change. This paper argues that assumption is wrong for real use—users interrupt, ask for faster answers, and change the problem while the model is still reasoning—and builds an evaluation suite to test what happens. Across math and programming problems, the paper finds that static accuracy overstates how well these models cope: accuracy can fall by up to 60% when updated information arrives late in the reasoning trace. It names three recurring failure modes—reasoning leakage (continued thinking smuggled into the answer), panic (abandoning reasoning under speedup pressure), and self-doubt (refusing to trust a valid update)—and shows that a short 'verified update' prompt helps on easy math but not on hard problems. The paper's point is that robust interruptibility is something to be designed and measured, not an automatic property of a strong reasoner.

Core claim

The central discovery is that today's reasoning models are only superficially robust to being interrupted. When a hard stop is forced, they behave approximately like anytime algorithms—later interrupts yield better answers—but the answer segment can absorb the unfinished reasoning, sometimes growing ten times longer and hiding real compute. When asked to speed up, models can panic and shut down reasoning almost immediately, producing significantly worse answers on hard tasks. When the problem itself changes mid-reasoning, especially late, accuracy drops by up to 60%, and about 80% of the mistakes under updates are classified as self-doubt: the model questions the new information and keeps so

What carries the argument

The machinery is a two-stage inference protocol. The model first produces a reasoning trace r up to a chosen point X, measured as a fraction of the full trace; then interruption tokens i are inserted and the model is asked to continue and produce an answer. The paper defines the interruption-conditioned accuracy A_i(X) = Pr[a' = a* | X,i], where a* is the ground truth for the (possibly updated) problem, and the notion of a necessary update: an update for which a*(q) ≠ a*(q,u). This controlled setup is what lets the authors vary the timing of an interruption, the form of the interrupt signal (end-thinking, force-answer, speedup, or a mid-reasoning update), and the resulting answer length, and

Load-bearing premise

The paper's central claim rests on the data-quality premise that every manually verified update really is necessary—the corrected problem genuinely has a different right answer—and unambiguous; if a substantial share of updates are malformed or not actually answer-changing, the measured accuracy drops would not cleanly measure failed adaptation.

What would settle it

Use exact solvers or verifiers to check each problem in the public update dataset: for every 'necessary update,' confirm a*(q) ≠ a*(q,u) by solving both versions independently. If many updates do not change the ground truth, or if injecting updates that are known not to change the answer produces the same late-stage accuracy collapse, then the 60% drop is an artifact of the update set rather than evidence that models cannot adapt.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Static evaluation scores should be treated as upper bounds for interactive use; a model that looks excellent on a benchmark may still be unsafe to deploy where users interrupt or edit the task.
  • Any 'thinking vs. accuracy' analysis that ignores the answer region undercounts compute, because interrupted models can leak a long chain of thought into the answer; measuring total output length changes the efficiency comparison.
  • Speedup prompts on hard tasks can backfire: instead of compressing reasoning, models sometimes abandon it and return wrong answers, with accuracy drops of up to 30% on challenging problems.
  • The timing of new information is a first-order variable: late updates cost up to 60% accuracy, so systems that inject updates late into a long-running reasoning trace should expect sharp degradation.
  • Prompt-level reassurance that an update is verified partially recovers accuracy on easier math but not on harder math or coding, so interface tricks are not a substitute for making models update-tolerant.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the paper is right, efficiency studies that report only token counts inside explicit thinking blocks are mismeasuring cost; a re-run that counts answer-region tokens after early interrupts would likely change which models look efficient.
  • A concrete system-level extension: rather than injecting updates into a running trace, a serving layer could pause the generation, summarize the trace, and restart with an explicit 'earlier trace may be stale' flag; the paper's prompt-guidance results suggest much of the damage is at the interface level, so this is testable without retraining.
  • The self-doubt trope invites a training-side hypothesis: if models were rewarded for emitting an explicit 'assumption changed' signal and then recomputing, the late-update accuracy cliff should shrink; measuring that gap before and after such an intervention would test the causal story behind the 60% drop.
  • Because the experiments cover math and coding with single clean updates, the three failure modes may manifest differently in conversational or tool-using agents; building update-driven versions of open-ended agent tasks would show whether the 60% cliff is universal.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper challenges the 'frozen world' assumption commonly made in LRM evaluation. It introduces two dynamic scenarios: time-constrained interruptions (hard and soft) and update-driven interruptions, where task specifications change mid-reasoning. The authors formalize interruption-conditioned accuracy and length metrics, build a new benchmark by augmenting GSM8K, MATH500, AIME24/25, and LiveCodeBench-v6, and evaluate Qwen3-8B, GPT-OSS-20B, and Magistral-S-1.2. They report that hard interrupts produce roughly anytime behavior but cause 'reasoning leakage' (continued reasoning in the answer segment), soft interrupts can induce 'panic' (premature termination), and update-driven interruptions cause 'self-doubt' and accuracy drops of up to 60%. They also show that a simple prompt-guidance postfix mitigates some of the degradation, and they ablate model scale and assistant-turn vs. user-turn injection. The paper's central claim is that static evaluations overestimate LRM robustness in dynamic, interruptible settings.

Significance. If the results hold, this is a timely and important contribution. The paper identifies a real evaluation gap: LRMs are deployed in interactive settings where contexts change mid-inference, yet standard benchmarks assume a static query. The proposed protocols and public dataset could become useful evaluation infrastructure, and the failure-mode taxonomy (reasoning leakage, panic, self-doubt) is intuitive and actionable. The experimental design has notable strengths: relative interrupt positions rather than absolute token thresholds, direct accuracy measurements with bootstrapped CIs, ablations on model scale and on user-turn vs. assistant-turn injection, and a large amount of qualitative analysis. However, the headline 60% drop and the failure-mode attributions rest on two pieces of evidence that are not yet fully validated: the manually verified but not independently audited update-driven dataset, and the GPT-5-based self-doubt classifier. Because these are central to the paper's claims, the manuscript needs additional validation before the conclusions can be accepted at face value.

major comments (4)
  1. [§3.2, Appendix D] The update-driven benchmark is the empirical basis for the headline 'up to 60%' drop (§1, Fig. 4). The 'necessary update' condition a*(q) ≠ a*(q,u) is established only by manual author verification of GPT-5-generated content. The paper reports no inter-annotator agreement, no count of rejected/replaced generations, and no demonstration that every update is unambiguous and sufficient. The Appendix D examples contain multi-part natural-language updates (e.g., the AIME2025 update changes denominations, comparison set, and name); if any component is ambiguous or not strictly required, final outputs may be scored against the wrong ground truth and the measured accuracy drops would be inflated. Please provide an independent annotation pass, or at least detailed audit statistics, and release the annotation instructions.
  2. [§5, Fig. 1(b)] The 'self-doubt' attribution, including the claim that roughly 80% of update-driven errors stem from self-doubt, relies on an LLM-based classifier (GPT-5) that is not validated. There is no precision/recall, no human agreement, and no error analysis. Because this failure-mode accounting is a stated contribution, please validate the classifier on a human-annotated sample or provide manual annotation of all failures. Without this, the doubt-rate numbers are difficult to interpret.
  3. [Abstract, §5, Fig. 4] The abstract's 'performance dropping by up to 60%' is not tied to a specific model, dataset, or interrupt position, and it is unclear whether the drop is relative or absolute. This makes the headline difficult to interpret and potentially misleading. Please specify the exact setting (model, dataset, X, with/without prompt guidance), report the underlying accuracies in the relevant table or figure, and qualify the claim to reflect that the maximum occurs in a particular condition rather than uniformly across models.
  4. [§4, Fig. 1(b)] The 'panic' definition is 'model closing its thinking after using less than 1% of its left context limit after the soft interrupt.' The threshold is arbitrary, and the claim that up to 80% of soft-interrupt errors are attributable to panic is sensitive to it. Please justify the threshold or provide a sensitivity analysis across thresholds (e.g., 0.5%, 1%, 2%, 5%) to show that the qualitative conclusion is robust.
minor comments (6)
  1. [§3.1, Figs. 2–3] The notation L*(X) is used inconsistently: §3.1 defines it as the static no-interruption cost, but Figure 3's caption uses L*(X) to mean the updated output length and compares it to L(X). Please standardize the notation.
  2. [Fig. 1 caption] The caption states 'Results are reported at 30% interruption points,' but the paper reports results at multiple interrupt positions (0.1–0.9) in other figures. Clarify which figure uses which setting.
  3. [Appendix D] In the math augmentation prompt template, the example output contains 'one-quarters' and similar typos; more importantly, the template examples sometimes modify fewer than four specifications or remove the requested output format. This is in the prompt template rather than the final dataset, but it would help to show a cleaned template.
  4. [Abstract] The abstract contains formatting artifacts: 'Project Page/githubCode' and 'Dataset' appear to be corrupted or misplaced. Also, the project page link is missing from the actual text.
  5. [References] Many author names appear with non-breaking spaces (e.g., 'Y ang', 'W e', 'T o'), which is clearly a LaTeX issue. Please fix the bibliography formatting.
  6. [§3.2, update-driven setup] The sentence 'For math tasks, we modify initial conditions ... so that the updated problem p′ together with u is semantically equivalent to the original problem p' is confusingly worded. Since p′ is the initially presented augmented problem and u is the correction, the composition p′ + u equals p. Rewording would improve clarity.

Circularity Check

0 steps flagged

No circularity: the accuracy measurements, failure-mode attributions, and scaling analyses are direct empirical observations on independently defined protocols.

full rationale

The paper's central claims are empirical measurements of LRM behavior under hard, soft, and update-driven interrupts. The accuracy metric Ai(X)=Pr[a'=a*|X,i] is a definition of the evaluation quantity, not a derived prediction. Time-constrained experiments use unmodified official benchmarks, and update-driven experiments use GPT-5-generated, manually verified updates whose ground-truth answers are taken from the original problems. Constructing updates that change the answer is a dataset-design requirement, not a fitted parameter subsequently renamed as a prediction. The failure-mode labels (reasoning leakage, panic, self-doubt) are operationally defined and then measured on generated traces; panic is defined by early termination, and self-doubt is classified by an LLM-based classifier, so these attributions are not numerically forced by the definitions of the accuracy metric. No load-bearing self-citations, imported uniqueness theorems, or ansatz-via-citation steps appear. Concerns about update ambiguity or classifier reliability are data-quality and correctness risks, not circularity, and do not change the verdict that the empirical derivation chain is self-contained.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 3 invented entities

The paper is an empirical study rather than a derivation, so the ledger focuses on design choices and measurement assumptions. The only hand-chosen numeric threshold is the panic definition; the main evidential weight rests on the validity of the update-driven dataset, the faithfulness of the truncation simulation, model representativeness, and the unvalidated GPT-5 doubt classifier.

free parameters (1)
  • panic threshold = 1% of remaining context
    Hand-chosen threshold used to define panic behavior in the soft-interrupt setting; reported panic rates and the claim that up to 80% of performance loss is attributable to panic depend on this cutoff.
axioms (4)
  • domain assumption Truncating a pre-computed full reasoning trace at fraction X and injecting interruption tokens faithfully represents a real-time interrupt.
    Section 3.3 conditions the model on the prefix r:X, which is identical to a real-time interrupt state, but the simulation assumes no dependence on the sampling trajectory beyond the prefix.
  • domain assumption Augmented problems and updates are constructed so that each update is necessary and sufficient to restore the original problem (a*(q) != a*(q,u)).
    Section 3.2 and Appendix D rely on GPT-5 generation and manual verification; if any update is ambiguous or incorrectly validated, accuracy drops could be misattributed to model behavior.
  • domain assumption Qwen3-8B/32B, GPT-OSS-20B, and Magistral-S-1.2 are representative state-of-the-art reasoning models.
    Section 7 notes findings may not generalize across all architectures, scales, or training paradigms; the paper's central claim depends on the sample of models studied.
  • domain assumption The GPT-5-based classifier accurately identifies self-doubt in truncated reasoning traces.
    Section 5 uses this classifier to attribute roughly 80% of update-driven errors to self-doubt; classifier accuracy is not reported, so the causal attribution is only as strong as this assumption.
invented entities (3)
  • reasoning leakage no independent evidence
    purpose: Describes the observed behavior where a model continues internal reasoning inside the final answer segment after a hard interrupt, inflating answer length and compute cost.
    Operationalized via answer-length measurements in Section 4; no external benchmark independently validates this as a distinct phenomenon.
  • panic no independent evidence
    purpose: Describes the behavior where a model prematurely terminates reasoning after a speedup instruction, causing incorrect outputs.
    Operationalized as closing thinking within 1% of remaining context in Section 4; the threshold is hand-chosen and not externally validated.
  • self-doubt no independent evidence
    purpose: Describes the behavior where a model doubts the validity of an injected update and does not incorporate it, leading to errors.
    Operationalized via a GPT-5 classifier in Section 5; no human-annotation validation or external measure is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 26117 in / 11031 out tokens · 98387 ms · 2026-08-04T10:03:00.537494+00:00 · methodology

0 comments
read the original abstract

Real-world applications of Large Reasoning Models (LRMs) often require reasoning about changing prompts or environments. In this work, we challenge the frozen world assumption and evaluate LRM robustness under two realistic dynamic scenarios: interruptions, which test the accuracy of model responses under budget-constrained outputs, and dynamic context, which tests model adaptation to in-flight changes. Across mathematics and programming benchmarks that require long-form reasoning, static evaluations consistently overestimate robustness: even state-of-the-art LRMs, which achieve high accuracy in static settings, can fail unpredictably when interrupted or exposed to changing context, with performance dropping by up to 60% when updates are introduced late in the reasoning process. Our analysis further reveals several novel failure modes, including reasoning leakage, where models fold the reasoning into their final answer when interrupted; panic, where under time pressure models abandon reasoning entirely and return incorrect answers; and self-doubt, where performance degrades when trying to incorporate updated information. Project Page: http://dynamic-lm.github.io/

Figures

Figures reproduced from arXiv: 2510.11713 by David M. Chan, Joseph E. Gonzalez, Mihran Miroyan, Narges Norouzi, Trevor Darrell, Tsung-Han Wu.

Figure 1
Figure 1. Figure 1: How do LRMs perform in dynamic worlds? (a) Unlike static ‘frozen world’ settings that assume users wait for completion, real-world scenarios often demand mid-inference updates, as LRM reasoning can be time-consuming. We introduce a public evaluation suite to assess how LRMs handle interruptions across math and coding tasks (section 3). We define two types of interruptions: time-constrained (hard: immediate… view at source ↗
Figure 2
Figure 2. Figure 2: Efficiency and Accuracy under Hard Interrupts. The top row reports model performance (Pass@1, denoted as A(X) in Section 3), while the bottom row shows absolute final answer lengths L(X) under two settings across different interrupt position X. In the top row, we observe that LRMs behave almost like anytime models, with performance improving as more reasoning budget is provided. In the bottom row, we find … view at source ↗
Figure 3
Figure 3. Figure 3: Answer Length Analysis under Soft Interrupts (Speedup). When receiving an instruction to speed up the reasoning process (i.e., soft interrupt), models generally comply, as the updated output length L ∗ (X) is shorter than the original L(X), with the exception of later interrupt positions (e.g., Magistral-S-1.2 on GSM8K at 0.9). However, soft interrupts can hurt performance on harder tasks such as AIME and … view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy under Update-Driven Interrupts. When provided with updates mid-reasoning, models often suffer substantial performance drops. Adding prompt guidance fully resolves the issue on GSM8K and MATH500 and reduces the gap between full thinking and interrupt settings on AIME and LiveCodeBench datasets. Problem [P]: You are given three integers…the jury will award each band a score in the range [0, y]… (a n… view at source ↗
Figure 5
Figure 5. Figure 5: We observe that one of the behaviors causing the significant performance drops under [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Efficiency and Accuracy under Update-Driven Interrupts. Performance generally decreases as updates come later in the reasoning process, despite increased reasoning effort in order to account for newly introduced information. While reasoning effort does increase, in some cases the overall reasoning effort needed to incorporate an update is far below the number of tokens which would be required to restart fr… view at source ↗
Figure 7
Figure 7. Figure 7: Efficiency and Accuracy under Hard Interrupts by Model Scale. Scaling does have effects on accuracy, primarily for hard AIME questions, and we see increased reasoning leakage for small models, even in the extreme hard interrupt setting. setting, we observe a scaling limit in interruptible robustness: while both Qwen3-8B and Qwen3-32B respond appropriately to updates, Qwen3-1.7B struggles to generalize and … view at source ↗
Figure 8
Figure 8. Figure 8: Comparison between Assistant-Turn and User-Turn Interruptions. In update-driven interruption scenarios, assistant-turn interruption with our prompt guidance achieves better performance than user-turn interruption. 7 LIMITATIONS AND CONCLUSION While our study highlights several failure modes of LRMs under interruptions and dynamic contexts, several limitations remain. First, our evaluation relies primarily … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 22 linked inside Pith

  1. [1]

    gpt-oss-120b & gpt-oss-20b model card

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Training language models to reason efficiently

    Daman Arora and Andrea Zanette. Training language models to reason efficiently. arXiv preprint arXiv:2502.04463, 2025

    arXiv 2025

  3. [3]

    Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching

    Simon A Aytes, Jinheon Baek, and Sung Ju Hwang. Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching. arXiv preprint arXiv:2503.05179, 2025

    arXiv 2025

  4. [4]

    Exploring efficiency frontiers of thinking budget in medical reasoning: Scaling laws between computational resources and reasoning quality

    Ziqian Bi, Lu Chen, Junhao Song, Hongying Luo, Enze Ge, Junmin Huang, Tianyang Wang, Keyu Chen, Chia Xin Liang, Zihan Wei, et al. Exploring efficiency frontiers of thinking budget in medical reasoning: Scaling laws between computational resources and reasoning quality. arXiv preprint arXiv:2508.12140, 2025

    arXiv 2025

  5. [5]

    K2-think: A parameter-efficient reasoning system

    Zhoujun Cheng, Richard Fan, Shibo Hao, Taylor W Killian, Haonan Li, Suqi Sun, Hector Ren, Alexander Moreno, Daqian Zhang, Tianjun Zhong, et al. K2-think: A parameter-efficient reasoning system. arXiv preprint arXiv:2509.07604, 2025

    arXiv 2025

  6. [6]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  7. [7]

    American invitational mathematics examination-aime 2024, 2024, 2024

    MAA Codeforces. American invitational mathematics examination-aime 2024, 2024, 2024

    2024

  8. [8]

    An analysis of time-dependent planning

    Thomas L Dean and Mark S Boddy. An analysis of time-dependent planning. In AAAI, volume 88, pp.\ 49--54, 1988

    1988

  9. [9]

    Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514, 2025

    Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Token-budget-aware llm reasoning

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Thinkless: A training-free inference-efficient method for reducing reasoning redundancy

    Gengyang Li, Yifeng Gao, Yuming Li, and Yunfang Wu. Thinkless: A training-free inference-efficient method for reducing reasoning redundancy. arXiv preprint arXiv:2505.15684, 2025 a

    Pith/arXiv arXiv 2025

  16. [16]

    Steering llm thinking with budget guidance

    Junyan Li, Wenshuo Zhao, Yang Zhang, and Chuang Gan. Steering llm thinking with budget guidance. arXiv preprint arXiv:2506.13752, 2025 b

    Pith/arXiv arXiv 2025

  17. [17]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=v8L0pN6EOi

    2024

  18. [18]

    Can language models learn to skip steps? Advances in Neural Information Processing Systems, 37: 0 45359--45385, 2024

    Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. Can language models learn to skip steps? Advances in Neural Information Processing Systems, 37: 0 45359--45385, 2024

    2024

  19. [19]

    Reasoning models can be effective without thinking

    Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Self-training elicits concise reasoning in large language models

    Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. Self-training elicits concise reasoning in large language models. arXiv preprint arXiv:2502.20122, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Concise thoughts: Impact of output length on llm reasoning and cost

    Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length on llm reasoning and cost. arXiv preprint arXiv:2407.19825, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Nvidia nemotron nano 2: An accurate and efficient hybrid mamba-transformer reasoning model, 2025

    NVIDIA. Nvidia nemotron nano 2: An accurate and efficient hybrid mamba-transformer reasoning model, 2025. URL https://arxiv.org/abs/2508.14444

    Pith/arXiv arXiv 2025

  24. [24]

    Magistral

    Abhinav Rastogi, Albert Q Jiang, Andy Lo, Gabrielle Berrada, Guillaume Lample, Jason Rute, Joep Barmentlo, Karmesh Yadav, Kartik Khandelwal, Khyathi Raghavi Chandu, et al. Magistral. arXiv preprint arXiv:2506.10910, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

    2024

  26. [26]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025 a

    Pith/arXiv arXiv 2025

  27. [27]

    Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models

    Yuan Sui, Yufei He, Tri Cao, Simeng Han, Yulin Chen, and Bryan Hooi. Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models. arXiv preprint arXiv:2502.19918, 2025 b

    Pith/arXiv arXiv 2025

  28. [28]

    Wait, we don't need to" wait"! removing thinking tokens improves reasoning efficiency

    Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou. Wait, we don't need to" wait"! removing thinking tokens improves reasoning efficiency. arXiv preprint arXiv:2506.08343, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Tokenskip: Controllable chain-of-thought compression in llms

    Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502.12067, 2025

    arXiv 2025

  30. [30]

    Chain of draft: Thinking faster by writing less

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Inftythink: Breaking the length limits of long-context reasoning in large language models

    Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. Inftythink: Breaking the length limits of long-context reasoning in large language models. arXiv preprint arXiv:2503.06692, 2025

    arXiv 2025

  32. [32]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Distilling system 2 into system 1

    Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023, 2024

    Pith/arXiv arXiv 2024

  34. [34]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  35. [35]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  36. [36]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  37. [37]

    hurry up

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv 2024