Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Reasoning in hybrid LLMs is switched by a few trigger tokens, not by instructions, and the paper exploits this with a training-free prompt format that delivers intermediate-budget thinking.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:11 UTC pith:4QTCFJWS

load-bearing objection A practical training-free budget dial for hybrid reasoning models, with a mechanistic story that overreaches but a trick that works. the 3 major comments →

arxiv 2601.07036 v2 pith:4QTCFJWS submitted 2026-01-11 cs.CL cs.AIcs.LG

Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers

Wang Yang , Debargha Ganguly , Xinpeng Li , Chaoda Song , Shouren Wang , Vikash Singh , Vipin Chaudhary , Xiaotian Han This is my paper
classification cs.CL cs.AIcs.LG
keywords token-level triggershybrid reasoning modelsattention sinkintermediate budgetreasoning controlprompt engineeringreinforcement learning post-trainingPareto trade-off
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that hybrid reasoning models enter 'think' mode because of a handful of token cues, not because of the high-level instructions they are given. In particular, a leading 'Okay' token flips the model into reasoning, while the newline that follows the closing tag pushes it into no-think mode. The authors propose Mid-Think, a simple prompt format that combines both cues, so the model produces a middle amount of reasoning with no extra training. That matters because it offers a cheap, interpretable dial for trading accuracy against output length, and it also makes reinforcement-learning post-training faster and more accurate.

Core claim

On the paper's own terms: for Qwen3-series hybrid models, the mode switch between thinking and not thinking is not controlled by the user's instruction but by a small set of token-level triggers. Attention analysis shows that later generated tokens concentrate their attention on an opening 'Okay' token when the model reasons, and on the newline pattern after </think> when it does not. Controlled prompt experiments confirm that adding 'Okay' reproduces Think-mode accuracy and wait counts, while the </think>+newline pattern suppresses them. Mid-Think (<think>\n\n</think>\n\n<reason>\nOkay…) combines both triggers to induce intermediate-budget reasoning that sits on or beyond the Pareto frontie

What carries the argument

The central objects are the token-level triggers: the leading 'Okay' token that acts as a reasoning-on switch, and the newline pair after </think> that acts as a reasoning-off switch. The Mid-Think prompt format stacks both in one context. It works by keeping the no-think structural cue while adding the think trigger, so the model settles into a middle reasoning budget automatically, with no training and no per-instance token limit. Attention analysis is the evidence, and the prompt itself is the mechanism.

Load-bearing premise

The load-bearing premise is that the 'Okay' token is the active cause of reasoning behavior, not a side effect of attention-sink or position effects; the paper's evidence is observational (attention patterns and prompt variants) and does not include a control that replaces 'Okay' with another token in the same position.

What would settle it

A concrete test: in the standard Think prompt, replace the leading 'Okay' with a neutral token such as 'The' or 'Mm' in the same position, and measure wait count and accuracy. If the model still produces full reasoning traces, the 'Okay' token is not a necessary trigger and the mechanistic claim collapses. Alternatively, patch or ablate the attention from generated tokens to the 'Okay' token during generation; if reasoning persists, attention to that token is not causal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the trigger-token claim holds, reasoning budget can be tuned without retraining or per-instance token limits; a single prompt change yields a middle accuracy-length operating point.
  • The same format carries over to RL after SFT, implying that other post-training phases can inherit the efficiency and accuracy gains without architectural changes.
  • Because the trigger works across model scales and training regimes listed in the paper, the finding suggests a general mechanism in hybrid reasoning models: the policy has overfit to low-level lexical cues, which can be exploited for control.
  • Mid-Think can be used as a baseline for future budget-control research, since it needs no training and no search over token limits.
  • In RL, Mid-Think preserves no-think behavior better than Think training while achieving higher think-mode accuracy, suggesting it is a better objective for hybrid models that need to do both.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The trigger-token story is likely not limited to these specific models: any model fine-tuned on reasoning traces that uniformly start with 'Okay' could develop the same overfit, so the format may transfer to other open and closed models as long as their SFT data shares that lexical pattern.
  • A natural extension is to find other trigger tokens (e.g., 'Let's', 'Hmm', 'wait') and see whether each maps to a different reasoning depth; if so, one could build a discrete family of budget settings from token choice alone.
  • A stronger causal test would be to patch or ablate the 'Okay' token's attention during generation; if reasoning persists without it, the mechanism is likely an attention-sink artifact rather than true causality, though the Mid-Think prompt might still work for other reasons.
  • If token-level triggers are so dominant, safety or evaluation work on hybrid models should control for them; otherwise a small prompt change could silently switch a model in or out of reasoning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper claims that the Think/No-think behavior of hybrid reasoning models is governed not by high-level instructions but by a small set of token-level triggers: a leading “Okay” token induces reasoning, while the newline pattern after “</think>” suppresses it. The authors support this with attention visualizations and prompting experiments on Qwen3-8B, then propose Mid-Think, a training-free prompt format (<think>\n\n</think>\n\n<reason>\nOkay...) that combines both triggers to obtain an intermediate reasoning budget. They evaluate Mid-Think across Qwen3-8B/14B/32B and DeepSeek-Qwen-7B on MATH500, AIME22–24, and GPQA, comparing against fixed-token and prompt-based budget-control baselines. They also apply Mid-Think to GRPO post-training after SFT and report improved accuracy and ~15% training-time reduction for Qwen3-8B on AIME and GPQA.

Significance. If the token-trigger claim were established, it would be a noteworthy mechanistic finding about hybrid reasoning models and could inform both inference-time control and RL post-training. The practical Mid-Think prompt is simple, training-free, and the reported accuracy–length trade-offs in Table 4 are encouraging. The paper also ships code and evaluates on multiple benchmarks and model families, which is a strength. However, the central causal attribution is not yet supported by the evidence as presented; the budget-control baseline is a proxy; and the RL results lack variance. These issues are addressable, and the empirical prompt format may still be a useful contribution even if the mechanistic story needs to be softened.

major comments (3)
  1. [§2.2–2.3, Fig. 2, Table 2] The central claim that a leading “Okay” token actively induces reasoning and the newline after “</think>” suppresses it is not identified by the presented evidence. Fig. 2 reports average attention, and Table 2 compares prompt formats, but no condition replaces “Okay” with a control token in the same slot, and no intervention (e.g., attention patching or ablation) is performed. Because early tokens are known to attract high attention regardless of content (attention sinks; Xiao et al. 2023, Gu et al. 2024), the attention concentration alone cannot establish causation. Moreover, the comparisons in Table 2 vary several factors at once: “No Think + Okay” differs from “Think” in both the position of “Okay” (after “</think>\n\n” rather than after “<think>”) and the presence of the “</think>” boundary. A minimal control would compare <think>\nOkay, <think>\nThe, <think>\n###, and <think>\n\n</
  2. [§3.2.1, Fig. 4] The budget-controlled baseline does not measure a model’s reasoning under a token budget; it measures continuation from a truncated prefilled chain. The protocol first generates a full Think-mode trace, keeps the first fraction of the reasoning tokens, concatenates them into the prompt, and lets the model finish. Under this protocol the model is effectively given the beginning of the answer, so the monotonic accuracy–budget curve in Fig. 3 is expected even without any notion of intermediate reasoning. Consequently, the statements that Mid-Think “corresponds to a budget of 0.5” and “surpasses the Pareto frontier” (Figs. 5–6) are comparisons against an axis that is not a true budget-control mechanism. Please either replace this baseline with an actual constrained-generation protocol (e.g., max-token early stopping or a forced-exit mechanism) or explicitly reframe the comparison as “prefill
  3. [§4, Table 5] The RL training results are reported as single runs with no variance or seed information. For Qwen3-8B, RL Mid-Think improves Think-test AIME accuracy from 69.8 to 72.4, but the Mid-Think run also produces longer outputs (15,318 vs. 13,330 tokens) and more “wait” tokens (44,142 vs. 34,701), so the gain may partly reflect a test-time compute effect. The claimed ~15% training-time reduction is not backed by a precise definition (wall-clock? GPU-hours?) in the experimental section; Figure 8 shows relative time but no quantitative table. Please report at least three seeds with standard deviations for both accuracy and training time, and, if the test-time length differs, include a length-controlled comparison or discuss the confound.
minor comments (5)
  1. [Table 3 and Fig. 1] The notation “Okay...” is ambiguous: does the literal prompt contain “Okay” followed by an ellipsis, or is the ellipsis a placeholder for model-generated content? Please clarify in the caption or table footnote.
  2. [§2.2] The attention analysis averages over all layers and heads. This may hide head-specific or layer-specific roles. Please report whether the concentration on “Okay” and on the newline is consistent across layers and heads, or at least across a few representative layers.
  3. [Table 1] The checkmark table is vague. For each model/dataset, report the frequency of “Okay” and related tokens, or at least a concrete count rather than a binary checkmark.
  4. [§3.1] The sentence “The specific token choice is not essential; we evaluate three variants: <reason>, <begin>, and <less think>” refers to the wrapper tags, not to the lexical trigger. This is fine, but it should be stated more precisely to avoid confusing tag choice with trigger-token choice.
  5. [§3.2 and §4] Wording such as “Pareto-optimal” and “consistently outperforms” is stronger than what single-seed, no-error-bar experiments can support. Please either add error bars or soften the wording.

Circularity Check

0 steps flagged

No significant circularity; trigger-token claims are empirically tested and Mid-Think is evaluated on held-out benchmarks, not derived from its own outputs.

full rationale

The paper's derivation chain is empirical and self-contained. Section 2 identifies candidate trigger tokens by attention analysis and then tests them with compositional prompting formats in Table 2; Mid-Think in Table 3 is constructed from those tested cues and evaluated on held-out MATH500/AIME/GPQA. No parameter is fitted to the evaluation benchmarks, and the budget-controlled baseline is used only as a comparison axis, not as an input that defines Mid-Think. The RL-training results are direct measurements of accuracy, length, and wall-clock time, not predictions derived from the trigger hypothesis. The only self-citation (Speculative Thinking, Yang et al. 2025c) appears in the introduction as background on structural newline patterns and is not load-bearing; the paper's own attention and prompting experiments support the trigger claim. Concerns about causal identification (e.g., no token-swap control for 'Okay' versus attention-sink position effects) are validity or correctness issues, not circularity, and do not constitute a reduction of the paper's conclusions to its inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper rests on empirical assumptions about attention-as-causation, benchmark reliability without error bars, the validity of a truncation-based budget baseline, and the semantic neutrality of newly introduced tags. No numerical free parameters are fitted; the method is a prompt template. No new theoretical entities are posited.

axioms (4)
  • domain assumption Average attention across all layers/heads and generated tokens is a valid proxy for causal control of reasoning mode.
    §2.2 uses attention heatmaps to conclude 'Okay' triggers thinking; no causal intervention (e.g., patching/ablating the token) and no control for position or attention-sink effects.
  • domain assumption MATH500/AIME/GPQA point estimates, without error bars or seeds, are reliable enough to support 1–4 point accuracy differences.
    Tables 2, 4, 5 report single-run accuracy; comparisons such as 72.4 vs 69.8 and 61.1 vs 58.5 lack variance or significance tests.
  • domain assumption Truncating a full Think-mode trajectory at budget b and then letting the model answer is a valid measure of budget-limited reasoning performance.
    §3.2.1 constructs the budget–accuracy curve this way; this defines the Pareto frontier against which Mid-Think is judged. A model that never generated the truncated portion may behave differently.
  • domain assumption The special tags <reason>, <begin>, <less think> act as neutral structural delimiters and do not themselves alter behavior beyond the intended cue.
    §3.1 asserts token choice is not essential and evaluates three variants, but the mechanism is inferred from attention to 'Okay', not independently verified for each tag.

pith-pipeline@v1.3.0-alltime-deepseek · 14499 in / 15535 out tokens · 152348 ms · 2026-08-03T11:11:21.803700+00:00 · methodology

0 comments
read the original abstract

Hybrid reasoning language models are commonly controlled through high-level Think/No-think instructions to regulate reasoning behavior, yet we found that such mode switching is largely driven by a small set of trigger tokens rather than the instructions themselves. Through attention analysis and controlled prompting experiments, we show that a leading ``Okay'' token induces reasoning behavior, while the newline pattern following ``</think>'' suppresses it. Based on this observation, we propose Mid-Think, a simple training-free prompting format that combines these triggers to achieve intermediate-budget reasoning, consistently outperforming fixed-token and prompt-based baselines in terms of the accuracy-length trade-off. Furthermore, applying Mid-Think to RL training after SFT reduces training time by approximately 15% while improving final performance of Qwen3-8B on AIME from 69.8% to 72.4% and on GPQA from 58.5% to 61.1%, demonstrating its effectiveness for both inference-time control and RL-based reasoning training.

Figures

Figures reproduced from arXiv: 2601.07036 by Chaoda Song, Debargha Ganguly, Shouren Wang, Vikash Singh, Vipin Chaudhary, Wang Yang, Xiaotian Han, Xinpeng Li.

Figure 1
Figure 1. Figure 1: Illustration of Mid-Think and its performance [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Average attention from generated tokens to different opening tokens under five generation modes. Darker [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the budget-controlled method. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison between hybrid-thinking models (from Qwen3 family) under different reasoning budgets [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison between different budget reasoning and Mid-Think on pure-think and RL-based models across [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Entropy during GRPO. We compare Qwen3- 8B-Base trained in the Think mode with Qwen3-8B trained under Think, No-think, and Mid-Think modes. The panel plots entropy versus training steps. Notably, Mid-Think both increases training entropy. balancing reasoning quality and output length. 4 Applying Mid-Think to RL Training after SFT This section applies the proposed Mid-Think mode to RL training on top of supe… view at source ↗
Figure 9
Figure 9. Figure 9: Comparison between different budget reasoning and Mid-Think on pure-think and RL-based models [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CausalGuard: Conformal Inference under Graph Uncertainty

    cs.LG 2026-05 unverdicted novelty 6.0

    CausalGuard aggregates LLM-proposed and data-pruned DAGs to weight doubly robust pseudo-outcomes and applies conformal calibration to deliver finite-sample marginal coverage for conditional average treatment effects u...

  2. Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

    cs.LG 2026-05 unverdicted novelty 6.0

    RMemSafe gates source anchoring via entropy in CTTA, reducing error by 1.05pp on ResNet-50 when source accuracy collapses and showing shallower degradation slope than prior methods.

  3. Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

    cs.LG 2026-05 unverdicted novelty 6.0

    RMemSafe attenuates source anchoring via entropy gating when the frozen source model degrades, yielding lower error than prior methods on continual corruption benchmarks and shallower degradation under source failure.

Reference graph

Works this paper leans on

43 extracted references · 27 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925

    Pith/arXiv arXiv 2025

  2. [2]

    Dibyanayan Bandyopadhyay, Soham Bhattacharjee, and Asif Ekbal. 2025. Thinking machines: A survey of llm based reasoning strategies. arXiv preprint arXiv:2503.10814

    Pith/arXiv arXiv 2025

  3. [3]

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187

    Pith/arXiv arXiv 2024

  4. [4]

    Gongfan Fang, Xinyin Ma, and Xinchao Wang. 2025. Thinkless: Llm learns when to think. arXiv preprint arXiv:2505.13379

    Pith/arXiv arXiv 2025

  5. [5]

    Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. 2024. When attention sink emerges in language models: An empirical view. arXiv preprint arXiv:2410.10781

    Pith/arXiv arXiv 2024

  6. [6]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    Pith/arXiv arXiv 2025

  7. [7]

    Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. 2025. Rethinking entropy interventions in rlvr: An entropy change perspective. arXiv preprint arXiv:2510.10150

    Pith/arXiv arXiv 2025

  8. [8]

    Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. 2025 a . Skywork open reasoner 1 technical report. arXiv preprint arXiv:2505.22312

    Pith/arXiv arXiv 2025

  9. [9]

    Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Yang Liu, and Yahui Zhou. 2025 b . Skywork open reasoner series. Notion Blog

    2025

  10. [10]

    Hugging Face . 2025. https://github.com/huggingface/open-r1 Open r1: A fully open reproduction of deepseek-r1

    2025

  11. [11]

    Feiyang Kang, Michael Kuchnik, Karthik Padthe, Marin Vlastelica, Ruoxi Jia, Carole-Jean Wu, and Newsha Ardalani. 2025. Quagmires in sft-rl post-training: When high sft scores mislead and what to use instead. arXiv preprint arXiv:2510.01624

    arXiv 2025

  12. [12]

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, and 1 others. 2025. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419

    Pith/arXiv arXiv 2025

  13. [13]

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, and 1 others. 2024. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434

    Pith/arXiv arXiv 2024

  14. [14]

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783

    Pith/arXiv arXiv 2025

  15. [15]

    Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. 2025. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset. arXiv preprint arXiv:2504.16891

    Pith/arXiv arXiv 2025

  16. [16]

    Gongrui Nan, Siye Chen, Jing Huang, Mengyu Lu, Dexun Wang, Chunmei Xie, Weiqi Xiong, Xianzhou Zeng, Qixuan Zhou, Yadong Li, and 1 others. 2025. Ngrpo: Negative-enhanced group relative policy optimization. arXiv preprint arXiv:2509.18851

    arXiv 2025

  17. [17]

    NVIDIA, :, Aarti Basant, Abhijit Khairnar, Abhijit Paithankar, Abhinav Khattar, Adithya Renduchintala, Aditya Malte, Akhiad Bercovich, and 1 others. 2025. https://arxiv.org/abs/2508.14444 Nvidia nemotron nano 2: An accurate and efficient hybrid mamba-transformer reasoning model . Preprint, arXiv:2508.14444

    Pith/arXiv arXiv 2025

  18. [18]

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. 2024. Reasoning with large language models, a survey. arXiv preprint arXiv:2407.11511

    arXiv 2024

  19. [19]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling

    2024

  20. [20]

    Brian S Robinson, Nathan Drenkow, Colin Conwell, and Michael Bonner. 2024. A sparse null code emerges in deep neural networks. In Proceedings of UniReps: The First Workshop on Unifying Representations in Neural Models, pages 302--314. PMlR

    2024

  21. [21]

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, and 1 others. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419

    Pith/arXiv arXiv 2025

  22. [22]

    Mingjie Sun, Xinlei Chen, J Zico Kolter, and Zhuang Liu. 2024. Massive activations in large language models. arXiv preprint arXiv:2402.17762

    Pith/arXiv arXiv 2024

  23. [23]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, and 1 others. 2025 a . Gemma 3 technical report. arXiv preprint arXiv:2503.19786

    Pith/arXiv arXiv 2025

  24. [24]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025 b . Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    Pith/arXiv arXiv 2025

  25. [25]

    NovaSky Team. 2025. Think less, achieve more: Cut reasoning costs by 50 https://novasky-ai.github.io/posts/reduce-overthinking. Accessed: 2025-01-23

    2025

  26. [26]

    Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou. 2025. Wait, we don't need to" wait"! removing thinking tokens improves reasoning efficiency. arXiv preprint arXiv:2506.08343

    Pith/arXiv arXiv 2025

  27. [27]

    Yihong Wu, Liheng Ma, Lei Ding, Muzhi Li, Xinyu Wang, Kejia Chen, Zhan Su, Zhanguang Zhang, Chenyang Huang, Yingxue Zhang, and 1 others. 2025. It takes two: Your grpo is secretly dpo. arXiv preprint arXiv:2510.00977

    Pith/arXiv arXiv 2025

  28. [28]

    Zhiheng Xi, Xin Guo, Yang Nan, Enyu Zhou, Junrui Shen, Wenxiang Chen, Jiaqi Liu, Jixuan Huang, Zhihao Zhang, Honglin Guo, and 1 others. 2025. Bapo: Stabilizing off-policy reinforcement learning for llms via balanced policy optimization with adaptive clipping. arXiv preprint arXiv:2510.18927

    arXiv 2025

  29. [29]

    Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. 2025. https://arxiv.org/abs/2502.12067 Tokenskip: Controllable chain-of-thought compression in llms . Preprint, arXiv:2502.12067

    arXiv 2025

  30. [30]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2023. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453

    Pith/arXiv arXiv 2023

  31. [31]

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, and 1 others. 2025 a . Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686

    Pith/arXiv arXiv 2025

  32. [32]

    Xin Xu, Cliveb AI, Kai Yang, Tianhao Chen, Yang Wang, Saiyong Yang, and Can Yang. 2025 b . Thinking-free policy initialization makes distilled reasoning models more effective and efficient reasoners. arXiv preprint arXiv:2509.26226

    arXiv 2025

  33. [33]

    Yuhui Xu, Hanze Dong, Lei Wang, Doyen Sahoo, Junnan Li, and Caiming Xiong. 2025 c . Scalable chain of thoughts via elastic reasoning. arXiv preprint arXiv:2505.05315

    Pith/arXiv arXiv 2025

  34. [34]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025 a . Qwen3 technical report. arXiv preprint arXiv:2505.09388

    Pith/arXiv arXiv 2025

  35. [35]

    Rubing Yang, Huajun Bai, Song Liu, Guanghua Yu, Runzhi Fan, Yanbin Dang, Jiejing Zhang, Kai Liu, Jianchen Zhu, and Peng Chen. 2025 b . Specexit: Accelerating large reasoning model via speculative exit. arXiv preprint arXiv:2509.24248

    arXiv 2025

  36. [36]

    Wang Yang, Xiang Yue, Vipin Chaudhary, and Xiaotian Han. 2025 c . Speculative thinking: Enhancing small-model reasoning with large model guidance at inference time. arXiv preprint arXiv:2504.12329

    Pith/arXiv arXiv 2025

  37. [37]

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476

    Pith/arXiv arXiv 2025

  38. [38]

    Charlie Zhang, Graham Neubig, and Xiang Yue. 2025 a . On the interplay of pre-training, mid-training, and rl on reasoning language models. arXiv preprint arXiv:2512.07783

    arXiv 2025

  39. [39]

    Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. 2025 b . Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:2502.15589

    arXiv 2025

  40. [40]

    Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, and 1 others. 2025 c . A survey of reinforcement learning for large reasoning models. arXiv preprint arXiv:2509.08827

    Pith/arXiv arXiv 2025

  41. [41]

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025. Group sequence policy optimization. arXiv preprint arXiv:2507.18071

    Pith/arXiv arXiv 2025

  42. [42]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  43. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...