Pith. sign in

REVIEW 3 major objections 8 minor 29 references

Masked distillation can push chain-of-thought computation into a model's parameters, but a small emitted reasoning scaffold tunes the accuracy–cost frontier.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 10:43 UTC pith:NHTZQZHJ

load-bearing objection A useful α-suffix scaffolding axis for CoT internalization, with a likely-real task-dependence story—but every headline number is a single run, so the operating-point recommendations need variance estimates before they can be trusted. the 3 major comments →

arxiv 2607.22629 v1 pith:NHTZQZHJ submitted 2026-06-18 cs.AI cs.CL

Masked Distillation: Internalizing the Chain-of-Thought in Language Models

Durgesh Kalwar , Vardhan Palod , Subbarao Kambhampati This is my paper
classification cs.AI cs.CL
keywords masked distillationchain-of-thought internalizationknowledge distillationreasoning modelsinference costscaffoldingself-distillationout-of-distribution generalization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces masked distillation, a way to train a language model to answer directly without emitting the chain-of-thought tokens, using the same or a larger model in thinking mode as teacher. The central finding is that full internalization is task-dependent: it succeeds on grade-school arithmetic, where the student model already knows the task from pretraining, but collapses on a search-style number puzzle, dropping below the student's own base accuracy. The paper then shows that exposing a small suffix of the teacher's reasoning trace to the student at inference—a scaffold controlled by parameter α—closes the gap. With α=0.3 on the puzzle, accuracy rises from 41.7% to 86.2%, near the teacher's 87.3%, at about 1.3× fewer inference tokens than the non-masked variant. The paper argues this makes suffix scaffolding a controlled axis for trading accuracy against inference cost, with the best operating point set by the task and the capacity gap between teacher and student.

Core claim

The paper establishes that intermediate reasoning tokens can be treated as a scaffold: a distillation objective can push some of that computation into the student's weights, and the amount pushed in is a smooth dial. Masked distillation trains the student on student-sampled rollouts with a reverse-KL loss against a teacher that sees the question plus a prefix of its own intermediate trace. Varying the exposed suffix fraction α from 0 (fully masked, student emits only the answer) to 1 (non-masked, student reproduces the full trace) sweeps the whole frontier. On grade-school arithmetic, full internalization works: the student reaches 76.0% versus the teacher's 87.9% while emitting 6.5× fewer t

What carries the argument

The central object is the α-suffix masked-distillation loss, an on-policy reverse-KL divergence between the student's and teacher's next-token distributions. The teacher is conditioned on the question plus the first (1−α) fraction of its own chain-of-thought trace; the student is conditioned only on the question and is trained to emit the remaining α-suffix of the trace along with the solution. This single parameter interpolates between full internalization (α=0, direct answer) and no internalization (α=1, full trace), and it is what lets the paper map the accuracy-versus-inference-cost frontier cleanly.

Load-bearing premise

All the headline accuracies are single validation rollouts at temperature 0.6 with no seeds or error bars, so the task-dependence story and the α=0.3 recovery could shift by several points on a rerun.

What would settle it

Run the Countdown self-distillation sweep with multiple seeds and multiple validation rollouts: the central claim predicts α=0 accuracy below base and α=0.3 near teacher; if either relation flips in any seed, the paper's task-dependence and operating-point claims are falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On tasks where a student already knows the domain, direct-answer models can match or approach teacher accuracy at a large fraction of the inference cost (6.5× fewer tokens in the arithmetic experiments).
  • On search-like tasks, a small exposed scaffold (α=0.3) recovers teacher-level accuracy at about 1.3× fewer tokens than emitting the full reasoning trace, so serving can be made cheaper without giving up accuracy.
  • Scaffold-trained students transfer cleanly under target-range distribution shift, and in self-distillation they can outperform the non-masked variant under search-depth shift—so the internalized version is not simply a less general copy of the teacher.
  • The training objective matters: reverse-KL distillation exploits scaffolding, whereas supervised fine-tuning on the same teacher rollouts does not, so the result is tied to the on-policy, student-sampled feedback mechanism.
  • The α dial gives a practical knob: model providers can choose an operating point per task or per capacity gap, rather than treating direct answers and full chain-of-thought as a binary choice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: the task-dependence pattern suggests that internalization only amortizes computation the student can already approximately represent; for genuinely novel search, the emitted suffix acts as an external scratchpad the model has not yet learned to simulate. A direct test would be to compare tasks matched for difficulty but differing only in pretraining coverage.
  • Inference: the non-monotonic out-of-distribution results (e.g., a mid-scaffold variant scoring above the teacher on hard math) hint at a possible regularization effect of partial masking, but with tiny test sets this is not established; it is worth checking whether the effect survives multiple seeds.
  • Inference: the scaffold idea should extend to non-teacher sources of intermediate signal, such as environment feedback; if the paper's future-work direction holds, the same α dial could control how much environment search is done internally versus externally.
  • Inference: if single-run noise is small, the α=0.3 operating point could be a deployment recommendation for search-like reasoning tasks, but that recommendation is stronger than what the paper itself claims.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper introduces masked distillation, a knowledge-distillation framework in which a student LM is trained with an on-policy reverse-KL objective to match a reasoning teacher's distribution over solution tokens, while the student is conditioned only on the question. An α-suffix variant supervises the student to emit the last α fraction of the teacher's intermediate trace plus the answer, interpolating between full internalization (α=0) and full trace reproduction (α=1). The method is instantiated in self-distillation (Qwen3 thinking vs. non-thinking) and dual-model (Qwen3 teacher vs. Qwen2-0.5B student) settings on GSM8K and Countdown. Reported results: on GSM8K α=0 reaches 76.0% vs. teacher 87.9% with ~6.5× fewer tokens; on Countdown α=0 collapses to 41.7% (below a 56.2% base) but α=0.3 recovers 86.2% vs. teacher 87.3%; OOD transfer is mixed. An SFT ablation and teaching-token scaffold are also reported.

Significance. If the point estimates were stable, the paper would make a useful contribution: it defines a simple, tunable axis (α) controlling the accuracy/latency trade-off of reasoning models, and it documents a clear task-dependence of full internalization. The two-setting comparison and the SFT ablation are valuable and go beyond prior context-distillation work. The main strengths are the transparent α sweep, use of external benchmarks for teacher accuracy, and a clear separation of the internalization question from trace-fidelity questions. However, the evidence is currently single-run with one validation rollout per item and no code release, so the quantitative claims—especially the task-dependence and the 'most efficient operating point'—are not yet established at journal standard.

major comments (3)
  1. [Appendix A; §5.2, Figs. 2–5] All headline numbers are single-run point estimates. Appendix A states that validation uses one rollout per question at temperature 0.6 with top-p 0.95, and no training seeds or checkpoint-selection rule are reported. The central landscape in §5.2 — Countdown self-distillation collapsing to 41.7% at α=0 versus 56.2% base, then jumping to 86.2% at α=0.3 versus teacher 87.3%, with α=0.3 called the most efficient operating point — rests entirely on these point estimates. For the 1,024-question ID split the standard error at 86.2% is about 1.1 pp and at 41.7% about 1.5 pp; for AIME-25 (n≈30) a 3.3 pp difference is one problem, so the claimed 'α=0.7 beats teacher' is within noise. Because the paper's task-dependence story and its operating-point recommendation depend on these comparisons, I ask for multiple seeds (at least three) with mean±std, and preferably multiple validation rollouts or p
  2. [§5.4, Table 1] The conclusion that SFT cannot exploit scaffolding is confounded by per-variant hyperparameter tuning. Appendix C reports different learning rates and batch sizes for each SFT variant (e.g., Masked-SFT at 1e-4/batch 8, 100-TT at 5e-4/batch 4), with no protocol for selecting them, and no analogous hyperparameter table for the reverse-KL variants. The gap between SFT (38.5% for 100 TTs) and reverse-KL (67.25% for 100 TTs) could therefore be due to different optimization budgets or insufficient tuning of the SFT variants rather than to the objective. This is load-bearing for RQ4. Please use a matched hyperparameter budget, report the sweep, and show multiple seeds.
  3. [§5.1, Figs. 2–5; abstract] The abstract/conclusion claim that internalization success 'tracks the student's prior exposure to the task during pretraining' is an interpretive inference from two domains (GSM8K base 61.0% vs. Countdown base 56.2% in self-distillation), but the pattern is not fully consistent: in the dual-model Countdown setting, fully masked distillation improves over a 0% base student to 34.1%, and in self-distillation GSM8K α=0 (76.0%) lies well above base. The evidence supports a task-dependent effect, but the specific attribution to pretraining exposure is not tested. I suggest either softening this claim to 'correlates with base-model accuracy in the two settings studied' or providing a controlled test (e.g., varying the amount of task data in pretraining/fine-tuning).
minor comments (8)
  1. [Abstract] Grammar: 'By treating intermediate tokens as a scaffold which reasoning models use to fit over the solution tokens, We additionally vary...' — capitalize or rephrase; the clause is a fragment.
  2. [Figure 3 caption] Typo: 'distialltion' should be 'distillation'.
  3. [§5.2] Grammar: 'α=0.5 is pareto-optimal than the non-masked variant' should be 'Pareto-optimal to' or 'dominates'.
  4. [§5.3] Missing punctuation: 'slightly below base but every scaffolded variant lifts accuracy above the base model' — use a semicolon or period before 'but'.
  5. [Section 3] The text says 'In practice we report α∈{0.3,0.5,0.7}' but the results also include α=0 and α=1.0. Clarify that these are endpoints of the sweep.
  6. [Appendix B] The sentence 'the instruction prompt and statistics on the generated teaching tokens are reported in Appendix .' has a missing appendix number/reference.
  7. [Reproducibility] No code, checkpoints, or training logs are provided. For an empirical method paper, releasing these (or at least a public repository with configs) is strongly recommended.
  8. [References and notation] Minor formatting: in the Introduction the citation '[L1 [2], CoT-Valve...]' has an extra bracket; the reference for LoRA 'Iclr' should be 'ICLR'. In Eq. (1) the divergence D is used without defining its form.

Circularity Check

2 steps flagged

No significant circularity: α-sweep results are transparent measurements against externally benchmarked teacher accuracy. Two minor, non-load-bearing signals — the motivating trace/answer-decoupling premise rests only on author-overlapping citations, and the self-distillation α=1 endpoint equals the teacher by explicitly disclosed construction.

specific steps
  1. self citation load bearing [Abstract; Section 1 (Introduction), first paragraph]
    "Our recent work has shown that there is no causal connection between trace correctness and final-answer correctness [26, 13, 4], and the trace length is not connected to the computational complexity of the problem instance being solved [19]."

    The paper's foundational motivating premise — that trace correctness is causally disconnected from answer correctness — is supported only by four citations ([26],[13],[4],[19]) whose author lists overlap the present paper (Kambhampati, Palod, Kalwar, et al.). This premise is asserted as fact in the abstract and motivates the internalization research question. However, it drives framing, not results: no headline accuracy is derived from it, the α grid and teacher baselines are external measurements, and the reported Countdown α=0 collapse (41.7% below the 56.2% base) actually runs against the premise. It is therefore a minor, framing-level self-citation, not load-bearing circularity.

  2. self definitional [Section 5.1/5.2 (self-distillation); Eqs. 5–6]
    "In the self-distillation setting, since the teacher and the student are the same model, the non-masked variant reduces to running the teacher in thinking mode at inference."

    By construction, the α=1 non-masked endpoint in self-distillation shares the teacher's weights, so its reported accuracies (87.9% GSM8K, 87.3% Countdown) are definitionally the teacher's own scores, not independent student measurements. The paper is fully transparent about this reduction, and neither the Countdown α=0 failure nor the α=0.3 recovery depends on the α=1 point; the headline 'essentially matching the teacher' compares the separately trained α=0.3 student (86.2%) with an external teacher reference (87.3%), a genuine measurement. Hence this is a disclosed, non-load-bearing definitional endpoint rather than circular evidence.

full rationale

The paper's central quantitative claims — the α-sweep accuracies, the Countdown α=0 collapse (41.7% vs 56.2% base), the α=0.3 recovery (86.2%), and the ~1.3× token reduction — are direct measurements along a transparently reported grid α∈{0,0.3,0.5,0.7,1.0}. Teacher accuracies (87.9% GSM8K, 87.3% Countdown) are external reference points of the base models in thinking mode, not fitted quantities; student accuracies are separate rollouts taken after training. No parameter is fitted to a target and then reported as a prediction, and no equation reduces a headline result to its input: the training objectives (Eqs. 4–6) define the loss, and the reported accuracies are downstream measurements that could have come out differently (as the Countdown α=0 failure shows). The 'prior exposure' explanation of task dependence uses the base student's own accuracy, measured before distillation, as the exposure proxy, so it is not defined in terms of the outcome it explains. The only by-construction identity in the self-distillation setting is the α=1 endpoint, which the paper explicitly discloses; this is a transparent definitional anchor, and the α=0 and α=0.3 headline results do not rely on it. The single flagged framing choice is the motivating premise that trace correctness is not causally connected to answer correctness, asserted as fact and supported exclusively by author-overlapping citations [26,13,4,19]; this premise motivates the research question but is not load-bearing for any measured result — indeed, the Countdown results complicate it. Overall, no significant circularity; score 2 reflects two minor, non-load-bearing signals, with the central empirical derivation self-contained.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The paper is an empirical training recipe, not a derivation, so the ledger is short on fitted parameters. The hand-chosen knobs are α and the teaching-token length k, both transparently reported and swept. The load-bearing background assumptions are: (1) the contested, self-cited claim that trace correctness is not causally linked to answer correctness, which frames traces as a discardable scaffold; (2) the post-hoc attribution of task-dependence to pretraining exposure, inferred from base accuracy; and (3) the core transfer assumption that a teacher's solution-token distribution conditioned on (x, ITs) is a learnable target. No invented entities beyond the teaching-token construct.

free parameters (3)
  • α (suffix scaffold fraction) = swept {0, 0.3, 0.5, 0.7, 1.0}
    Hand-chosen knob splitting the teacher trace into internalized prefix vs emitted suffix (Eq. 6). Reported transparently as a sweep, but the 'most efficient operating point α=0.3' narrative selects from a noisy grid without error bars.
  • Teaching-token target length k = 100 / 1,000 words (actual means 80 / 389)
    Instruction-prompt word-count request for the Table 1 ablation; teacher undershoots targets (Table 2). Affects only the teaching-token variants.
  • SFT hyperparameters (LoRA r=64, scale=128; per-variant LR and batch) = chosen by sweep (LR 1e-4/5e-4, batch 4/8)
    Appendix C reports hyperparameter sweeps for the SFT ablation; these do not affect the main reverse-KL masked-distillation claim.
axioms (4)
  • domain assumption Final-answer correctness is not causally related to trace correctness (cited to [26,13,4], all from the authors' group)
    Stated in the Abstract and §1 as established fact; it licenses treating intermediate tokens as a discardable scaffold. The paper offers no independent evidence for this contested claim.
  • domain assumption Trace length is not a reliable indicator of problem complexity (cited to [19], authors' group)
    Abstract and §1; motivates why removing traces is cost-free.
  • ad hoc to paper The driver of internalization success is the student's pretraining exposure to the task, inferred from base-model accuracy
    §5.1 explains GSM8K success ('GSM8K style problems are likely well-represented in its pretraining distribution') and Countdown failure ('likely not well represented') from base accuracies (61.0%/15.69% vs 56.2%/0%). Base accuracy conflates exposure with capacity and difficulty; the dual-model GSM8K case (base only 15.69%) undercuts the exposure story.
  • domain assumption The teacher's solution-token distribution conditioned on (x, ITs) is a useful and learnable target for a student conditioned on x alone (Eq. 4)
    This is the method's core transfer assumption; the paper's experiments are the only evidence for it, so the axiom is effectively what the paper tests rather than an independent premise.
invented entities (1)
  • Teaching tokens (TTs) no independent evidence
    purpose: Teacher-generated k-word summaries (k in {100, 1000}) of its own reasoning, used as alternative scaffolding in the Table 1 ablation; produced by an instruction prompt, not a new architectural component.
    Included for completeness as a new training-signal construct; no falsifiable handle outside the paper beyond the reported accuracy numbers.

pith-pipeline@v1.3.0-alltime-deepseek · 13590 in / 25058 out tokens · 203807 ms · 2026-08-02T10:43:33.190833+00:00 · methodology

0 comments
read the original abstract

Large Reasoning Models (LRMs) produce long, explicit chains of intermediate steps before generating a final answer at inference time. These intermediate traces dominate latency, memory usage, and serving cost, even though the final answer correctness is not causally related to the trace correctness and the trace length is not a reliable indicator of the problem complexity. This raises a natural question: can the computation expressed in these intermediate tokens be internalized into the parameters of a language model, enabling it to produce answers directly (or with much shorter intermediate traces)? We introduce \textit{masked distillation}, a knowledge-distillation framework in which a student LLM is trained to predict only the solution tokens conditioned on the question, while a reasoning teacher provides feedback on the student's responses after conditioning on the question and its own CoT trace. We instantiate this framework in two settings: (i) a \textit{self-distillation} setting, in which the same model serves as the teacher in thinking mode and as the student in non-thinking mode, and (ii) a \textit{dual-model} setting, in which a larger reasoning teacher supervises a separate smaller non-thinking student over the solution tokens. By treating intermediate tokens as a scaffold which reasoning models use to fit over the solution tokens, We additionally vary the length of intermediate-token scaffolding the student is supervised on, interpolating between full internalization (the student emits only the solution) and no internalization (the student emits the full trace before the answer). We evaluate the framework through controlled experiments on two reasoning domains: GSM8K (grade-school arithmetic) and Countdown (a number-puzzle search task).

Figures

Figures reproduced from arXiv: 2607.22629 by Durgesh Kalwar, Subbarao Kambhampati, Vardhan Palod.

Figure 1
Figure 1. Figure 1: Overview of the masked distillation framework. Phase 1, intermediate-token collection (top): for each input question x ∼ D we sample a response from the teacher π T . The response has the form <think>(ITs) <\think><answer>(STs)<\answer>, from which we extract the intermediate tokens (ITs) and the solution tokens (STs). Phase 2, training (bottom): the non-thinking student π S θ is conditioned only on x, whi… view at source ↗
Figure 2
Figure 2. Figure 2: Accuracy vs. average response length on math domain (self-distillation setting) for suffix-scaffold masked [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Accuracy vs. average response length on the math domain (dual-model setting), for suffix-scaffold masked [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy vs. average response length on Countdown (self-distillation setting), for suffix-scaffold masked [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Accuracy vs. average response length on Countdown (dual model setting), for suffix-scaffold masked [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 18 linked inside Pith

  1. [1]

    On-policy distillation of language models: Learning from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. InInterna- tional Conference on Learning Representations, volume 2024, pages 21246–21263, 2024

    2024

  2. [2]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Training language models to reason efficiently, 2025.URL https://arxiv

    Daman Arora and Andrea Zanette. Training language models to reason efficiently, 2025.URL https://arxiv. org/abs/2502.04463, 2025

    arXiv 2025

  4. [4]

    Interpretable traces, unexpected outcomes: Investigating the disconnect in trace-based knowledge distillation.arXiv preprint arXiv:2505.13792, 2025

    Siddhant Bhambri, Upasana Biswas, and Subbarao Kambhampati. Interpretable traces, unexpected outcomes: Investigating the disconnect in trace-based knowledge distillation.arXiv preprint arXiv:2505.13792, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Countdown (game show)

    Countdown. Countdown (game show). https://en.wikipedia.org/wiki/Countdown_(game_show). [Ac- cessed 13-05-2025]

    2025

  7. [7]

    Implicit chain of thought reasoning via knowledge distillation.arXiv preprint arXiv:2311.01460, 2023

    Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. Implicit chain of thought reasoning via knowledge distillation.arXiv preprint arXiv:2311.01460, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    From explicit cot to implicit cot: Learning to internalize cot step by step.arXiv preprint arXiv:2405.14838, 2024

    Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step.arXiv preprint arXiv:2405.14838, 2024. 9 Preprint

    Pith/arXiv arXiv 2024

  9. [9]

    Minillm: Knowledge distillation of large language models

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. In International Conference on Learning Representations, volume 2024, pages 32694–32717, 2024

    2024

  10. [10]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    Pith/arXiv arXiv 2015

  11. [11]

    Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    2022

  12. [12]

    Reinforcement learning via self-distillation.arXiv preprint arXiv:2601.20802, 2026

    Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, et al. Reinforcement learning via self-distillation.arXiv preprint arXiv:2601.20802, 2026

    Pith/arXiv arXiv 2026

  13. [13]

    Stop anthropomorphizing intermediate tokens as reasoning/thinking traces! InNeurIPS 2025 Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning, 2025

    Subbarao Kambhampati, Kaya Stechly, Karthik Valmeekam, Lucas Paul Saldyt, Siddhant Bhambri, Vardhan Palod, Atharva Gundawar, Soumya Rani Samineni, Durgesh Kalwar, and Upasana Biswas. Stop anthropomorphizing intermediate tokens as reasoning/thinking traces! InNeurIPS 2025 Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning, 2025

    2025

  14. [14]

    Sequence-level knowledge distillation

    Yoon Kim and Alexander M Rush. Sequence-level knowledge distillation. InProceedings of the 2016 conference on empirical methods in natural language processing, pages 1317–1327, 2016

    2016

  15. [15]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InInternational Conference on Learning Representations, volume 2024, pages 39578–39601, 2024

    2024

  16. [16]

    O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Cot-valve: Length-compressible chain-of-thought tuning

    Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compressible chain-of-thought tuning. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6025–6035, 2025

    2025

  18. [18]

    American invitational mathematics examination 2025 (aime25) dataset, 2025

    Contributors on Hugging Face. American invitational mathematics examination 2025 (aime25) dataset, 2025. URLhttps://huggingface.co/datasets/math-ai/aime25

    2025

  19. [19]

    Performative thinking? the brittle correlation between cot length and problem complexity.arXiv preprint arXiv:2509.07339, 2025

    Vardhan Palod, Karthik Valmeekam, Kaya Stechly, and Subbarao Kambhampati. Performative thinking? the brittle correlation between cot length and problem complexity.arXiv preprint arXiv:2509.07339, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019

    Pith/arXiv arXiv 1910

  21. [21]

    Self-distillation enables continual learning

    Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning. arXiv preprint arXiv:2601.19897, 2026

    Pith/arXiv arXiv 2026

  22. [22]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Sample more to think less: Group filtered policy optimization for concise reasoning.arXiv preprint arXiv:2508.09726, 2025

    Vaishnavi Shrivastava, Ahmed Awadallah, Vidhisha Balachandran, Shivam Garg, Harkirat Behl, and Dimitris Papailiopoulos. Sample more to think less: Group filtered policy optimization for concise reasoning.arXiv preprint arXiv:2508.09726, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Learning by distilling context.arXiv preprint arXiv:2209.15189, 2022

    Charlie Snell, Dan Klein, and Ruiqi Zhong. Learning by distilling context.arXiv preprint arXiv:2209.15189, 2022

    Pith/arXiv arXiv 2022

  25. [25]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Beyond semantics: The unreasonable effectiveness of reasonless intermediate tokens.arXiv preprint arXiv:2505.13775, 2025

    Karthik Valmeekam, Kaya Stechly, Vardhan Palod, Atharva Gundawar, and Subbarao Kambhampati. Beyond semantics: The unreasonable effectiveness of reasonless intermediate tokens.arXiv preprint arXiv:2505.13775, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Qwen2 technical report, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei...

    Pith/arXiv arXiv 2024

  28. [28]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Chain of thought imitation with procedure cloning.Advances in Neural Information Processing Systems, 35:36366–36381, 2022

    Mengjiao Sherry Yang, Dale Schuurmans, Pieter Abbeel, and Ofir Nachum. Chain of thought imitation with procedure cloning.Advances in Neural Information Processing Systems, 35:36366–36381, 2022. 11 Preprint A Hyperparameter details In our masked-distillation experiments, the student uses a maximum prompt length of 2,048 tokens and a maximum response length...

    2022