Pith. sign in

REVIEW 4 major objections 5 minor 28 references

DART: Distilling Autoregressive Reasoning to Silent Thought

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DART distills chain-of-thought reasoning into silent thought tokens, achieving the best non-autoregressive accuracy on GSM8K with only about 1 ms extra latency.

desk verdict A solid, honest non-autoregressive reasoning paper with a useful new method, a narrow validation, and a clear need for more controlled baselines and longer-chain tests. read the letter →

arxiv 2506.11752 v2 pith:GYU3ZTFY submitted 2025-06-13 cs.CL

Nan Jiang , Ziming Wu , De-Chuan Zhan , Fuming Lai , Shaobing Lian This is my paper · ORCID
classification cs.CL
keywords non-autoregressivereasoningsilentthoughtself-distillationchain-of-thoughthidden-statealignmentefficientinferencemathLLMfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Chain-of-thought reasoning makes LLMs accurate but slow because every reasoning step is generated one token at a time. DART proposes to distill that step-by-step process into a short sequence of silent thought tokens that the model processes in a single non-autoregressive pass, producing the answer directly. The authors show that aligning the hidden state of the last separator token between the chain-of-thought and silent-thought pathways transfers most of the reasoning gain. On GSM8K, DART beats all other non-autoregressive baselines by 10.1 accuracy points at essentially the same inference latency as direct answering. A sympathetic reader would take this as evidence that implicit, non-autoregressive reasoning can be a practical alternative to explicit chain-of-thought.

What carries the argument

The central mechanism is the Reasoning Evolvement Module (REM), a plug-in set of low-rank key and value projection modifications paired with a distillation loss. At each decoder layer, REM adapts the attention keys and values for the sequence consisting of the question and the silent tokens, and the distillation loss forces the hidden state of the last separator token in the silent-thought pathway to match that of the chain-of-thought pathway. This makes the silent tokens carry an approximation of the hidden-state shift that the chain-of-thought intermediate tokens would have produced. The shared separator tokens, fixed to the answer prompt, give the alignment a stable anchor.

What would settle it

A decisive check: train DART on GSM8K-Aug but attach the distillation loss to a random token instead of the final separator $z_N$, then evaluate on GSM-HARD. If the 10.1-point GSM8K gain persists, the stated mechanism is not the active one; if the out-of-distribution accuracy collapses to the No-CoT level, then aligning only $z_N$ is insufficient to generalize.

Watch

Extended reading notes

Core claim

The paper claims that an LLM can internalize autoregressive chain-of-thought reasoning into non-autoregressive silent thought by training two pathways at once. In the CoT pathway the model predicts the intermediate reasoning steps and the answer; in the silent thought pathway the model receives a fixed number of special tokens after the question and produces only the answer. A lightweight Reasoning Evolvement Module inserted at each decoder layer rewrites the attention key and value projections so that the hidden state of the last separator token in the silent pathway converges to the corresponding hidden state in the CoT pathway. Once trained, only the silent pathway is active at inference, and the silent tokens evolve into informative embeddings as they pass through the layers. The authors report the best accuracy among non-autoregressive baselines on GSM8K, consistent gains on out-of-distribution math benchmarks, and improvements across several decoder-only model sizes.

Load-bearing premise

The load-bearing premise is that matching the hidden state of the single final separator token between the chain-of-thought and silent-thought pathways transfers the whole reasoning process; the paper's own limitations section acknowledges that this one-position supervision may miss information in intermediate tokens.

Editorial extensions

If this is right

  • If DART works as claimed, an LLM can answer reasoning questions in roughly the same wall-clock time as direct answering while keeping a large share of chain-of-thought accuracy.
  • Since no autoregressive reasoning tokens are emitted during inference, per-query compute on math benchmarks drops to the cost of one forward pass over the question and silent tokens.
  • The method transfers across model scales, with accuracy gains reported on small and larger decoder-only models, not just the primary 1-billion-parameter model.
  • Because only the silent pathway is used at inference, DART can be added to existing decoder-only LLMs as a fine-tuned plug-in with a small number of additional parameters.
  • Gains on out-of-distribution benchmarks such as GSM-HARD, SVAMP, and MultiArith suggest the distilled reasoning behavior generalizes beyond the exact training distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test this, but the same single-position alignment should transfer to other tasks that have a stable separator token before the answer; if so, DART becomes a general recipe for silent reasoning rather than a math-specific trick.
  • Because the decoded silent tokens mostly match ground-truth chain-of-thought words when the answer is correct, DART could be used as a probe: the interpretability of silent tokens may serve as a diagnostic for whether distilled reasoning is actually being used rather than memorized.
  • A natural pressure test, which the paper names but does not run, is to generate chain-of-thought data with a larger model and distill it into a smaller one; if the gain survives, DART could act as a reasoning-preserving model-compression tool.
  • The reported latency is single-query latency on one GPU; an extension to batched serving and throughput would clarify whether the near-zero overhead persists at scale.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DART, a self-distillation framework that trains a decoder-only LLM with two pathways: a chain-of-thought (CoT) pathway and a silent-thought (ST) pathway in which a fixed number of <st> tokens are inserted before an answer prompt. A lightweight Reasoning Evolvement Module (REM) adapts the key and value projections, and an L1 distillation loss aligns the hidden state of the last separator token between the two pathways. At inference, only the ST pathway is used, so no autoregressive intermediate tokens are generated. Experiments on GSM8K, GSM-HARD, SVAMP, MultiArith, ProsQA, and CommonsenseQA-CoT, plus results with GPT2, Qwen2.5-1.5B, and Llama-3.2-3B, report accuracy and latency. The paper includes ablations, a sensitivity study of the number of ST tokens, a qualitative analysis of decoded ST tokens, and a Limitations section.

Significance. If the reported results are reliable, DART offers a simple and potentially practical way to compress chain-of-thought reasoning into a fixed number of non-autoregressive latent tokens, with a favorable accuracy-latency trade-off on short-chain benchmarks. The REM module is a clean plug-in design with few additional parameters, and the ablations help localize the contribution of each component. The authors also make an honest attempt at out-of-distribution evaluation and include multiple base models. However, the evidence currently supports only short-chain arithmetic and commonsense tasks; the claimed generality of the approach, and the exact position of DART relative to continuous-thought baselines such as CODI and Coconut, are not fully established by the experiments as presented.

major comments (4)
  1. [§4.2, Table 1] The sentence "DART achieves the best performance among all NAR baselines on GSM8K" is not decidable from the manuscript as rendered. In §2, Coconut and CODI are described as limited by the autoregressive pattern, which implies they are not NAR baselines, yet Table 1 lists them among the compared methods and its "Is NAR?" column carries no visible values in the text. If Coconut and CODI are counted as non-autoregressive, DART's 42.6% is below Coconut's 50.6% and CODI's 55.6%, contradicting the claim. If they are excluded, the claim should say so explicitly, because the abstract's phrase 'existing non-autoregressive baselines' is then misleading. Please clarify the taxonomy and the exact scope of the claim.
  2. [§4.2, Table 1] The CODI and iCoT accuracies are marked as taken from Shen et al. (2025) rather than re-run under the same training conditions, and no error bars or multiple-seed results are reported for any method. Since the central quantitative claim is a 10.1-point gain over No-CoT, a single run without variance cannot establish that this difference is significant. Please report multiple seeds with standard deviations, or at least state clearly that the comparison is a single-run qualitative indicator. If the CODI and iCoT checkpoints were not produced with the same base model, dataset, and training epochs as DART, the comparison in Table 1 is not controlled.
  3. [§3.2, Limitations] The only distillation signal used to transfer reasoning is L_distill on the hidden state of the last separator token z_N. The paper's own Limitations section concedes that this single-position alignment may overlook information in intermediate tokens. No experiment tests tasks with longer reasoning chains: all math benchmarks have short CoT traces of roughly two to five arithmetic steps, and ProsQA and CommonsenseQA-CoT are similarly short. Since the introduction and abstract frame DART as a general alternative to CoT for complex tasks, the sufficiency of single-position alignment for longer chains should be demonstrated, for example with a dataset such as MATH or with a breakdown of GSM8K by number of reasoning steps. Without such evidence, the generalization claim is not supported by the current experiments.
  4. [Figure 2, §4.2] The sensitivity analysis for the number of ST tokens C is performed on GSM8K, which is the same benchmark used for the main comparison and for reporting the headline accuracy. This risks selecting hyperparameters on the test set. The analysis should be conducted on a held-out development set or clearly described as model selection rather than as evidence of robustness. The same concern applies to the fixed choices λ = 20, d = 128, and α = 32, which are reported without sensitivity checks.
minor comments (5)
  1. [Throughout] There are several typos: 'SV AMP' should be 'SVAMP' in Table 1 and §4.1; 'DARt' appears in §4.2; and 'Ldistll' appears in the overall objective LDART = LCoT + LST + λLdistll instead of 'Ldistill'.
  2. [Appendix A] The derivation drops the softmax and scaling factor and then continues as if the simplified expression were exact. Please state explicitly that this is a rough approximation intended only to motivate the REM design, and that no formal error bound is claimed.
  3. [§4.3, Qualitative Analysis] The statement that 69.9% of translated ST tokens match ground-truth CoT words needs a precise definition of 'match': exact token identity, case-insensitive, stemmed, and whether the comparison is position-wise or set-based. Without this definition the number is hard to interpret.
  4. [§4.2, Table 4] The CoT inference time on CommonsenseQA-CoT (1471 ms) is much larger than on ProsQA (882 ms) despite similar benchmark sizes; a brief description of the decoding protocol (e.g., max new tokens, batch size, and whether times include prompt processing) would help the reader interpret the latency numbers.
  5. [Appendix C] The implementation details say that the configuration is 'consistent with the configuration used for LoRA' to fine-tune the CoT pathway, but it is not clear which parameters are trainable in each pathway and whether REM replaces or supplements the LoRA adaptation. Please specify the LoRA rank, target modules, and the exact role of REM relative to LoRA for the ST pathway.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: DART's central accuracy claims rest on external held-out benchmarks and standard baselines; the ST-token 'evolution' is a supervised training effect reported as an empirical observation, not a derived prediction.

full rationale

DART's load-bearing empirical claims — best non-autoregressive accuracy on GSM8K (42.6% vs. 32.5% No-CoT, a 10.1% gain) and consistent out-of-distribution gains on GSM-HARD, SVAMP, and MultiArith — are evaluated on standard held-out test splits (GSM8K test set of 1319 examples; the OOD sets are used only for evaluation), so the central result is self-contained against external data and baselines. The distillation loss L_distill is a training objective, not a renamed prediction: the paper openly states that by aligning the z_N hidden states, 'the function gθ1:l,ϕ1:l([Q; X]) is encouraged to approximate gθ1:l(hθ(Q)), thereby distilling the reasoning capability from the CoT pathway into the ST pathway', and the ablation study shows that dropping L_distill collapses accuracy to 33.7 (near the 32.5 No-CoT floor), confirming the mechanism is the supervised objective itself rather than an independently derived quantity. The theoretical motivation (Appendix A) is an acknowledged approximation of the external Dai et al. (2023) result; it motivates the REM design and does not feed back into the evaluation. There is no load-bearing self-citation chain: none of the central references (Dai et al. 2023; Hao et al. 2024; Shen et al. 2025; Deng et al. 2024) are authored by the present paper's authors, and no uniqueness argument is imported from prior author work. The Limitations section (flagged per review instruction) concedes that aligning only the last separator's activation 'may be suboptimal, as it can overlook some information in intermediate tokens'; this is an honest generalization/completeness risk given the short reasoning chains in the benchmarks, but it is a correctness concern, not a circularity. The only mildly circular-adjacent element is interpretive: the qualitative claim that ST tokens 'evolve into informative embeddings' (Section 4.3, with 69.9% of translated tokens matching ground-truth CoT words) largely documents the effect of the supervised alignment objective itself. However, the paper frames this as an empirical post-hoc observation rather than as a prediction or first-principles derivation, and the central benchmark claims are evaluated externally, so this does not meet the hard-evidence bar for a circular step. Overall, DART is a supervised self-distillation method whose value is established by independent test-set comparisons, not by definition.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on hyperparameters (C, alpha, d, lambda) chosen by hand, and on the assumed sufficiency of the single-position hidden-state alignment and the expressiveness of the REM low-rank form. The invented components (ST tokens and REM) are evaluated only through the paper's own experiments, not through external evidence.

free parameters (4)
  • C (number of ST tokens) = 20
    Number of silent-thought tokens inserted between question and answer; chosen by hand and validated by sensitivity analysis on GSM8K (Figure 2), which is also the main in-distribution test set.
  • alpha (REM scaling) = 32
    Scaling hyperparameter in the REM low-rank update; set to 32 based on configuration used for LoRA (Appendix C).
  • d (REM projection dimension) = 128
    Dimension of REM low-rank matrices; set to 128, consistent with the LoRA configuration (Appendix C).
  • lambda (distillation loss weight) = 20
    Trade-off weight for L_distill; chosen because the distillation loss is much smaller than the other two losses (Appendix C).
assumptions (4)
  • domain assumption The softmax attention operation can be omitted in the approximate shift analysis, yielding a linear approximation of the effect of intermediate tokens on the last separator's hidden state.
    Used in Appendix A to derive the shift formula that motivates the REM design; the paper does not validate this approximation empirically.
  • domain assumption Aligning the hidden state of the last separator token z_N between the CoT and ST pathways is sufficient to transfer the reasoning capability.
    The distillation loss only supervises this single position; the paper's Limitations section acknowledges this may be suboptimal.
  • domain assumption Fixed special <st> tokens can carry evolving reasoning information across decoder layers when trained with the alignment loss.
    Empirical claim supported only by the paper's own qualitative decoding analysis; no independent evidence is provided.
  • ad hoc to paper The REM low-rank perturbation form W_R2 W_R1^T + I can express the shift introduced by CoT intermediate tokens.
    The specific functional form is chosen by the authors; no theorem guarantees it can represent the required shift.
invented entities (2)
  • Silent Thought (ST) tokens (<st>)
    purpose: Fixed special tokens concatenated after the question to provide a non-autoregressive latent reasoning buffer during inference.
    The paper shows they improve accuracy, but this evidence is generated by the same framework; no external evidence exists.
  • Reasoning Evolvement Module (REM)
    purpose: Layer-wise low-rank modification of key/value projections that aligns ST hidden states with CoT hidden states.
    Ablations show it helps, but this is internal to the paper; no independent verification is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DART: Distilling Autoregressive Reasoning to Silent Thought." pith.science (2026). https://pith.science/paper/GYU3ZTFY

@misc{pith2026250611752,
  author       = {Pith},
  title        = {Pith review of: DART: Distilling Autoregressive Reasoning to Silent Thought},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GYU3ZTFY}},
  note         = {Machine review of arXiv:2506.11752}
}
read the original abstract

Chain-of-Thought (CoT) reasoning has significantly advanced Large Language Models (LLMs) in solving complex tasks. However, its autoregressive paradigm leads to significant computational overhead, hindering its deployment in latency-sensitive applications. To address this, we propose \textbf{DART} (\textbf{D}istilling \textbf{A}utoregressive \textbf{R}easoning to Silent \textbf{T}hought), a self-distillation framework that enables LLMs to replace autoregressive CoT with non-autoregressive Silent Thought (ST). Specifically, DART introduces two training pathways: the CoT pathway for traditional reasoning and the ST pathway for generating answers directly from a few ST tokens. The ST pathway utilizes a lightweight Reasoning Evolvement Module (REM) to align its hidden states with the CoT pathway, enabling the ST tokens to evolve into informative embeddings. During inference, only the ST pathway is activated, leveraging evolving ST tokens to deliver the answer directly. Extensive experimental results demonstrate that DART offers significant performance gains compared with existing non-autoregressive baselines without extra inference latency, serving as a feasible alternative for efficient reasoning.

Figures

Figures reproduced from arXiv: 2506.11752 by the authors.

Figure 1
Figure 1. Overall Framework of DART. During inference, we employ the ST pathway to respond directly without [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Accuracy and inference time on GSM8K with [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 8 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Jeffrey Cheng and Benjamin Van Durme. 2024. https://doi.org/10.48550/ARXIV.2412.13171 Compressed chain of thought: Efficient reasoning through dense representations . CoRR, abs/2412.13171

    arXiv doi:10.48550/arxiv.2412.13171 2024

  4. [4]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . CoRR, abs/2110.14168

    arXiv 2021

  5. [5]

    Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. 2023. https://doi.org/10.18653/v1/2023.findings-acl.247 Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers . In Findings of the Association for Computational Linguistics, pages 4005--4019

    doi:10.18653/v1/2023.findings-acl.247 2023

  6. [6]

    DeepSeek - AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 81 others. 2025. https://doi.org/10.48550/ARXIV.2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinfor...

    arXiv doi:10.48550/arxiv.2501.12948 2025

  7. [7]

    Yuntian Deng, Yejin Choi, and Stuart M. Shieber. 2024. https://doi.org/10.48550/ARXIV.2405.14838 From explicit cot to implicit cot: Learning to internalize cot step by step . CoRR, abs/2405.14838

    arXiv doi:10.48550/arxiv.2405.14838 2024

  8. [8]

    Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart M. Shieber. 2023. https://doi.org/10.48550/ARXIV.2311.01460 Implicit chain of thought reasoning via knowledge distillation . CoRR, abs/2311.01460

    arXiv doi:10.48550/arxiv.2311.01460 2023

Show all 28 references
  1. [9]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Z...

    2024 doi arXiv

  2. [10]

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/dfc310e81992d2e4cedc09ac47eff13e-Abstract-Conference.html Towards revealing the mystery behind chain of thought: A theoretical perspective . In Adv...

    2023

  3. [11]

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. https://proceedings.mlr.press/v202/gao23f.html PAL: program-aided language models . In International Conference on Machine Learning, volume 202, pages 10764--10799

    2023

  4. [12]

    Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. 2024. https://openreview.net/forum?id=ph04CRkPdC Think before you speak: Training language models with pause tokens . In The Twelfth International Conference on Learning Rep...

    2024

  5. [13]

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. https://doi.org/10.48550/ARXIV.2412.06769 Training large language models to reason in a continuous latent space . CoRR, abs/2412.06769

    2024 doi arXiv

  6. [14]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representations

    2022

  7. [15]

    Zhiyuan Liu, Hong Liu, Denny Zhou, and Tengyu Ma. 2024. https://openreview.net/forum?id=3EWTEy9MTM Chain of thought empowers transformers to solve inherently serial problems . In The Twelfth International Conference on Learning Representations

    2024

  8. [16]

    Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net

    2019

  9. [17]

    OpenAI. 2025. https://openai.com/index/learning-to-reason-with-llms/ Learning to reason with llms

    2025

  10. [18]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. https://doi.org/10.18653/v1/2021.naacl-main.168 Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling...

    2021 doi

  11. [19]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners

    2019

  12. [20]

    Subhro Roy and Dan Roth. 2015. http://arxiv.org/abs/1608.01413 Solving general arithmetic word problems . In Proceedings of the 2015 Conference on Empirical Methods in NaturalLanguage Processing, pages 1743--1752

    2015 arXiv

  13. [21]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. https://doi.org/10.48550/ARXIV.2402.03300 Deepseekmath: Pushing the limits of mathematical reasoning in open language models . CoRR, abs/2402.03300

    2024 doi arXiv

  14. [22]

    Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. 2025. https://doi.org/10.48550/ARXIV.2502.21074 CODI: compressing chain-of-thought into continuous space via self-distillation . CoRR, abs/2502.21074

    2025 doi arXiv

  15. [23]

    Yang Sui, Yu - Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Ben Hu. 2025. https://doi.org/10.48550/ARXIV.2503.16419 Stop overthinking: A survey on efficient reasoning for large language models . ...

    2025 doi arXiv

  16. [24]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  17. [25]

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. https://doi.org/10.18653/v1/2024.acl-long.510 Math-shepherd: Verify and reinforce llms step-by-step without human annotations . In Proceedings of the 62nd Annual Meeting ...

    2024 doi

  18. [26]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning...

    2022

  19. [27]

    Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. https://openreview.net/forum?id=N8N0hgNDRt Metamath: Bootstrap your own mathematical questions for large language models . In The Twelfth I...

    2024

  20. [28]

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. https://openreview.net/forum?id=yLClGs770I Mammoth: Building math generalist models through hybrid instruction tuning . In The Twelfth International Conference on Learning Representations

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.