Pith. sign in

REVIEW 4 major objections 5 minor 82 references

The paper claims that continuous latent reasoning becomes strong and efficient when each latent token is supervised by a lossless, position-weighted superposition of a span of discrete reasoning subwords, and supports the claim with proofs

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 13:37 UTC pith:FAP2ZORH

load-bearing objection Solid, empirically strong latent-reasoning paper with clean losslessness theory; the parallel-search claim rests on an existence proof that overreaches, but the central method and results hold up. the 4 major comments →

arxiv 2607.18264 v1 pith:FAP2ZORH submitted 2026-05-19 cs.AI cs.CLcs.LG

MUX: Continuous Reasoning via Multiplexed Tokens

Ayhan Suleymanzade , Halil Alperen Gozeten , Michael Bronstein , \.Ismail \.Ilkan Ceylan , Jinwoo Kim This is my paper
classification cs.AI cs.CLcs.LG
keywords latent reasoningcontinuous reasoningmultiplexingsuperpositionchain-of-thought distillationparallel searchtoken-level supervisionlanguage models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that language models can reason more strongly and cheaply in a continuous latent space if each latent "thought" token is trained to encode a complete span of discrete reasoning subwords, rather than a single subword. The encoding is a position-weighted linear mixture of the subwords' one-hot vectors, chosen so that the mixture is lossless: the original span can be recovered exactly. The paper proves that this lossless multiplexing prevents latent tokens from collapsing into uninformative placeholders, and that multiplexed tokens can carry an entire search frontier at once. Across 32 evaluation settings on four language models, the method outperforms other latent-reasoning approaches and, in 15 cases, also beats the discrete chain-of-thought it was distilled from. A sympathetic reader would therefore see the paper as making the case that lossless superposition of discrete reasoning spans is a sufficient local training target for strong continuous reasoning.

Core claim

MUX's proposal is to supervise each latent token by mux(ri), a normalized position-weighted sum of the one-hot encodings of the subwords in the aligned span. The target is lossless whenever the weights satisfy the subset-sum separation property — no two distinct position subsets have the same total weight — and the paper proves this is exactly equivalent to injectivity of the span-to-simplex map. Geometric, sinusoidal, and rotary weightings satisfy the condition for suitable hyperparameters, so every span can in principle be demultiplexed. The paper then shows that local KL distillation to such diverse targets prevents latent collapse, and that a sequence of tokens can implement breadth-firs

What carries the argument

The load-bearing object is the multiplexed target mux(ri): a probability vector over the vocabulary formed as a weighted average of one-hot encodings, with position weights chosen so that subset sums never collide. The same construction does three jobs: it makes each local supervision target lossless, thereby preventing latent collapse through target diversity; it makes decoded tokens interpretable through the language model's output head; and, because a superposition can represent a set, it lets a single token encode an entire breadth-first search frontier. The training loop maps each latent token to the vocabulary simplex through a linear-softmax head and minimizes KL divergence to the mul

Load-bearing premise

The load-bearing premise is that ordinary gradient training on multiplexed targets will make the model implement the hand-coded breadth-first-search recurrence the paper constructs; the paper proves the recurrence exists, not that learning finds it.

What would settle it

If removing the subset-sum separation property (for example, using uniform positional weights) produced accuracy and decoding fidelity equal to the lossless weightings across the same 32 settings, then losslessness would not be the active ingredient behind the method's success.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Latent reasoning can be supervised without auxiliary autoregressive decoders or compressed cache targets; a KL divergence to multiplexed targets suffices.
  • Individual latent tokens remain human-readable: reading the vocabulary distribution recovers the aligned reasoning span, which the paper verifies by probing.
  • The non-collapse guarantee means local distillation to lossless targets avoids the shortcut behavior associated with trajectory-level supervision.
  • A model whose latent tokens are supervised on serial discrete traces can still represent and expand an entire search frontier in one token, so parallel search need not be taught explicitly.
  • Because six latent tokens replace the full discrete trace, inference uses roughly 2.4 to 5.9 times fewer generated tokens than chain-of-thought while matching or exceeding its accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: the same subset-sum separation criterion could be applied to other discrete outputs, such as code or structured data, where one lossless mixture could carry multiple candidate parses in parallel.
  • The theoretical breadth-first-search construction uses hand-coded indicator vectors rather than the learned multiplexed targets; a stronger test of the paper's narrative would train on BFS traces and check directly whether intermediate activations match the constructed frontier/visited recurrence.
  • The losslessness guarantee is tied to vocabulary size and numerical precision, so for very long spans the separation margin shrinks; the method's practical ceiling may be set by finite-precision arithmetic rather than by the learning objective.
  • If losslessness is the active ingredient, then deliberately introducing small controlled collisions into the targets should degrade performance predictably; this would turn the paper's sufficient-condition claim into a testable causal claim.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MUX, a method for continuous latent reasoning in which each latent token is trained, via local KL distillation, to match a position-weighted linear superposition of one-hot encodings of a span of discrete reasoning tokens. The authors prove that several positional weightings (geometric, sinusoidal, rotary) yield lossless multiplexing under a subset-sum separation condition, that local distillation with diverse targets prevents latent collapse, and that multiplexed tokens can in principle carry an entire BFS frontier. The empirical section evaluates MUX on mathematical reasoning (32 settings across four LMs and two training corpora), on two search tasks (MNNS and a Game-of-24 variant), and includes ablations and probing analyses. The paper claims that lossless superposition as a local learning target is a sufficient condition for strong, efficient latent reasoning, and that parallel search can emerge from serial supervision via multiplexing.

Significance. If the empirical findings hold, MUX is a conceptually simple and practically attractive alternative to global-supervision latent reasoning methods and to local methods that require auxiliary decoders or KV-cache distillation. The theoretical results on lossless multiplexing (Propositions 3, 5, and 9) are clean and the proofs in Appendix C.1 are careful; the finite-precision analysis in Appendix C.2 is a welcome addition. The paper also provides reproducible code, detailed hyperparameter tables, and a broad set of ablations and interpretability probes. These are genuine strengths. The main caveat is that the theoretical bridge to parallel search is an expressivity construction, not a learnability theorem, and the paper's central claim overreaches at exactly that point.

major comments (4)
  1. [Section 4.3, Proposition 10 and Section 5.2, Table 3] Proposition 10 constructs continuous tokens x_k=(f_k,u_k) in {0,1}^{2|N|} that exactly track BFS frontiers. This is an existence proof over hand-coded indicator pairs, not over the mux(r_k) targets that MUX actually supervises with. The dimension requirement 2|N| is not met in the search experiments (d=32, while the state spaces are much larger), and no argument shows that gradient training under L_local+L_answer learns this recurrence. Table 3 and Figure 6b show that MUX tokens carry frontier information, but do not verify the exact BFS update mechanism. The sentence 'this result implies that parallel search can naturally emerge from serial supervision via multiplexing' overstates what is proven. Please reframe Proposition 10 as an expressivity/sufficiency result and add an explicit caveat, or provide a learnability experiment that tests whether the recurrence is actually learned.
  2. [Appendix C.2 and Table 9] The finite-precision analysis certifies losslessness for geometric weights with rho=0.9 only up to span length S=11. However, Table 9 shows that the experiments use sinusoidal weighting for GPT-2 and LLaMA 1B on GSM8K-AUG, and geometric weighting elsewhere. For sinusoidal weights, no finite-precision margin E(alpha) is computed, and the statement 'for all span lengths faced in experiments (S<=11)' is not substantiated in the appendix. Please either compute the separation margin for all used weightings and span lengths actually encountered, or restrict the finite-precision claim to the geometric case and state the remaining cases as unverified.
  3. [Section 5.3, Table 4 and Figure 4] The comparison between lossless weightings and uniform weighting shows only a modest gap (e.g., 57.2 vs. 54.2 ID accuracy for geometric vs. uniform). The text attributes this to the possibility that subword order can often be inferred from bag-of-subwords in the GSM8K traces. This is a reasonable post-hoc explanation, but it weakens the paper's emphasis on losslessness as a necessary condition. The paper should acknowledge more directly that the empirical benefit of losslessness over uniform weighting is small in this benchmark, and that the theoretical arguments establish sufficiency rather than necessity.
  4. [Appendix A.5, Table 5] Table 5 is a self-assessment table: the favorable/unfavorable marks for each method are assigned by the authors, and the claim that 'MUX is the only method in this comparison that satisfies all five properties' depends on those subjective assignments. For example, marking CODI as 'not lossless' and KaVa as 'lossy' is reasonable but not a formal result, and 'train eff.' and 'interpretable' are not crisp criteria. Please present Table 5 as a positioning summary, not as evidence, and consider adding a caveat that the properties are author-defined.
minor comments (5)
  1. [Section 3.1] The symbol K is used both for the total number of latent tokens and for the subset of latent-token positions with nonempty span. This is confusing; please use a different symbol, e.g., K_active or K_nonempty.
  2. [Equation (2)] The notation alpha^(i)_j is defined via a normalization of w_j, but the later propositions state results for alpha without indicating the dependence on span length S_i. Clarify that E(alpha) is defined for each span length separately.
  3. [Table 1] The caption says 'We did not conduct iCoT/Coconut OOD tests on NL due to their low ID scores.' This is stated in the caption but not in the main text; please move this methodological note into the setup paragraph.
  4. [Appendix C.1, Lemma 11] The proof of Lemma 11 uses Rolle's theorem on the entire real line. This is valid because exponential polynomials are differentiable everywhere, but a brief justification would help readers.
  5. [Section 5.2, Game of 24] The Game of 24 benchmark is introduced as 'new' but the reference to Yao et al. (2023) is given only later. Please state explicitly that this is a new sequential left-to-right variant, since the original Game of 24 allows arbitrary parenthesization.

Circularity Check

0 steps flagged

No significant circularity found.

full rationale

MUX's derivation chain is largely self-contained. Eq. (4) defines a new supervision target mux(r_i) from discrete reasoning spans, and Propositions 3 and 5 characterize losslessness through an independent subset-sum separation condition; these are mathematical results, not restatements of the training objective. The main empirical claims are held-out accuracies on GSM8K/SVAMP/GSM-Hard/MultiArith and search benchmarks (Tables 1-3), which are external to target construction. Proposition 10 is an expressivity construction: it hand-codes continuous tokens as BFS-state indicators and verifies the BFS update, so it does not prove that gradient-trained MUX on mux(r_k) targets implements BFS. That is an unproven learnability bridge and a correctness concern, not a circularity. Similarly, the interpretability metrics compare decoded tokens to the spans used to build mux targets, so high Recall@5/Step Alignment are partly by construction, but the central comparative claim does not rest on those metrics. I found no load-bearing self-citation, no imported uniqueness theorem, and no ansatz justified only by prior author work. The limitations section acknowledges finite-precision and interpretability caveats. Overall, the paper is not circular in its central derivation; the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

No physical entities are introduced. The ledger shows the method relies on several hand-chosen hyperparameters and, critically, on an unproven expressivity bridge between the theoretical BFS construction and the actual trained architecture.

free parameters (6)
  • latent token count K = 6 (sequential), 24 (Jacobi)
    Number of continuous reasoning tokens; fixed across models and affects capacity/efficiency.
  • geometric decay rho = 0.9
    Controls subset-sum separation for geometric weighting; chosen by hand.
  • sinusoidal scale lambda = 1.0
    Positional score scale for sinusoidal weighting; chosen by hand.
  • softmax temperature tau = 1.0
    Temperature of the linear-softmax decoder used for local distillation.
  • loss weights beta, gamma = beta=1; gamma=1 (GPT-2), gamma=20 (LLaMA)
    Balance local and global distillation losses; selected per model.
  • chunking strategy = random
    How M>K spans are merged; random resampling was selected after ablations showed it performed best.
axioms (6)
  • standard math Exact-arithmetic subset-sum separation suffices for spanwise injectivity of weighted one-hot sums.
    Proposition 3 proves this directly using elementary counting; no unproved background beyond set arithmetic.
  • standard math Rational root theorem, Rolle's theorem, Pinsker's inequality, and softmax Jacobian bounds.
    Used in Propositions 5, 9, and Lemma 11; standard background results.
  • standard math Standard unit-roundoff floating-point model (Goldberg, Higham).
    Used in Appendix C.2 for finite-precision bounds.
  • domain assumption Discrete reasoning traces (q,r,a) are available and can be chunked into aligned spans with M <= K.
    The method depends on distillation from explicit reasoning traces; used throughout Section 3.
  • domain assumption The pretrained unembedding W has enough rank/expressivity to realize multiplexed targets via softmax(Wx/tau).
    Equation (3) assumes the linear-softmax decoder can approach the simplex targets; the paper does not analyze representability.
  • ad hoc to paper The transformer can implement the BFS recurrence of Proposition 10 with d=32 latent tokens under the MUX training objective.
    Proposition 10 uses dimension >= 2|N| and hand-coded indicators, which the experiments do not satisfy; bridging this gap is assumed rather than proved.

pith-pipeline@v1.3.0-alltime-deepseek · 34781 in / 13650 out tokens · 159417 ms · 2026-08-02T13:37:11.680514+00:00 · methodology

0 comments
read the original abstract

Language models solve complex problems by articulating intermediate reasoning steps in natural language. While effective, this process is computationally bottlenecked: each reasoning step conveys only a single subword, and many are spent expressing a thought instead of carrying out computation. We propose MUX, a simple method for high-bandwidth and compact reasoning based on distillation of discrete reasoning into continuous multiplexed tokens in a latent space. Here, each latent token is trained to represent a weighted linear superposition (multiplexing) of a span of discrete reasoning subwords, where this superposition is lossless by construction and the span can be fully recovered (demultiplexing). We prove that simple position-dependent weightings, such as suitable geometric decay, support lossless multiplexing, which in turn prevents shortcut behaviors caused by latent collapse. We further show that multiplexed reasoning can perform parallel exploration in problems that require search. Across 32 evaluation settings spanning four language models, MUX outperforms strong latent reasoning baselines. Ablation and probing analyses further show that the learned latent tokens encode faithful and interpretable reasoning. Our results suggest that lossless superposition as local learning targets constitutes a sufficient condition for achieving strong and efficient latent continuous reasoning.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 21 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2112.00114 , year=

    Show Your Work: Scratchpads for Intermediate Computation with Language Models , author=. arXiv preprint arXiv:2112.00114 , year=

    Pith/arXiv arXiv

  2. [2]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    Star: Bootstrapping reasoning with reasoning , author=. Advances in Neural Information Processing Systems , volume=

  4. [4]

    The Eleventh International Conference on Learning Representations , year=

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. The Eleventh International Conference on Learning Representations , year=

  5. [5]

    The Eleventh International Conference on Learning Representations , year=

    Least-to-Most Prompting Enables Complex Reasoning in Large Language Models , author=. The Eleventh International Conference on Learning Representations , year=

  6. [6]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  7. [7]

    arXiv preprint arXiv:2311.01460 , year=

    Implicit chain of thought reasoning via knowledge distillation , author=. arXiv preprint arXiv:2311.01460 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2405.14838 , year=

    From explicit cot to implicit cot: Learning to internalize cot step by step , author=. arXiv preprint arXiv:2405.14838 , year=

    Pith/arXiv arXiv

  9. [9]

    The Twelfth International Conference on Learning Representations , year=

    Think before you speak: Training Language Models With Pause Tokens , author=. The Twelfth International Conference on Learning Representations , year=

  10. [10]

    arXiv preprint arXiv:2412.13171 , year=

    Compressed chain of thought: Efficient reasoning through dense representations , author=. arXiv preprint arXiv:2412.13171 , year=

    Pith/arXiv arXiv

  11. [11]

    Second Conference on Language Modeling , year=

    Training Large Language Models to Reason in a Continuous Latent Space , author=. Second Conference on Language Modeling , year=

  12. [12]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Softcot: Soft chain-of-thought for efficient reasoning with llms , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  13. [13]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Codi: Compressing chain-of-thought into continuous space via self-distillation , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  14. [14]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Parallel continuous chain-of-thought with jacobi iteration , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  15. [15]

    Xilin Wei and Xiaoran Liu and Yuhang Zang and Xiaoyi Dong and Yuhang Cao and Jiaqi Wang and Xipeng Qiu and Dahua Lin , booktitle=

  16. [16]

    KaVa: Latent Reasoning via Compressed

    Anna Kuzina and Maciej Pi. KaVa: Latent Reasoning via Compressed. The Fourteenth International Conference on Learning Representations , year=

  17. [17]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  18. [18]

    The Fourteenth International Conference on Learning Representations , year=

    Continuous Chain of Thought Enables Parallel Exploration and Reasoning , author=. The Fourteenth International Conference on Learning Representations , year=

  19. [19]

    arXiv preprint arXiv:2602.08332 , year=

    Latent Reasoning with Supervised Thinking States , author=. arXiv preprint arXiv:2602.08332 , year=

    arXiv

  20. [20]

    arXiv preprint arXiv:2602.10229 , year=

    Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens , author=. arXiv preprint arXiv:2602.10229 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2602.04246 , year=

    CoLT: Reasoning with Chain of Latent Tool Calls , author=. arXiv preprint arXiv:2602.04246 , year=

    arXiv

  22. [22]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Tokenskip: Controllable chain-of-thought compression in llms , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  23. [23]

    Making Slow Thinking Faster: Compressing

    Zeju Li and Jianyuan Zhong and Ziyang Zheng and Xiangyu Wen and Zhijian Xu and Yingying Cheng and Fan Zhang and Qiang Xu , booktitle=. Making Slow Thinking Faster: Compressing

  24. [24]

    arXiv preprint arXiv:2405.08644 , year=

    Thinking tokens for language modeling , author=. arXiv preprint arXiv:2405.08644 , year=

    Pith/arXiv arXiv

  25. [25]

    Forty-second International Conference on Machine Learning , year=

    Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning , author=. Forty-second International Conference on Machine Learning , year=

  26. [26]

    Think Silently, Think Fast: Dynamic Latent Compression of

    Wenhui Tan and Jiaze Li and Jianzhong Ju and Zhenbo Luo and Ruihua Song and Jian Luan , booktitle=. Think Silently, Think Fast: Dynamic Latent Compression of

  27. [27]

    Jihoon Tack and Jack Lanchantin and Jane Yu and Andrew Cohen and Ilia Kulikov and Janice Lan and Shibo Hao and Yuandong Tian and Jason E Weston and Xian Li , booktitle=

  28. [28]

    The Thirteenth International Conference on Learning Representations , year=

    Reasoning with Latent Thoughts: On the Power of Looped Transformers , author=. The Thirteenth International Conference on Learning Representations , year=

  29. [29]

    arXiv preprint arXiv:2510.25741 , year=

    Scaling latent reasoning via looped language models , author=. arXiv preprint arXiv:2510.25741 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2601.21358 , year=

    Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization , author=. arXiv preprint arXiv:2601.21358 , year=

    arXiv

  31. [31]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Hybrid Latent Reasoning via Reinforcement Learning , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  32. [32]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  33. [33]

    Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    Are NLP models really able to solve simple math word problems? , author=. Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    2021

  34. [34]

    International conference on machine learning , pages=

    Pal: Program-aided language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  35. [35]

    Proceedings of the 2015 conference on empirical methods in natural language processing , pages=

    Solving general arithmetic word problems , author=. Proceedings of the 2015 conference on empirical methods in natural language processing , pages=

    2015

  36. [36]

    OpenAI blog , volume=

    Language models are unsupervised multitask learners , author=. OpenAI blog , volume=

  37. [37]

    Llama 3.2:

    Meta , year =. Llama 3.2:

  38. [38]

    arXiv preprint arXiv:1606.08415 , year=

    Gaussian error linear units (gelus) , author=. arXiv preprint arXiv:1606.08415 , year=

    Pith/arXiv arXiv

  39. [39]

    Eric Zelikman and Georges Raif Harik and Yijia Shao and Varuna Jayasiri and Nick Haber and Noah Goodman , booktitle=. Quiet-

  40. [40]

    Fast Quiet-

    Huang, Wei and Xiong, Yizhe and Ye, Xin and Deng, Zhijie and Chen, Hui and Lin, Zijia and Ding, Guiguang , booktitle=. Fast Quiet-

  41. [41]

    arXiv preprint arXiv:2505.11484 , year=

    Softcot++: Test-time scaling with soft chain-of-thought reasoning , author=. arXiv preprint arXiv:2505.11484 , year=

    Pith/arXiv arXiv

  42. [42]

    Xu, Jingxian and Zhou, Mengyu and Liu, Weichang and Liu, Hanbing and Han, Shi and Zhang, Dongmei , booktitle=

  43. [43]

    arXiv preprint arXiv:2507.06203 , year =

    A Survey on Latent Reasoning , author =. arXiv preprint arXiv:2507.06203 , year =

    Pith/arXiv arXiv

  44. [44]

    arXiv preprint arXiv:2505.16782 , year=

    Reasoning beyond language: A comprehensive survey on latent chain-of-thought reasoning , author=. arXiv preprint arXiv:2505.16782 , year=

    arXiv

  45. [45]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Dart: Distilling autoregressive reasoning to silent thought , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  46. [46]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Lightthinker: Thinking step-by-step compression , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  47. [47]

    arXiv preprint arXiv:2601.21576 , year=

    Chain Of Thought Compression: A Theoritical Analysis , author=. arXiv preprint arXiv:2601.21576 , year=

    Pith/arXiv arXiv

  48. [48]

    Soft Thinking: Unlocking the Reasoning Potential of

    Zhen Zhang and Xuehai He and Weixiang Yan and Ao Shen and Chenyang Zhao and Xin Eric Wang , booktitle=. Soft Thinking: Unlocking the Reasoning Potential of

  49. [49]

    arXiv preprint arXiv:2601.08808 , year=

    Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge , author=. arXiv preprint arXiv:2601.08808 , year=

    arXiv

  50. [50]

    arXiv preprint arXiv:2511.06411 , year=

    SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimization , author=. arXiv preprint arXiv:2511.06411 , year=

    arXiv

  51. [51]

    Junhong Wu and Jinliang Lu and Zixuan Ren and Gangqiang Hu and Zhi Wu and Dai Dai and Hua Wu , booktitle=

  52. [52]

    Neurocomputing , volume=

    Roformer: Enhanced transformer with rotary position embedding , author=. Neurocomputing , volume=. 2024 , publisher=

    2024

  53. [53]

    The Eleventh International Conference on Learning Representations , year=

    Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought , author=. The Eleventh International Conference on Learning Representations , year=

  54. [54]

    arXiv preprint arXiv:2410.21276 , year=

    Gpt-4o system card , author=. arXiv preprint arXiv:2410.21276 , year=

    Pith/arXiv arXiv

  55. [55]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    Pith/arXiv arXiv

  56. [56]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    Pith/arXiv arXiv

  57. [57]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Measuring chain of thought faithfulness by unlearning reasoning steps , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  58. [58]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    From reasoning to answer: Empirical, attention-based and mechanistic insights into distilled deepseek r1 models , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  59. [59]

    arXiv preprint arXiv:2510.15522 , year=

    Latent reasoning in llms as a vocabulary-space superposition , author=. arXiv preprint arXiv:2510.15522 , year=

    arXiv

  60. [60]

    Advances in neural information processing systems , volume=

    Large language models are zero-shot reasoners , author=. Advances in neural information processing systems , volume=

  61. [61]

    arXiv preprint arXiv:1503.02531 , year=

    Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=

    Pith/arXiv arXiv

  62. [62]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  63. [63]

    ACM computing surveys (CSUR) , volume=

    What every computer scientist should know about floating-point arithmetic , author=. ACM computing surveys (CSUR) , volume=. 1991 , publisher=

    1991

  64. [64]

    2002 , publisher=

    Accuracy and stability of numerical algorithms , author=. 2002 , publisher=

    2002

  65. [65]

    Advances in neural information processing systems , volume=

    Deep sets , author=. Advances in neural information processing systems , volume=

  66. [66]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  67. [67]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    Pith/arXiv arXiv

  68. [68]

    Attention is not explanation , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

  69. [69]

    Attention is not not explanation , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=

    2019

  70. [70]

    arXiv preprint arXiv:2512.21711 , year=

    Do latent tokens think? a causal and adversarial analysis of chain-of-continuous-thought , author=. arXiv preprint arXiv:2512.21711 , year=

    arXiv

  71. [71]

    50 Years of Integer Programming 1958-2008: from the Early Years to the State-of-the-Art , pages=

    Reducibility among combinatorial problems , author=. 50 Years of Integer Programming 1958-2008: from the Early Years to the State-of-the-Art , pages=. 2009 , publisher=

    1958

  72. [72]

    International Conference on Learning Representations , year=

    Decoupled Weight Decay Regularization , author=. International Conference on Learning Representations , year=

  73. [73]

    Edward J Hu and yelong shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo

  74. [74]

    arXiv preprint arXiv:2602.22441 , year=

    How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision? , author=. arXiv preprint arXiv:2602.22441 , year=

    arXiv

  75. [75]

    arXiv preprint arXiv:2604.04902 , year=

    Are Latent Reasoning Models Easily Interpretable? , author=. arXiv preprint arXiv:2604.04902 , year=

    Pith/arXiv arXiv

  76. [76]

    arXiv preprint arXiv:2505.14352 , year=

    Towards eliciting latent knowledge from LLMs with mechanistic interpretability , author=. arXiv preprint arXiv:2505.14352 , year=

    Pith/arXiv arXiv

  77. [77]

    arXiv preprint arXiv:2602.00449 , year=

    Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks , author=. arXiv preprint arXiv:2602.00449 , year=

    arXiv

  78. [78]

    2000 , publisher=

    Iterative solution of nonlinear equations in several variables , author=. 2000 , publisher=

    2000

  79. [79]

    arXiv preprint arXiv:2001.08361 , year=

    Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year=

    Pith/arXiv arXiv 2001

  80. [80]

    Commonsenseqa: A question answering challenge targeting commonsense knowledge , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

Showing first 80 references.