Machine Learning · arXiv:2610.06666 v1 · submitted 5 Oct 2026 · 76 references
What Matters for Latent Reasoning with Flow Matching
The paper says
“Latent reasoning lets a large language model (LLM) think in a continuous space and verbalize only the answer.”
more of the abstract
We argue that an effective latent thought must meet five requirements: it should be useful, helping produce the correct answer rather than merely changing it, diverse, so that resampling yields different reasoning trajectories, explainable, so that a decoded chain of thought (CoT) reflects reasoning the answer actually follows, refinable with more inference compute, and efficient, costing less than an explicit CoT at comparable accuracy. Current methods rarely meet these requirements: they learn shortcuts from the question, distill the explicit CoT into their weights, or imitate it one token at a time. We focus on flow matching in a learned latent space, the family we argue is best placed to meet them, and identify the training choices that make it work. The result is Flow-based Latent Reasoning (FLaRe), a simple recipe covering what the latent space encodes and how to shape it, where to train the flow, how to read out the answer, and a final stage of training on the model's own verified thoughts. A probe for each requirement shows that FLaRe improves on prior latent methods in all five. It also compares favorably with them on arithmetic benchmarks, while reaching 97% of the accuracy of explicit CoT at a quarter of its latency.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 6 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Solid, honest engineering paper on latent flow-matching reasoning; the refinable-requirement critique is real but lands as a framing problem, not a fatal flaw — send to peer review. This line is wrong
2 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 76 references say.
The paper says, at §4.4, Tab. 7, Sec. E.4, Tab. 20
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.64):
“The decoded reading is also more forgiving, because the decoder generates the CoT token by token and can turn a latent near a valid code into a well-formed CoT, so it still relies on explicit CoT generation.”
Pith says: Objection 1 of 2 · major
The refinability claim is load-bearing for the 'improves on prior latent methods in all five' headline and rests entirely on the decoded reading. Refinability is defined as an inference-time property ('At inference, the thoughts should remain refinable with more compute' [b170]) and §2.3 states the direct reading is 'the only reading used at inference'.
more of objection 1
On that reading the gain from 0.1x to 0.5x of explicit-CoT compute is +1.8 (stage 2) and +0.4 (stage 1), below CODI's +6.1, and Tab. 20 shows the direct reading saturates by S=2 (57.4 at S=2, 58.5 at S=5, 59.1 at S=20). The +24.2/+18.2 gains are carried by the decoded reading, which the paper itself concedes 'still relies on explicit CoT generation' and is 'more forgiving' because the decoder 'can turn a latent near a valid code into a well-formed CoT' [b74]. As reported, FLaRe does not beat the baselines on refinability along the pathway actually used at inference. Please report the direct-reading gain as the headline number, or recast the claim as refinability of the decoded (non-inference) thought and state explicitly that additional compute does not improve the inference-time answer.
The paper says, at §4.2, Tab. 7, Tab. 17, Sec. E.2
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The diversity claim is partly decoded-only as well. The '+16.6/+3.3' and '+14.7/+1.8' pass@16/voting entries in Tab. 7 are measured on the decoded reading. On the direct reading (Tab. 17), coverage still improves (+11.8 and +7.1 over greedy for stages 1 and 2), but majority voting improves by only +0.3 (stage 1) and −0.9 (stage 2), while the decoded reading gains +3.3 and +1.8, respectively.
more of objection 2
The statement in Sec. 4.2 that voting turns the coverage into higher accuracy therefore holds only on the decoded reading. Please report the direct-reading voting results in Tab. 7 or qualify the claim, and state how each requirement is to be evaluated when the two readings disagree.
4 minor objections
The paper says, at §2.3, Tab. 8, Tab. 20
Pith says: minor objection 1 of 4
The explicit CoT reference for the same backbone is 62.5 in Tab. 8 but 59.3 in Tab. 20, and the 97%-at-quarter-latency figure (57.4/59.3 = 96.8%) uses the Tab. 20 value. Please reconcile the two references or explain the difference, and report the efficiency claim against a consistent reference.
The paper says, at Abstract, §1
Pith says: minor objection 2 of 4
The abstract contains run-together words such as 'trajectories,explainable' and 'refinablewith'; several places in §1 and §4 have the same missing-space style. Please clean these up.
show 2 more
The paper says, at Tab. 10, §5
Pith says: minor objection 3 of 4
All ablations and headline numbers are single runs with seed 42. Given that the paper reports precise point gains (e.g., +3.8 direct from stage 2 in Tab. 6), please add variance estimates or at least 2–3 seeds for the main claims, or state explicitly that single-seed results are reported.
The paper says, at Tab. 7, Sec. E.5
Pith says: minor objection 4 of 4
The Efficient column may mislead: the 3.9x/97% entry uses the S=2 direct reading, while the decoded reading at S=20 is about 3.6x slower than explicit CoT (959ms vs 267ms, Tab. 20). A note stating that the efficiency claim is specific to the direct reading would prevent conflation of the two readings.
What would settle it
Re-run the refinable budget sweep of Sec. E.4 on the fixed stage-1 and stage-2 checkpoints, measuring decoded accuracy over 1 to 5 Euler steps, but swap the frozen corruption-robust 3B decoder for the decoder of a VAE trained WITHOUT corruption (the 'None' row of Tab. 2, keeping the same encoder and flow targets), and compare the resulting decoded gain against both the reported +24.2 (standard decoder) and the +1.8 direct gain. If the decoded refinable gain collapses toward +1.8 with the non-robust decoder, the refinability claim is substantially an artifact of the decoder's corruption robustness regenerating CoTs from near-code latents, and the 'refinable requirement met' claim must be re-scoped to the decoded-only pathway.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 6 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Short version: a careful, unusually honest engineering paper, and the stress test's refinability concern is legitimate but over-stated. It's a calibration issue, not a collapse.
What's actually new: the paper doesn't invent a mechanism — it assembles LaDiR-style latent diffusion, a corruption-smoothed VAE in the Cosmos spirit, an answer pass trained on the model's own detached endpoints, and STaR-style verified self-training. The contribution is the systematic ablation of which choices make the pipeline work, plus the five-requirement probe framework. That is a real contribution, and the ablations are good: the reconstruction-vs-downstream decoupling (Tab 1, Fig 2), the noise-shifted time distribution, the finding that the answer pass must read predicted endpoints, the diverse-CoTs effect. The paper reports both the direct and decoded readings throughout and explicitly concedes the decoded reading is more forgiving because the decoder repairs near-code latents [b74]. The low circularity burden is genuine — targets verified against external answers, headline numbers on the held-out GSM8K test.
The soft spots, in proportion. The refinable requirement leans almost entirely on the decoded reading: +1.8 direct versus +18.2/+24.2 decoded across the probed budget. The stress test is right about the asymmetry — baselines report refinable gains on their actual outputs, FLaRe on an auxiliary pathway. But it's not the takedown it's framed as. The decoded curve climbs from 36.6 at one step to 62.6 at twenty; a decoder merely trained to tolerate corruption would not produce that climb from a one-step latent that already decodes badly. The thought genuinely moves toward valid code with more compute. The real problem is interpretive: the requirement is stated as an inference-time property [b170], and at inference only the direct reading is used, which saturates by two steps. So refinement helps the thought, not the answer — the reader was trained on one-step endpoints and caps the direct curve. The paper should either reframe refinable as thought-quality refinement or state plainly that answer-level refinability is not delivered. The 97%-at-quarter-latency headline is on the direct reading at S=2, so the efficiency claim is sound; at S=20 the direct reading matches explicit CoT within noise. Minor items: no code or checkpoints, probe tables lack error bars, most key baselines are the authors' own runs (though they document protocol agreement against printed numbers), and the explicit CoT baseline differs between Tab 8 (62.5) and Tab 20 (59.3) for the same backbone — that needs reconciling.
Who it's for: anyone working on latent or continuous-space reasoning; the ablation findings on the answer pass and time distribution are citable regardless of whether the refinable framing survives.
Recommendation: send to peer review. Ask for a reframed refinable requirement, artifacts, and a reconciled baseline table.
The strongest objection
The paper defines refinability as an inference-time property: 'At inference, the thoughts should remain refinable with more compute' (b170). Yet the refinable probe of Sec. E.4 / Tab. 7 measures the gain from 0.1x to 0.5x of explicit CoT compute (1 to 5 Euler steps) and reports +1.8/+24.2 (direct/decoded) for stage 2 and +0.4/+18.2 for stage 1. The direct reading — the only one used at inference (Sec. 2.3, E.5) — improves by +1.8 over the entire probed budget and saturates almost immediately (Tab. 20: 57.4 at S=2, 58.5 at S=5, 59.1 at S=20), while the decoded reading climbs from 36.6 at one step to 62.6 at twenty. The entire refinable gain is therefore carried by decoded accuracy, which the authors concede is 'more forgiving, because the decoder generates the CoT token by token and can turn a latent near a valid code into a well-formed CoT, so it still relies on explicit CoT generation' (b74). The decoder was deliberately trained to be robust to corruption (token substitution p_sub=0.3, latent noise, latent dropout, Tab. 2 and Sec. 3.1.2), so a latent that merely approaches the VAE code manifold can be repaired into a correct CoT regardless of whether the flow's extra steps added genuine reasoning content. This creates a probe asymmetry: the baselines (Coconut, CODI, PCCoT) report refinable gains on their actual decoded outputs, while FLaRe reports gains on a non-inference auxiliary pathway. Since stage 2 already backpropagates the answer loss through full 10-step rollouts from noise (Eq. 8) and the direct reading still gains only +1.8, the evidence suggests the decoded gains reflect decoder regeneration of near-code latents rather than flow-level thought refinement. The consequence is that the refinable requirement — the one the paper motivates as the core advantage of flow matching — is met only on the reading the paper itself labels more forgiving, weakening the central claim that FLaRe 'improves on prior latent methods in all five' requirements (b218, b5).
Reading in good faith: the central construction is well-executed and the benchmark wins of Tab. 8 use the honest direct reading, so the recipe itself is not in question. The weakest link is exactly what the reader flagged: the requirement-level claims, and in particular refinability, are demonstrated only on the decoded reading, which the authors themselves concede relies on token-by-token explicit CoT generation (b74). I confirmed the numbers: Tab. 7 refinable +1.8/+24.2, Fig. 7/E.4 showing direct saturation by two steps and decoded gains continuing through twenty, and stage 2's reader already trained on full 10-step rollouts yet still showing only +1.8 direct. That the readout is already well-adapted and still shows no inference-time refinability is strong evidence the decoded gains are decoder-mediated repair of near-code latents, not flow refinement. Because the paper defines refinability as an inference-time property (b170) and measures it on a non-inference pathway, the 'improves in all five requirements' claim over-reaches. I part slightly from the reader's efficiency framing: the 97%-at-quarter-latency headline at S=2 checks out arithmetically (57.4/59.3 = 96.8%) and is honestly labelled as the S=2 configuration; the load-bearing issue is the refinable requirement, not the efficiency arithmetic. The concern is real but disclosed and does not collapse the central construction, so the verdict stays CONDITIONAL rather than moving to ACCEPT or REJECT: the paper should either demonstrate refinability on the direct reading (or on any answer-extraction path that does not regenerate the CoT) or re-scope the requirement claim, and the concrete decoder-swap test above would settle which is warranted.
The referee's report
The recipe components are well supported by careful, mostly single-sweep ablations: VAE corruption (Tab. 2), symbolic input and dual routes (Tab. 3), time distribution (Fig. 4), answer-pass input (Tab. 4), diverse CoTs (Tab. 5), and the stage-2 loop (Tab. 6). The manuscript is unusually candid in reporting both a 'direct' and a 'decoded' reading and in conceding that the decoded reading relies on explicit CoT generation because the decoder can repair a near-valid latent [b74]. Baseline numbers come with an explicit source ledger (Tab. 11), self-training targets are verified against reference answers, and headline accuracies are reported on the held-out GSM8K test. If the requirement-level claims could be supported along the inference pathway, or honestly recast, the paper would be a solid contribution. As it stands, the load-bearing requirement claims (refinable, and partly diverse) are stronger than the reported measurements sustain.
Pith's reading of the paper (a machine's summary, not the authors')
The paper argues that an effective latent thought must be useful, diverse, explainable, refinable, and efficient, and that flow matching in a learned latent space is the family best placed to satisfy all five at once. It then claims, through controlled ablations of every stage of the pipeline, that this goal is reachable only when five training choices are made right: a small code that compresses a symbolic chain of thought, a latent space smoothed by corruption rather than by reconstruction, a flow trained mostly near pure noise, an answer reader trained on the model's own imperfect thoughts, and a final stage of self-training on verified rollouts. Those choices, packaged as FLaRe, are reported to improve on prior latent reasoning methods on a probe for each of the five requirements, and to reach 97% of explicit chain-of-thought accuracy at a quarter of its latency. The efficiency figure is measured with the direct reading, the only one used at inference, while several requirement probes, including refinability, rely on the more forgiving decoded reading that the paper concedes still leans on explicit CoT generation.
What carries the argument. The carrying object is a latent thought $z \in \mathbb{R}^{8 \times 512}$ generated by flow matching in a space learned by a VAE that compresses symbolic chains of thought (GSM8K-Aug). Three mechanisms do the work: corruption-based smoothing of the VAE latent space rather than reconstruction tightening; a logit-normal time distribution $t = \mathrm{sigmoid}(s)$ with $s \sim \mathcal{N}(-2, 0.8^2)$ that concentrates the flow loss near pure noise where inference begins; and a stage-2 self-training loop in which, for each block of new questions with answers only, the EMA model proposes $K = 8$ thoughts whose decoded final answer is verified against the reference, re-encodes the accepted CoTs as flow targets, and trains them with an answer loss backpropagated through the full rollout of 10 guided Euler steps.
Core claim. FLaRe's central claim is that latent reasoning with flow matching works once five choices are aligned, and that no single choice alone suffices. The VAE encodes only the symbolic CoT — the steps and intermediate values, stripped of wording — into an 8×512 code, and its space is shaped by corrupting inputs and codes in training (token substitution, variance-preserving noise, and latent dropout) so that nearby codes decode consistently and the flow only has to reach a region rather than a point. The flow is trained with a logit-normal time distribution shifted toward noise, where inference starts, and with several CoTs per question so the model learns to cover the space of valid solutions rather than memorize a single code per question. The answer reader is trained on a 50/50 mix of noised codes and the model's own detached one-step endpoints, so it learns to read the imperfect thoughts actually produced at test time. Finally, stage 2 self-trains on the model's own verified rollouts: for new questions with answers only, the EMA model proposes several thoughts, those whose decoded answer matches the reference are re-encoded as flow targets, and an answer loss is backpropagated through the full guided denoising path. The paper reports that this recipe leads or ties prior latent methods across scales on GSM8K and GSM8K-Hard, and reaches 97% of explicit CoT accuracy at a quarter of its latency with the direct reading.
The premise it rests on. Everything rides on the decoded reading being the right measure of whether a latent thought is good: the paper concedes that this reading still relies on explicit CoT generation, while the direct reading, the only one used at inference, gains at most 1.8 points on the refinability probe that underlies the efficiency claim.
If the paper is right (Pith's inferences, not the authors')
Because the flow is trained across noise levels, more Euler steps keep refining the thought rather than saturating at the training budget, so inference compute can be traded against accuracy.
Resampling from noise yields genuinely different reasoning trajectories: pass@16 and majority-voting gains that prior latent methods, which produce a single deterministic thought, do not show.
Stage 2 trains from questions and reference answers only, without additional CoT annotations or a reward model, extending the recipe to data where chains of thought are unavailable.
The decoded thoughts recover reference intermediate values in order on about half the questions, giving an explainability check before an answer is decoded.
At fixed training data and backbone, stage 2 improves every accuracy column at every scale tested, by 2.7 to 3.8 points on GSM8K.
If the decoded reading — which the paper concedes still leans on explicit CoT generation — is set aside, the refinable requirement is largely unmet at inference: the direct reading, the only one used to answer, gains at most 1.8 points when the compute budget is raised over the tested range.
The headline efficiency claim is sensitive to that same choice: at the peak configuration the direct reading sits near 94.6% of explicit CoT accuracy, not 97%, suggesting the reported margin is carried by the decoded probe rather than by the deployed path.
The recipe depends on a symbolic CoT language that is short and compressible; a testable prediction is that on reasoning data that does not collapse into short symbolic steps, either the VAE capacity or the diversity requirement would need rethinking.
Stage 2 verifies only the final decoded answer and does not certify intermediate steps; a natural extension the paper does not run is verifying the decoded chain's intermediate values, which would tighten the explainability claim.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Latent reasoning lets a large language model (LLM) think in a continuous space and verbalize only the answer. We argue that an effective latent thought must meet five requirements: it should be useful, helping produce the correct answer rather than merely changing it, diverse, so that resampling yields different reasoning trajectories, explainable, so that a decoded chain of thought (CoT) reflects reasoning the answer actually follows, refinable with more inference compute, and efficient, costing less than an explicit CoT at comparable accuracy. Current methods rarely meet these requirements: they learn shortcuts from the question, distill the explicit CoT into their weights, or imitate it one token at a time. We focus on flow matching in a learned latent space, the family we argue is best placed to meet them, and identify the training choices that make it work. The result is Flow-based Latent Reasoning (FLaRe), a simple recipe covering what the latent space encodes and how to shape it, where to train the flow, how to read out the answer, and a final stage of training on the model's own verified thoughts. A probe for each requirement shows that FLaRe improves on prior latent methods in all five. It also compares favorably with them on arithmetic benchmarks, while reaching 97% of the accuracy of explicit CoT at a quarter of its latency.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-06
The authors' main claim, as the classifier picked it out 0.71
“A probe for each requirement shows that FLaRe improves on prior latent methods in all five.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.22 · human participants 0.03 · controlled comparison 0.81 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Latent reasoning lets a large language model (LLM) think in a continuous space and verbalize only the answer. | background 1.00 | 0.06 | 0.03 | asserted 1.00 |
| We argue that an effective latent thought must meet five requirements: it should be useful, helping produce the correct answer rather than merely changing it, diverse, so that resampling yields different reasoning trajectories, explainable, so that a decoded chain of thought (CoT) reflects reasoning the answer actually follows, refinable with more inference compute, and efficient, costing less than an explicit CoT at comparable accuracy. | contribution 0.48 | 0.27 | 0.15 | asserted 0.91 |
| Current methods rarely meet these requirements: they learn shortcuts from the question, distill the explicit CoT into their weights, or imitate it one token at a time. | gap 0.88 | 0.09 | 0.03 | asserted 0.94 |
| We focus on flow matching in a learned latent space, the family we argue is best placed to meet them, and identify the training choices that make it work. | contribution 0.66 | 0.51 | 0.12 | asserted 0.78 |
| The result is Flow-based Latent Reasoning (FLaRe), a simple recipe covering what the latent space encodes and how to shape it, where to train the flow, how to read out the answer, and a final stage of training on the model's own verified thoughts. | contribution 0.83 | 0.91 | 0.28 | asserted 1.00 |
| A probe for each requirement shows that FLaRe improves on prior latent methods in all five. | result 0.98 | 0.92 | 0.26 | asserted 1.00 |
| It also compares favorably with them on arithmetic benchmarks, while reaching 97% of the accuracy of explicit CoT at a quarter of its latency. | result 1.00 | 0.93 | 0.35 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.85
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-06: 1 request, 5,744 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (8)
Showing 7 of the paper's 8 figures. Figures are the authors', reproduced from the paper.
References (76)
- [1] Pranjal Aggarwal and Sean Welleck. L1 : Controlling how long a reasoning model thinks with reinforcement learning. In Conference on Language Modeling, 2025 on Pith
- [2] Darpan Aswal, Thomas Palmeira Ferraz, Yongxin Zhou, and Maxime Peyrard. Observable patterns are not explanations: A causal-geometric analysis of latent reasoning models. arXiv preprint arXiv:2606.12689, 2026 on PithPith's reviewarXiv
- [3] Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, O g uzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan. FlexTok : Resampling images into 1d token sequences of flexible length. In International Conference on Machine Learning, 2025 on Pith
- [4] Junyeob Baek, Mingyu Jo, Minsu Kim, Mengye Ren, Yoshua Bengio, and Sungjin Ahn. Generative recursive reasoning. arXiv preprint arXiv:2605.19376, 2026 on PithPith's reviewarXiv
- [5] Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, and Amir Globerson. Hopping too late: Exploring the limitations of large language models on multi-hop queries. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024 on Pith
- [6] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021 on PithPith's reviewarXiv
- [7] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023 on Pith
- [8] Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, and Xiaoyu Shen. Reasoning beyond language: A comprehensive survey on latent chain-of-thought reasoning. arXiv preprint arXiv:2505.16782, 2025 on PithPith's reviewarXiv
- [9] Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171, 2024 on PithPith's reviewarXiv
- [10] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021 on PithPith's reviewarXiv
- [11] Yingqian Cui, Zhenwei Dai, Bing He, Zhan Shi, Hui Liu, Rui Sun, Zhiji Liu, Yue Xing, Jiliang Tang, and Benoit Dumoulin. How do latent reasoning methods perform under weak and strong supervision? arXiv preprint arXiv:2602.22441, 2026 on PitharXiv
- [12] Timoth \'e e Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In International Conference on Learning Representations, 2024 on Pith
- [13] DeepSeek-AI , Daya Guo, Dejian Yang, Haowei Zhang, et al. DeepSeek-R1 : Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025 on PithPith's reviewarXiv
- [14] Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and ukasz Kaiser. Universal transformers. In International Conference on Learning Representations, 2019 on Pith
- [15] Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. Implicit chain of thought reasoning via knowledge distillation. arXiv preprint arXiv:2311.01460, 2023 on PithPith's reviewarXiv
- [16] Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit CoT to implicit CoT : Learning to internalize CoT step by step. arXiv preprint arXiv:2405.14838, 2024 on PithPith's reviewarXiv
- [17] Connor Dilgren and Sarah Wiegreffe. Are latent reasoning models easily interpretable? arXiv preprint arXiv:2604.04902, 2026 on PithPith's reviewarXiv
- [18] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow tr... on Pith
- [19] Ying Fan, Yilun Du, Kannan Ramchandran, and Kangwook Lee. Looped transformers for length generalization. In International Conference on Learning Representations, 2025 on Pith
- [20] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL : Program-aided language models. In International Conference on Machine Learning, 2023 on Pith
show 56 more
- [21] Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. In Advances in Neural Information Processing Sy... on Pith
- [22] Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D Lee, and Dimitris Papailiopoulos. Looped transformers as programmable computers. In International Conference on Machine Learning, pages 11398--11442, 2023 on Pith
- [23] Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens. In International Conference on Learning Representations, 2024 on Pith
- [24] Aaron Grattafiori et al. The L lama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024 on PithPith's reviewarXiv
- [25] Xiangming Gu, Chao Du, Tianyu Pang, Chongxuan Li, Min Lin, and Ye Wang. On memorization in diffusion models. Transactions on Machine Learning Research, 2025 on Pith
- [26] Hongcan Guo, Qinyu Zhao, Yian Zhao, Shen Nie, Rui Zhu, Qiushan Guo, Feng Wang, Tao Yang, Hengshuang Zhao, Guoqiang Wei, and Yan Zeng. Continuous latent diffusion language model. arXiv preprint arXiv:2605.06548, 2026 on PithPith's reviewarXiv
- [27] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. In Second Conference on Language Modeling, 2025 on Pith
- [28] David Herel and Tomas Mikolov. Thinking tokens for language modeling. arXiv preprint arXiv:2405.08644, 2024 on PithPith's reviewarXiv
- [29] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta- VAE : Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2017 on Pith
- [30] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022 on PithPith's reviewarXiv
- [31] Alexia Jolicoeur-Martineau. Less is more: Recursive reasoning with tiny networks. arXiv preprint arXiv:2510.04871, 2025 on PithPith's reviewarXiv
- [32] Zahra Kadkhodaie, Florentin Guth, Eero P. Simoncelli, and St \'e phane Mallat. Generalization in diffusion models arises from geometry-adaptive harmonic representations. In International Conference on Learning Representations, 2024 on Pith
- [33] Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Nicklas Majamaki, Navdeep Jaitly, Yi-An Ma, and Lianhui Qin. LaDiR : Latent diffusion enhances LLMs for text reasoning. In International Conference on Learning Representations, 2026 on Pith
- [34] Diederik P. Kingma and Max Welling. Auto-encoding variational B ayes. In International Conference on Learning Representations, 2014 on Pith
- [35] Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. Matryoshka representation learning. In Advances in Neural Information Processing Systems, volume 35, 2022 on Pith
- [36] Anna Kuzina, Maciej Pi \'o ro, and Babak Ehteshami Bejnordi. KaVa : Latent reasoning via compressed KV -cache distillation. In International Conference on Learning Representations, 2026 on Pith
- [37] Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026 on Pith
- [38] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. In Advances in Neural Information Processing Systems, volume 37, 2024 on Pith
- [39] Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, et al. On the biology of a large language model. Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/2025/attrib... on Pith
- [40] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2023 on Pith
- [41] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, 2023 on Pith
- [42] Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, and Kilian Q Weinberger. Latent diffusion for language generation. In Advances in Neural Information Processing Systems, volume 36, 2023 on Pith
- [43] Justin Lovelace, Christian Belardi, Sofian Zalouk, Adhitya Polavaram, Srivatsa Kundurthy, and Kilian Q Weinberger. Stop-Think-AutoRegress : Language modeling with latent diffusion planning. In Conference on Language Modeling, 2025 on Pith
- [44] Matthew V. Macfarlane, Cl \'e ment Bonnet, Herke van Hoof, and Levi H. S. Lelis. Gradient-based program synthesis with neurally interpreted languages. In International Conference on Learning Representations, 2026 on Pith
- [45] Sean McLeish, Ang Li, John Kirchenbauer, Dayal Singh Kalra, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Jonas Geiping, Tom Goldstein, and Micah Goldblum. Teaching pretrained language models to think deeper with retrofitted recurrence. arXiv preprint arXiv:2511.07... on PitharXiv
- [46] Viacheslav Meshchaninov, Egor Chimbulatov, Alexander Shabalin, Aleksandr Abramov, and Dmitry Vetrov. Cosmos : Compressed and smooth latent space for text diffusion modeling. In Advances in Neural Information Processing Systems, volume 38, 2025 on Pith
- [47] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Languag... on Pith
- [48] Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021 on Pith
- [49] Jacob Pfau, William Merrill, and Samuel R Bowman. Let's think dot by dot: Hidden computation in transformer language models. In First Conference on Language Modeling, 2024 on Pith
- [50] Qwen Team . Qwen3.8-Max : A new bar for coding and cowork, August 2026. URL https://qwen.ai/blog?id=qwen3.8 on Pith
- [51] Oren Rippel, Michael A. Gelbart, and Ryan P. Adams. Learning ordered representations with nested dropout. In International Conference on Machine Learning, 2014 on Pith
- [52] Michael Rizvi-Martel, Guillaume Rabusseau, and Marius Mosbach. The illusion of superposition? A principled analysis of latent thinking in language models. arXiv preprint arXiv:2604.06374, 2026 on PithPith's reviewarXiv
- [53] Subhro Roy and Dan Roth. Solving general arithmetic word problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015 on Pith
- [54] Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J Reddi. Reasoning with latent thoughts: On the power of looped transformers. In International Conference on Learning Representations, 2025 on Pith
- [55] Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. CODI : Compressing chain-of-thought into continuous space via self-distillation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 677--693, 2025 on Pith
- [56] Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Elsayed, Hanie Sedghi, Igor ... on Pith
- [57] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In International Conference on Learning Representations, 2025 on Pith
- [58] Wenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo, Ruihua Song, and Jian Luan. Think silently, think fast: Dynamic latent compression of LLM reasoning chains. In Advances in Neural Information Processing Systems, volume 38, 2025 on Pith
- [59] Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. OpenMathInstruct-2 : Accelerating AI for math with massive open-source instruction data. In International Conference on Learning Representations, 2025 on Pith
- [60] Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, and Yasin Abbasi Yadkori. Hierarchical reasoning model. arXiv preprint arXiv:2506.21734, 2025 on PithPith's reviewarXiv
- [61] Minghan Wang, Ye Bai, Thuy-Trang Vu, Ehsan Shareghi, and Gholamreza Haffari. GTS : Inference-time scaling of latent reasoning with a learnable G aussian thought sampler. arXiv preprint arXiv:2602.14077, 2026 a on PitharXiv
- [62] Minghan Wang, Thuy-Trang Vu, Ehsan Shareghi, and Gholamreza Haffari. Towards inference-time scaling for continuous space reasoning. In Findings of the Association for Computational Linguistics: ACL 2026, pages 26842--26856, 2026 b on Pith
- [63] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824--24837, 2022 on Pith
- [64] Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Jiaqi Wang, Xipeng Qiu, and Dahua Lin. SIM-CoT : Supervised implicit chain-of-thought. In International Conference on Learning Representations, 2026 on Pith
- [65] Haoyi Wu, Zhihao Teng, and Kewei Tu. Parallel continuous chain-of-thought with Jacobi iteration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 914--926, 2025 on Pith
- [66] Wilson Yan, Volodymyr Mnih, Aleksandra Faust, Matei Zaharia, Pieter Abbeel, and Hao Liu. ElasticTok : Adaptive tokenization for image and video. In International Conference on Learning Representations, 2025 on Pith
- [67] An Yang et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024 a on PithPith's reviewarXiv
- [68] Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024 b on Pith
- [69] Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023, 2024 on PithPith's reviewarXiv
- [70] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. STaR : Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems, 2022 on Pith
- [71] Yizhe Zhang, Jiatao Gu, Zhuofeng Wu, Shuangfei Zhai, Joshua Susskind, and Navdeep Jaitly. PLANNER : Generating diversified paragraph via latent language diffusion model. In Advances in Neural Information Processing Systems, volume 36, 2023 on Pith
- [72] Yuyi Zhang, Boyu Tang, Tianjie Ju, Sufeng Duan, and Gongshen Liu. Do latent tokens think? A causal and adversarial analysis of chain-of-continuous-thought. arXiv preprint arXiv:2512.21711, 2025 a on PithPith's reviewarXiv
- [73] Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, and Xin Eric Wang. Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space. In Advances in Neural Information Processing Systems, volume 38, 2025 b on Pith
- [74] Hanlin Zhu, Shibo Hao, Zhiting Hu, Jiantao Jiao, Stuart Russell, and Yuandong Tian. Reasoning by superposition: A theoretical perspective on chain of continuous thought. In Advances in Neural Information Processing Systems, volume 38, 2025 a on Pith
- [75] Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, et al. A survey on latent reasoning. arXiv preprint arXiv:2507.06203, 2025 b on PithPith's reviewarXiv
- [76] Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li, Haoran Que, Boyi Wei, Zixin Wen, Fan Yin, He Xing, et al. Scaling latent reasoning via looped language models. arXiv preprint arXiv:2510.25741, 2025 c on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment