SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning

Chunyan Miao; Xu Guo; Yige Xu; Zhiwei Zeng

REVIEW 5 major objections 4 minor 7 cited by

SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning

T0 review · 5 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SoftCoT++ scales soft chain-of-thought reasoning by generating diverse latent thoughts, and reports consistent gains over token-space scaling across five benchmarks and two model families.

desk verdict First serious attempt at test-time scaling in continuous CoT space, with consistent empirical gains, but the headline comparison is not compute-matched and the theory is circular. read the letter →

arxiv 2505.11484 v2 pith:5JEXOP2H submitted 2025-05-16 cs.CL

Yige Xu , Xu Guo , Zhiwei Zeng , Chunyan Miao This is my paper · ORCID

classification cs.CL

keywords test-timescalingsoftchain-of-thoughtlatent-spacereasoningcontrastivelearningself-consistencyLLMcontinuousrepresentations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SoftCoT++ claims that test-time scaling need not happen in token space: instead of generating many explicit reasoning chains, a small assistant model can generate several distinct continuous soft thoughts for the same question, and those latent thoughts can be gathered and voted on. The method replaces the assistant's placeholder tokens with distinct initial tokens, adds a contrastive loss so the resulting soft thoughts are pushed apart, and leaves the main LLM frozen. Across five reasoning benchmarks and two 8B model families, the paper reports that this thinking-stage scaling beats reasoning-stage scaling baselines, including SoftCoT-SC, and that it combines with self-consistency. If this is right, spending inference compute on widening the latent thinking distribution is a route to stronger reasoning from a frozen model, complementary to decoding more chains.

What carries the argument

The load-bearing object is the set of specialized initial tokens $[\text{INI}]^i_{1:L}$ that replace the standard placeholder tokens $[\text{UNK}]_{1:L}$ in the assistant model's input; different initial tokens yield different soft thought vectors $T^i_{\text{soft}}$ for the same question. Around these sits a contrastive learning loss $\mathcal{L}_{\text{cl}}$ that maximizes the distance between the soft thought representations, raising their variance. The theoretical support is Lemma 2: if the perturbed ensemble has variance $\mathrm{Var}[Q_1]$ and the new ensemble has variance $\mathrm{Var}[Q_2]$ with $\mathrm{Var}[Q_1] < \mathrm{Var}[Q_2] \le \mathrm{Var}[P]$, then $Q_2$ approximates the true latent-thought distribution $P$ better in KL divergence. Together, the tokens supply candidate thinking paths and the loss keeps those paths genuinely distinct; the LLM then decodes each path and takes a majority vote.

What would settle it

Run GSM8K with total inference compute held equal: give SoftCoT-SC $N=10$ reasoning chains from one soft thought, and give SoftCoT++ $M$ thinking chains with $M$ chosen so that $M\cdot(\text{assistant FLOPs}+\text{LLM FLOPs}) = (\text{assistant FLOPs}+10\cdot\text{LLM FLOPs})$; if the SoftCoT++ advantage disappears or reverses under equal compute, the headline claim is not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that continuous-space chain-of-thought reasoning can itself be scaled at test time. SoftCoT++ splits generation into a thinking stage, where a frozen assistant model emits soft thought vectors $T_{\text{soft}}$, and a reasoning stage, where the main LLM decodes tokens conditioned on those vectors. Because a latent thought is deterministic for a given input, the paper treats it as one sample from an assumed smooth density $P_G(t|I,Q)$, then simulates sampling by feeding the assistant multiple distinct special initial tokens $[\text{INI}]_i$, each producing a different $T^i_{\text{soft}}$, and trains a contrastive objective that repels these representations. A lemma shows such a higher-variance ensemble approximates the true latent-thought distribution better than small perturbations of one thought, provided its variance stays below the true variance. Empirically, this soft-thought ensemble outperforms discrete self-consistency baselines across GSM8K, ASDiv-Aug, AQuA, StrategyQA, and Date Understanding on LLaMA-3.1-8B and Qwen3-8B, and it is orthogonal to self-consistency.

Load-bearing premise

The headline numbers give SoftCoT++ ten assistant runs (one per soft thought) but give self-consistency baselines only one assistant run before ten LLM reasoning chains, so if those assistant runs count as compute, the comparison is not apples-to-apples and thinking-stage scaling is not actually shown to be superior to reasoning-stage scaling.

Editorial extensions

If this is right

Thinking-stage scaling and reasoning-stage scaling are orthogonal: SoftCoT++ reports that 10 diverse soft thoughts followed by 10 reasoning chains per thought (100 total) beats either mode alone, so both axes can be spent simultaneously.
For frozen 8B models, the main bottleneck under expanding inference budgets can be diversity, not capacity; SoftCoT++ states that representation-level diversity unlocks reasoning that token sampling leaves untapped, with diminishing returns only on tasks where the model is already at capacity, such as StrategyQA.
The method is architecture-agnostic: it reports consistent gains on LLaMA-3.1-8B-Instruct and Qwen3-8B without architecture-specific tuning, so it transfers across pretraining corpora, tokenizers, and positional encodings.
An immediate corollary of the paper's own self-consistency analysis is that majority-vote scaling only helps when single paths are already better than chance ($p>0.5$); SoftCoT++ is framed as a way to raise $p$ through latent diversity rather than merely increasing the vote count.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the reported comparison does not equalize total inference compute. SoftCoT++'s 10 thinking chains require roughly 10 assistant forward passes, while SoftCoT-SC and CoT-SC use one assistant pass and 10 LLM decodes; a compute-matched comparison would be the cleanest test of whether latent-space scaling is inherently more efficient.
Editorial inference: the variance lemma suggests a calibration experiment. Measuring the empirical variance of soft thoughts generated by $[\text{INI}]_i$ tokens and by random perturbation, and checking $\mathrm{Var}[Q_1] < \mathrm{Var}[Q_2] \le \mathrm{Var}[\hat P]$, would test the mechanism directly; the paper does not report such measurements.
Editorial inference: because only the projection module is trained and the LLM stays frozen, the same machinery should transfer to larger or instruction-tuned models; the paper explicitly leaves scaling beyond 8B to future work.
Editorial inference: since thinking-stage and reasoning-stage scaling are presented as orthogonal axes, SoftCoT++ could also be combined with sequential TTS methods such as self-refinement or search, not only with parallel self-consistency; the paper does not test these combinations.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a legitimate and pretty clean extension of SoftCoT to test-time scaling, and the empirical pattern is consistent. But the main comparison is not at equal inference cost, and the theory does not carry weight. I'd send it to review, but only if the authors are willing to redo the comparison properly.

What's new: they generate multiple soft thoughts in latent space by feeding the assistant different special start tokens, and push them apart with a contrastive loss. That gives you something like self-consistency but at the thinking stage. It's the first thing I've seen that does TTS in continuous CoT space, and it's built directly on their own SoftCoT. The gains over SoftCoT-SC are consistent across five benchmarks and both LLaMA-3.1 and Qwen3, and the ablation shows the contrastive loss contributes beyond just having multiple tokens. Code is linked. That's a solid empirical contribution if it holds up.

The soft spots, in decreasing order of importance. First, the compute budget. In Table 1 and in the N=10 column of Table 2, SoftCoT++ runs the assistant 10 times (once per soft thought), while SoftCoT-SC runs it once and samples 10 reasoning chains. So SoftCoT++ is spending 10x more assistant FLOPs. The paper says in Appendix C.1 this is under 'the same computation budget,' but that is not supported unless you ignore assistant compute entirely. The headline result—that thinking-stage scaling beats reasoning-stage scaling—is therefore not yet established at equal cost. This is fixable: report wall-clock or FLOPs, or equalize the number of assistant calls, or show a controlled comparison.

Second, Lemma 2 is not doing any work. It assumes the variance ordering that the contrastive loss is designed to create, then shows (under a Gaussian assumption) that higher variance closer to the target gives lower KL. That is circular as an independent justification. The empirical results don't need it, so I'd cut or replace it.

Minor: no significance tests, and Coconut-SC is missing on StrategyQA and Date Understanding, so the 'consistent across five benchmarks' claim is really 'across three plus two.' Also, the baseline Zero-Shot CoT (SC) uses 10 reasoning chains, so the comparison to SoftCoT++ has the same compute imbalance.

Who this is for: anyone working on latent-space reasoning or test-time scaling. It's a useful extension, and the code means it can be reproduced. The central mechanism is plausible, and the empirical direction is consistent even if the headline is overstated. I'd seriously referee it, with the expectation that the compute issue is addressed before acceptance.

Referee Report

5 major / 4 minor

Summary. SoftCoT++ proposes to extend SoftCoT, a continuous-space chain-of-thought method, to the test-time scaling paradigm. The method generates multiple soft thoughts by feeding the assistant model distinct special [INI] tokens and adds a contrastive loss to increase separation between the resulting latent representations. The paper reports experiments on five benchmarks (GSM8K, ASDiv-Aug, AQuA, StrategyQA, Date Understanding) with LLaMA-3.1-8B-Instruct and Qwen3-8B, comparing SoftCoT++ against zero-shot CoT with self-consistency, Assist-CoT with self-consistency, Coconut-SC, and SoftCoT-SC. The authors claim that SoftCoT++ outperforms all baselines, that thinking-stage scaling is complementary to reasoning-stage scaling, and that the contrastive loss is essential to the improvement. A theoretical analysis in Appendix A.2 is offered to justify why diverse, higher-variance soft thoughts approximate the true latent-thought distribution better than small perturbations.

Significance. If the central empirical claim is correct, SoftCoT++ would demonstrate a new axis for test-time scaling: diversifying the continuous thinking stage rather than only sampling discrete reasoning chains. The idea is well-motivated, the method is simple, and the authors provide code. However, the headline comparison is confounded by unequal inference compute: SoftCoT++ spends more assistant-model forward passes than the SoftCoT-SC baseline at the same nominal chain count. Because of this, the paper does not yet establish that thinking-stage scaling is superior to reasoning-stage scaling at equal cost. The theoretical Lemma 2 is also not rigorous and largely assumes the variance ordering that the method is intended to create. The empirical results are plausible and the methodological direction is worth pursuing, but the paper needs additional experiments and a re-framing of the claims to be acceptable.

major comments (5)

[Table 1, Table 2, Appendix C.1] The headline comparison does not control inference compute. At N=10, SoftCoT++ performs 10 assistant forward passes to produce 10 soft thoughts, whereas SoftCoT-SC performs one assistant forward pass and then samples 10 reasoning chains from that single soft thought. At N=100 in Table 2, SoftCoT++ uses 10 assistant passes plus 100 LLM reasoning chains, while SoftCoT-SC uses 1 assistant pass plus 100 LLM chains. Appendix C.1 states that the comparison is made "under the same computation budget," but this is only true if the assistant model's FLOPs are negligible. The paper does not report assistant model size, per-call FLOPs, or wall-clock time. Please add a compute-matched comparison (e.g., total FLOPs or latency) or explicitly restrict the claim to a non-compute-matched setting.
[Appendix A.2, Lemma 2] The proof of Lemma 2 assumes Var[Q1] < Var[Q2] ≤ Var[P] as part of its statement, which is exactly the property the contrastive loss is designed to induce. The proof then specializes to Gaussians with equal means and an isotropic covariance, assumptions not derived from the construction in Section 3.5. Consequently, Lemma 2 does not provide an independent justification for why distinct initial tokens plus contrastive learning yield a better approximation to P. Please either derive the variance ordering from the method, or state the lemma as a hypothesis that the contrastive loss is intended to satisfy.
[Table 1, Section 5.1] No significance tests are reported, and on several tasks the difference between SoftCoT++ and SoftCoT-SC is within one standard deviation. For example, on StrategyQA with LLaMA-3.1-8B, SoftCoT++ gives 71.18±0.15 and SoftCoT-SC gives 71.14±0.10; with Qwen3-8B the corresponding values are 71.22±0.18 and 71.18±0.15. The claim that SoftCoT++ "significantly boosts" and "consistently outperforms" the baselines is not supported without at least a paired significance test or a per-seed comparison.
[Table 1, Section 4.3] Coconut-SC is not evaluated on StrategyQA or Date Understanding in Table 1, yet Section 5.1 and the abstract claim that SoftCoT++ outperforms all baselines across architectures and tasks. Please fill in the missing Coconut-SC results on those two benchmarks, or qualify the claim to the tasks where Coconut-SC is actually measured.
[Section 3.5, Equation (4); Section 4.2] The method introduces new special tokens [INI]^i, but the implementation details only state that "the parameters in the projection" are trained for 10 epochs. It is not specified how the [INI] embeddings are initialized, whether they are added to the assistant model's vocabulary, or whether they are trained or frozen. This is essential for reproducibility, since the entire method depends on these tokens.

minor comments (4)

[Equation (6)] The contrastive loss in Eq. (6) is not fully specified: the expectation is taken over an unspecified distribution, and the numerator exp(T_k·T_k) is a self-similarity term rather than a positive-pair similarity. Please clarify the exact objective and the sampling procedure.
[Section 3.5] The analogy to multi-head attention in the sentence "Inspiring by the multi-head attention ... that the structure as well as the computation graph among different head keeps the same but only the initial parameter differs" is confusing. Please rephrase to explain more directly why distinct initial tokens are expected to produce distinct soft thoughts.
[Section 5.1, paragraph (3)] The claim that "on StrategyQA, we observed diminishing returns when the number of reasoning chains increases to 100" is not supported by any table or figure in the paper; Table 2 only reports N=100 for GSM8K. Please provide the supporting results or remove the claim.
[Table 1] The Coconut-SC rows report zero standard deviation (e.g., 87.03±0.00), which is unusual for an evaluation over 5 random seeds. Please explain why the variance is zero.

Circularity Check

1 steps flagged · score 4.0 of 10

Theoretical support for SoftCoT++ is partially self-definitional: Q2 is defined as samples from the true distribution, so Lemma 2's conclusion is built into the definition; the empirical results are otherwise independent.

self definitional [Section 3.5, Definition 3 and Lemma 2 (Appendix A.2)]
"Let T scale soft ={T i soft} n i=1 be a set of representations sampled from PG(t|I,Q). The distribution Q2 is then estimated from the T scale soft. ... Lemma 2. The candidate distribution Q2 is better than Q1 to describe P, if Var[Q1]<Var[Q2]≤Var[P], subjects to ∀T i soft∼P."

Definition 3 defines Q2 as the empirical distribution of samples drawn from the true density PG. Hence 'Q2 approximates P better than Q1' holds by construction before any method is proposed. The paper then asserts that SoftCoT++ supplies such a Q2 via multiple [INI] tokens and the contrastive loss (Eq. 6), but the lemma assumes exactly the variance ordering that the contrastive loss is designed to produce (Var[Q1]<Var[Q2]≤Var[P]) without proving that soft thoughts generated by SoftCoT++ are actually samples from PG or that the contrastive regularizer keeps variance within Var[P].

full rationale

The paper's headline empirical result, SoftCoT++ outperforming CoT-SC, Coconut-SC, and SoftCoT-SC on GSM8K, ASDiv, AQuA, StrategyQA, and Date Understanding, is an independent, externally benchmarked comparison, so the central claim is not circular. The main circularity concern is confined to the theoretical framing: Definition 3 defines Q2 as samples from the true soft-thought density, and Lemma 2's conclusion follows from that definition together with an assumed variance ordering that the contrastive loss is supposed to create but is not shown to satisfy. This makes the Appendix A.2 theoretical analysis partly self-definitional rather than an independent justification. The self-citations to SoftCoT [Xu et al., 2025] are normal prior-work references and are not load-bearing in a circular way: SoftCoT-SC and SoftCoT++ are evaluated against external benchmarks, and the improvements are empirical. Separately, the claim in Appendix C.1 that thinking-scaling and reasoning-scaling are compared under the same computation budget is unsupported because SoftCoT++ uses N assistant forward passes for N thinking chains while SoftCoT-SC uses one; this is a correctness and fairness risk, not a circularity, and does not affect the circularity score. Overall score 4 reflects a partially self-definitional theory alongside an independent empirical core.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central empirical claim rests on the trained projection and [INI] token embeddings, plus the unstated contrastive loss weight. The theoretical motivation rests on Assumption 1 and the Gaussian/variance assumptions in Lemma 2, none of which are independently verified.

free parameters (3)

Number of soft thought tokens L = 4
Set to 4 following SoftCoT (Section 4.2); chosen by hand and affects how much latent context is used.
Contrastive loss weight = not reported
Eq (6) is added as a regularizer but its weight relative to the task loss is never specified in Section 4.2, which is needed to reproduce training.
Special [INI] token embeddings = learned, values not reported
The distinct initial tokens in Eq (4) are new trainable embeddings whose initialization and training details are not given.

assumptions (4)

ad hoc to paper Assumption 1: there exists a smooth, differentiable density P_G(t|I,Q) such that the deterministic soft thought is a single sample from it.
Introduced in Section 3.4 to justify perturbation-based scaling; no evidence is provided that deterministic hidden states are draws from such a density.
ad hoc to paper Gaussian model: P, Q1, Q2 are Gaussians with equal means.
The proof of Lemma 2 (Appendix A.2) states 'For convenience, we let P=N(...), Q1=N(...), Q2=N(...)' and ignores mean differences, an assumption the lemma does not state.
ad hoc to paper Variance ordering Var[Q1] < Var[Q2] <= Var[P].
This condition is the entire content of Lemma 2's conclusion and is assumed, not derived; the contrastive loss is engineered to produce it.
standard math Taylor expansion and KL divergence formulas.
Used in Lemma 1 and Lemma 2 proofs; standard results applied without issue.

invented entities (1)

Special initial tokens [INI]^i
purpose: Distinct input tokens for the assistant model so it produces multiple different soft thoughts for the same query.
These are newly introduced trainable model inputs. Their effectiveness is supported only by the paper's own experiments, with no external falsifiable handle, and their training details are not fully specified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning." pith.science (2026). https://pith.science/paper/5JEXOP2H

@misc{pith2026250511484,
  author       = {Pith},
  title        = {Pith review of: SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5JEXOP2H}},
  note         = {Machine review of arXiv:2505.11484}
}

read the original abstract

Test-Time Scaling (TTS) refers to approaches that improve reasoning performance by allocating extra computation during inference, without altering the model's parameters. While existing TTS methods operate in a discrete token space by generating more intermediate steps, recent studies in Coconut and SoftCoT have demonstrated that thinking in the continuous latent space can further enhance the reasoning performance. Such latent thoughts encode informative thinking without the information loss associated with autoregressive token generation, sparking increased interest in continuous-space reasoning. Unlike discrete decoding, where repeated sampling enables exploring diverse reasoning paths, latent representations in continuous space are fixed for a given input, which limits diverse exploration, as all decoded paths originate from the same latent thought. To overcome this limitation, we introduce SoftCoT++ to extend SoftCoT to the Test-Time Scaling paradigm by enabling diverse exploration of thinking paths. Specifically, we perturb latent thoughts via multiple specialized initial tokens and apply contrastive learning to promote diversity among soft thought representations. Experiments across five reasoning benchmarks and two distinct LLM architectures demonstrate that SoftCoT++ significantly boosts SoftCoT and also outperforms SoftCoT with self-consistency scaling. Moreover, it shows strong compatibility with conventional scaling techniques such as self-consistency. Source code is available at https://github.com/xuyige/SoftCoT.

Figures

Figures reproduced from arXiv: 2505.11484 by the authors.

**Figure 2.** Comparison of SoftCoT-P and SoftCoT-SC on GSM8K. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
cs.MM 2026-08 conditional novelty 6.0 of 10

AGS uses the ratio of attention to image tokens vs text tokens to switch multimodal LLMs between latent and explicit reasoning, improving accuracy and cutting decoding steps.
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
cs.LG 2026-06 unverdicted novelty 6.0 of 10

LOTUS uses a looped padded Transformer with parallel cross-entropy supervision on gold CoT tokens to match explicit CoT performance at 3B parameters while reducing thought-phase latency 2.5x-6.9x.
MUX: Continuous Reasoning via Multiplexed Tokens
cs.AI 2026-05 conditional novelty 6.0 of 10

MUX trains language models to reason with continuous latent tokens that encode spans of discrete reasoning as lossless weighted superpositions, improving accuracy and efficiency over latent-reasoning baselines.
LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning
physics.chem-ph 2026-02 conditional novelty 6.0 of 10

LatentChem reasons in continuous latent space for chemistry, achieving a 59.88% non-tie win rate over explicit CoT on ChemCoTBench with a 10.84x average reduction in reasoning overhead.
Measuring Reasoning Utility in LLMs via Conditional Entropy Reduction
cs.CL 2025-08 conditional novelty 6.0 of 10

In an oracle study on MATH problems, conditional entropy over the answer span decreases across reasoning steps for correct LLM solutions but stays flat or rises for incorrect ones.
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
cs.AI 2026-04 accept novelty 5.0 of 10

A large survey organizes latent-space work in language-based models by foundation, evolution, four mechanisms, seven abilities, and open challenges.
Implicit Reasoning in Large Language Models: A Comprehensive Survey
cs.CL 2025-09 conditional novelty 5.0 of 10

A survey organizing implicit (silent) reasoning in LLMs into three execution paradigms, plus evidence, benchmarks, and challenges.

Reference graph

Works this paper leans on

35 extracted references · 10 canonical work pages · cited by 7 Pith papers

[1]

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

BIG.Bench.authors. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res., 2023, 2023. URL https://openreview.net/forum?id=uyTL5Bvosj

work page 2023
[2]

Large language monkeys: Scaling inference compute with repeated sampling

Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R \'e , and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. URL https://arxiv.org/abs/2407.21787

arXiv 2024
[3]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

work page 2020
[4]

Iterative deepening sampling for large language models

Weizhe Chen, Sven Koenig, and Bistra Dilkina. Iterative deepening sampling for large language models. arXiv preprint arXiv:2502.05449, 2025. URL https://arxiv.org/abs/2502.05449

arXiv 2025
[5]

Teaching large language models to self-debug

Xinyun Chen, Maxwell Lin, Nathanael Sch \" a rli, and Denny Zhou. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=KuPixIqPiq

work page 2024
[6]

Compressed chain of thought: Efficient reasoning through dense representations

Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171, 2024. URL http://arxiv.org/abs/2412.13171

arXiv 2024
[7]

PaLM : Scaling language modeling with pathways

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM : Scaling language modeling with pathways. Journal of Machine Learning Research, 24 0 (240): 0 1--113, 2023. URL https://dl.acm.org/doi/pdf/10.5555/3648699.3648939

arXiv 2023
[8]

Training verifiers to solve math word problems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. URL http://arxiv.org/abs/2110.14168

arXiv

Show all 35 references

[9]

Deepseek-R1 : Incentivizing reasoning capability in llms via reinforcement learning

DeepSeek-AI. Deepseek-R1 : Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. URL https://arxiv.org/abs/2501.12948

2025 arXiv
[10]

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. URL http://arxiv.org/abs/2407.21783

2024 arXiv
[11]

Stream of search (sos): Learning to search in language

Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D Goodman. Stream of search (sos): Learning to search in language. arXiv preprint arXiv:2404.03683, 2024. URL http://arxiv.org/abs/2404.03683

2024 arXiv
[12]

Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies

Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. Trans. Assoc. Comput. Linguistics, 9: 0 346--361, 2021. doi:10.1162/TACL\_A\_00370. URL https://doi...

2021 doi
[13]

Training large language models to reason in a continuous latent space

Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024. URL http://arxiv.org/abs/2412.06769

2024 arXiv
[14]

S*: Test time scaling for code generation

Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E Gonzalez, and Ion Stoica. S*: Test time scaling for code generation. arXiv preprint arXiv:2502.14382, 2025. URL https://arxiv.org/abs/2502.14382

2025 arXiv
[15]

Let's verify step by step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/fo...

2024
[16]

Program induction by rationale generation: Learning to solve and explain algebraic word problems

Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, Ju...

2017 doi
[17]

PEARL : Parallel speculative decoding with adaptive draft length

Tianyu Liu, Yun Li, Qitan Lv, Kai Liu, Jianchen Zhu, Winston Hu, and Xiao Sun. PEARL : Parallel speculative decoding with adaptive draft length. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=QOXrVMiHGK

2025
[18]

Self-refine: Iterative refinement with self-feedback

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

2023
[19]

s1: Simple test-time scaling

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025. URL https://arxiv.org/abs/2501.19393

2025 arXiv
[20]

GPT-4 technical report

OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023. URL http://arxiv.org/abs/2303.08774

2023 arXiv
[21]

Qwen3, April 2025

Qwen Team . Qwen3, April 2025. URL https://qwenlm.github.io/blog/qwen3/

2025
[22]

The effect of sampling temperature on problem solving in large language models

Matthew Renze. The effect of sampling temperature on problem solving in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7346--7356, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi:10.1865...

2024 doi
[23]

Efficient reasoning with hidden thinking

Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. Efficient reasoning with hidden thinking. arXiv preprint arXiv:2501.19201, 2025. URL http://arxiv.org/abs/2501.19201

2025 arXiv
[24]

Scaling llm test-time compute optimally can be more effective than scaling model parameters

Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024. URL https://arxiv.org/abs/2408.03314

2024 arXiv
[25]

To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning

Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. arXiv preprint arXiv:2409.12183, 2024. URL h...

2024 arXiv
[26]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, De...

2017
[27]

Mixture-of-agents enhances large language model capabilities

Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances large language model capabilities. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net, 2025. URL https://o...

2025
[28]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali,...

2023
[29]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Inf...

2022
[30]

SoftCoT : Soft chain-of-thought for efficient reasoning with llms

Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. SoftCoT : Soft chain-of-thought for efficient reasoning with llms. arXiv preprint arXiv:2502.12134, 2025. URL https://arxiv.org/abs/2502.12134

2025 arXiv
[31]

Qwen2.5 technical report

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

2024 arXiv
[32]

Tree of thoughts: Deliberate problem solving with large language models

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processi...

2023
[33]

What, how, where, and how well? a survey on test-time scaling in large language models

Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. What, how, where, and how well? a survey on test-time scaling in large language models. arXiv preprint arXiv:2503.24235, 2025. URL https://arxiv.org/abs/2503.24235

2025 arXiv
[34]

Automatic chain of thought prompting in large language models

Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. URL https://openreview.net...

2023
[35]

Le, and Ed H

Denny Zhou, Nathanael Sch \" a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and Ed H. Chi. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on ...

2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

BIG.Bench.authors. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res., 2023, 2023. URL https://openreview.net/forum?id=uyTL5Bvosj

work page 2023

[2] [2]

Large language monkeys: Scaling inference compute with repeated sampling

Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R \'e , and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. URL https://arxiv.org/abs/2407.21787

arXiv 2024

[3] [3]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

work page 2020

[4] [4]

Iterative deepening sampling for large language models

Weizhe Chen, Sven Koenig, and Bistra Dilkina. Iterative deepening sampling for large language models. arXiv preprint arXiv:2502.05449, 2025. URL https://arxiv.org/abs/2502.05449

arXiv 2025

[5] [5]

Teaching large language models to self-debug

Xinyun Chen, Maxwell Lin, Nathanael Sch \" a rli, and Denny Zhou. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=KuPixIqPiq

work page 2024

[6] [6]

Compressed chain of thought: Efficient reasoning through dense representations

Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171, 2024. URL http://arxiv.org/abs/2412.13171

arXiv 2024

[7] [7]

PaLM : Scaling language modeling with pathways

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM : Scaling language modeling with pathways. Journal of Machine Learning Research, 24 0 (240): 0 1--113, 2023. URL https://dl.acm.org/doi/pdf/10.5555/3648699.3648939

arXiv 2023

[8] [8]

Training verifiers to solve math word problems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. URL http://arxiv.org/abs/2110.14168

arXiv