{"id":"6c130723-fa3d-4d22-987f-261c11c636a8","arxiv_id":"2505.11484","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SoftCoT++ generates multiple diverse latent thoughts through distinct initial tokens and a contrastive loss, improving reasoning accuracy beyond soft-thought and self-consistency baselines.","lead":"SoftCoT++ makes an LLM's thinking happen as a continuous vector instead of words, and creates several distinct soft thoughts for the same question so the model can vote on the final answer. It reports consistent accuracy gains over chain-of-thought with self-consistency and over the prior SoftCoT method on five reasoning benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Compute mismatch in Tables 1–2: SoftCoT++ spends 10 assistant forward passes to make 10 soft thoughts, while SoftCoT-SC spends one, so the headline 'thinking-stage beats reasoning-stage' is not yet established at equal inference cost.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the headline comparison does not equalize inference compute. This stress test agrees and makes the issue concrete. SoftCoT++ at N=10 uses 10 assistant forward passes (one per distinct initial token) plus 10 LLM reasoning chains, whereas SoftCoT-SC uses 1 assistant forward pass plus 10 LLM reasoning chains. At N=100, SoftCoT++ uses 10 assistant passes plus 100 LLM chains, while SoftCoT-SC uses 1 assistant pass plus 100 LLM chains. Because the assistant is described only as a 'small' model, the extra cost may be small, but it is never quantified, and Appendix C.1 explicitly asserts a 'same computation budget' comparison that the stated protocol does not provide. This is the most load-bearing issue because the central claim directly contrasts thinking-stage scaling with reasoning-stage scaling; if total compute differs, the result is confounded. The proposed FLOPs-matched rerun would settle the question. Other issues noted by the reader, such as missing Coconut-SC results on two benchmarks and absent significance tests, are secondary and do not change the verdict. Since the reader's verdict is already CONDITIONAL and this concern reinforces it, no adjustment is needed.","tokens_in":14177,"tokens_out":10803,"duration_ms":112648,"concrete_test":"Measure the assistant and LLM per-input FLOPs (or wall-clock time) in the released code. On GSM8K, rerun SoftCoT-SC with R = round(10 + 9 * FLOPs_assistant / FLOPs_LLM) reasoning chains so its total compute matches SoftCoT++ at N=10, and with R = round(100 + 9 * FLOPs_assistant / FLOPs_LLM) at N=100, using the same seeds and evaluation protocol. If SoftCoT-SC at matched FLOPs reaches or exceeds SoftCoT++ accuracy, the thinking-stage-scaling advantage disappears; if SoftCoT++ still wins, the compute mismatch is not the driver.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive comparison is Table 1's SoftCoT++ (10 thinking chains) versus SoftCoT-SC (10 reasoning chains), and Table 2's N=100 hybrid. In both, SoftCoT++ runs the assistant model once per soft thought: 10 assistant forward passes at N=10, and 10 assistant passes plus 100 LLM chains at N=100. SoftCoT-SC runs the assistant once and then samples 10 or 100 LLM reasoning chains from that single soft thought. Thus SoftCoT++ consumes strictly more total inference compute (10 times the assistant FLOPs), and Appendix C.1's claim that the comparison is made 'under the same computation budget' is unsupported unless assistant FLOPs are zero. The observed gains could therefore come from extra assistant compute or from having ten soft thoughts rather than one, rather than from any advantage of thinking-stage over reasoning-stage scaling. The paper never reports assistant model size, per-call FLOPs, or wall-clock time, so the imbalance cannot be dismissed. This does not refute the method, but it makes the headline empirical claim unproven as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SoftCoT++ proposes to extend SoftCoT, a continuous-space chain-of-thought method, to the test-time scaling paradigm. The method generates multiple soft thoughts by feeding the assistant model distinct special [INI] tokens and adds a contrastive loss to increase separation between the resulting latent representations. The paper reports experiments on five benchmarks (GSM8K, ASDiv-Aug, AQuA, StrategyQA, Date Understanding) with LLaMA-3.1-8B-Instruct and Qwen3-8B, comparing SoftCoT++ against zero-shot CoT with self-consistency, Assist-CoT with self-consistency, Coconut-SC, and SoftCoT-SC. The authors claim that SoftCoT++ outperforms all baselines, that thinking-stage scaling is complementary to reasoning-stage scaling, and that the contrastive loss is essential to the improvement. A theoretical analysis in Appendix A.2 is offered to justify why diverse, higher-variance soft thoughts approximate the true latent-thought distribution better than small perturbations.","tokens_in":14500,"tokens_out":5679,"duration_ms":53016,"significance":"If the central empirical claim is correct, SoftCoT++ would demonstrate a new axis for test-time scaling: diversifying the continuous thinking stage rather than only sampling discrete reasoning chains. The idea is well-motivated, the method is simple, and the authors provide code. However, the headline comparison is confounded by unequal inference compute: SoftCoT++ spends more assistant-model forward passes than the SoftCoT-SC baseline at the same nominal chain count. Because of this, the paper does not yet establish that thinking-stage scaling is superior to reasoning-stage scaling at equal cost. The theoretical Lemma 2 is also not rigorous and largely assumes the variance ordering that the method is intended to create. The empirical results are plausible and the methodological direction is worth pursuing, but the paper needs additional experiments and a re-framing of the claims to be acceptable.","major_comments":[{"comment":"The headline comparison does not control inference compute. At N=10, SoftCoT++ performs 10 assistant forward passes to produce 10 soft thoughts, whereas SoftCoT-SC performs one assistant forward pass and then samples 10 reasoning chains from that single soft thought. At N=100 in Table 2, SoftCoT++ uses 10 assistant passes plus 100 LLM reasoning chains, while SoftCoT-SC uses 1 assistant pass plus 100 LLM chains. Appendix C.1 states that the comparison is made \"under the same computation budget,\" but this is only true if the assistant model's FLOPs are negligible. The paper does not report assistant model size, per-call FLOPs, or wall-clock time. Please add a compute-matched comparison (e.g., total FLOPs or latency) or explicitly restrict the claim to a non-compute-matched setting.","section":"Table 1, Table 2, Appendix C.1"},{"comment":"The proof of Lemma 2 assumes Var[Q1] < Var[Q2] ≤ Var[P] as part of its statement, which is exactly the property the contrastive loss is designed to induce. The proof then specializes to Gaussians with equal means and an isotropic covariance, assumptions not derived from the construction in Section 3.5. Consequently, Lemma 2 does not provide an independent justification for why distinct initial tokens plus contrastive learning yield a better approximation to P. Please either derive the variance ordering from the method, or state the lemma as a hypothesis that the contrastive loss is intended to satisfy.","section":"Appendix A.2, Lemma 2"},{"comment":"No significance tests are reported, and on several tasks the difference between SoftCoT++ and SoftCoT-SC is within one standard deviation. For example, on StrategyQA with LLaMA-3.1-8B, SoftCoT++ gives 71.18±0.15 and SoftCoT-SC gives 71.14±0.10; with Qwen3-8B the corresponding values are 71.22±0.18 and 71.18±0.15. The claim that SoftCoT++ \"significantly boosts\" and \"consistently outperforms\" the baselines is not supported without at least a paired significance test or a per-seed comparison.","section":"Table 1, Section 5.1"},{"comment":"Coconut-SC is not evaluated on StrategyQA or Date Understanding in Table 1, yet Section 5.1 and the abstract claim that SoftCoT++ outperforms all baselines across architectures and tasks. Please fill in the missing Coconut-SC results on those two benchmarks, or qualify the claim to the tasks where Coconut-SC is actually measured.","section":"Table 1, Section 4.3"},{"comment":"The method introduces new special tokens [INI]^i, but the implementation details only state that \"the parameters in the projection\" are trained for 10 epochs. It is not specified how the [INI] embeddings are initialized, whether they are added to the assistant model's vocabulary, or whether they are trained or frozen. This is essential for reproducibility, since the entire method depends on these tokens.","section":"Section 3.5, Equation (4); Section 4.2"}],"minor_comments":[{"comment":"The contrastive loss in Eq. (6) is not fully specified: the expectation is taken over an unspecified distribution, and the numerator exp(T_k·T_k) is a self-similarity term rather than a positive-pair similarity. Please clarify the exact objective and the sampling procedure.","section":"Equation (6)"},{"comment":"The analogy to multi-head attention in the sentence \"Inspiring by the multi-head attention ... that the structure as well as the computation graph among different head keeps the same but only the initial parameter differs\" is confusing. Please rephrase to explain more directly why distinct initial tokens are expected to produce distinct soft thoughts.","section":"Section 3.5"},{"comment":"The claim that \"on StrategyQA, we observed diminishing returns when the number of reasoning chains increases to 100\" is not supported by any table or figure in the paper; Table 2 only reports N=100 for GSM8K. Please provide the supporting results or remove the claim.","section":"Section 5.1, paragraph (3)"},{"comment":"The Coconut-SC rows report zero standard deviation (e.g., 87.03±0.00), which is unusual for an evaluation over 5 random seeds. Please explain why the variance is zero.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the idea of scaling the thinking stage in latent space is timely. The main obstacle is the compute mismatch in the empirical comparison, which affects the central claim. I believe the authors can address this with additional compute-matched experiments and a more careful theoretical framing. I would also encourage the authors to add significance tests, as the observed differences are small on some tasks."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate and pretty clean extension of SoftCoT to test-time scaling, and the empirical pattern is consistent. But the main comparison is not at equal inference cost, and the theory does not carry weight. I'd send it to review, but only if the authors are willing to redo the comparison properly.\n\nWhat's new: they generate multiple soft thoughts in latent space by feeding the assistant different special start tokens, and push them apart with a contrastive loss. That gives you something like self-consistency but at the thinking stage. It's the first thing I've seen that does TTS in continuous CoT space, and it's built directly on their own SoftCoT. The gains over SoftCoT-SC are consistent across five benchmarks and both LLaMA-3.1 and Qwen3, and the ablation shows the contrastive loss contributes beyond just having multiple tokens. Code is linked. That's a solid empirical contribution if it holds up.\n\nThe soft spots, in decreasing order of importance. First, the compute budget. In Table 1 and in the N=10 column of Table 2, SoftCoT++ runs the assistant 10 times (once per soft thought), while SoftCoT-SC runs it once and samples 10 reasoning chains. So SoftCoT++ is spending 10x more assistant FLOPs. The paper says in Appendix C.1 this is under 'the same computation budget,' but that is not supported unless you ignore assistant compute entirely. The headline result—that thinking-stage scaling beats reasoning-stage scaling—is therefore not yet established at equal cost. This is fixable: report wall-clock or FLOPs, or equalize the number of assistant calls, or show a controlled comparison.\n\nSecond, Lemma 2 is not doing any work. It assumes the variance ordering that the contrastive loss is designed to create, then shows (under a Gaussian assumption) that higher variance closer to the target gives lower KL. That is circular as an independent justification. The empirical results don't need it, so I'd cut or replace it.\n\nMinor: no significance tests, and Coconut-SC is missing on StrategyQA and Date Understanding, so the 'consistent across five benchmarks' claim is really 'across three plus two.' Also, the baseline Zero-Shot CoT (SC) uses 10 reasoning chains, so the comparison to SoftCoT++ has the same compute imbalance.\n\nWho this is for: anyone working on latent-space reasoning or test-time scaling. It's a useful extension, and the code means it can be reproduced. The central mechanism is plausible, and the empirical direction is consistent even if the headline is overstated. I'd seriously referee it, with the expectation that the compute issue is addressed before acceptance.","headline":"First serious attempt at test-time scaling in continuous CoT space, with consistent empirical gains, but the headline comparison is not compute-matched and the theory is circular.","tokens_in":14880,"tokens_out":2813,"would_cite":true,"duration_ms":25570,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SoftCoT++ scales soft chain-of-thought reasoning by generating diverse latent thoughts, and reports consistent gains over token-space scaling across five benchmarks and two model families.","keywords":["test-time scaling","soft chain-of-thought","latent-space reasoning","contrastive learning","self-consistency","LLM reasoning","continuous representations"],"falsifier":"Run GSM8K with total inference compute held equal: give SoftCoT-SC $N=10$ reasoning chains from one soft thought, and give SoftCoT++ $M$ thinking chains with $M$ chosen so that $M\\cdot(\\text{assistant FLOPs}+\\text{LLM FLOPs}) = (\\text{assistant FLOPs}+10\\cdot\\text{LLM FLOPs})$; if the SoftCoT++ advantage disappears or reverses under equal compute, the headline claim is not established.","tokens_in":13995,"feed_emoji":"🧠","tokens_out":9155,"duration_ms":81204,"temperature":0.7,"pith_summary":"SoftCoT++ claims that test-time scaling need not happen in token space: instead of generating many explicit reasoning chains, a small assistant model can generate several distinct continuous soft thoughts for the same question, and those latent thoughts can be gathered and voted on. The method replaces the assistant's placeholder tokens with distinct initial tokens, adds a contrastive loss so the resulting soft thoughts are pushed apart, and leaves the main LLM frozen. Across five reasoning benchmarks and two 8B model families, the paper reports that this thinking-stage scaling beats reasoning-stage scaling baselines, including SoftCoT-SC, and that it combines with self-consistency. If this is right, spending inference compute on widening the latent thinking distribution is a route to stronger reasoning from a frozen model, complementary to decoding more chains.","feed_headline":"Scaling hidden 'soft thoughts' beats sampling more reasoning chains","feed_subtitle":"Diverse latent thoughts, not more decodes, drive the gain across math, commonsense, and symbolic tasks.","key_machinery":"The load-bearing object is the set of specialized initial tokens $[\\text{INI}]^i_{1:L}$ that replace the standard placeholder tokens $[\\text{UNK}]_{1:L}$ in the assistant model's input; different initial tokens yield different soft thought vectors $T^i_{\\text{soft}}$ for the same question. Around these sits a contrastive learning loss $\\mathcal{L}_{\\text{cl}}$ that maximizes the distance between the soft thought representations, raising their variance. The theoretical support is Lemma 2: if the perturbed ensemble has variance $\\mathrm{Var}[Q_1]$ and the new ensemble has variance $\\mathrm{Var}[Q_2]$ with $\\mathrm{Var}[Q_1] < \\mathrm{Var}[Q_2] \\le \\mathrm{Var}[P]$, then $Q_2$ approximates the true latent-thought distribution $P$ better in KL divergence. Together, the tokens supply candidate thinking paths and the loss keeps those paths genuinely distinct; the LLM then decodes each path and takes a majority vote.","core_discovery":"The paper's central claim is that continuous-space chain-of-thought reasoning can itself be scaled at test time. SoftCoT++ splits generation into a thinking stage, where a frozen assistant model emits soft thought vectors $T_{\\text{soft}}$, and a reasoning stage, where the main LLM decodes tokens conditioned on those vectors. Because a latent thought is deterministic for a given input, the paper treats it as one sample from an assumed smooth density $P_G(t|I,Q)$, then simulates sampling by feeding the assistant multiple distinct special initial tokens $[\\text{INI}]_i$, each producing a different $T^i_{\\text{soft}}$, and trains a contrastive objective that repels these representations. A lemma shows such a higher-variance ensemble approximates the true latent-thought distribution better than small perturbations of one thought, provided its variance stays below the true variance. Empirically, this soft-thought ensemble outperforms discrete self-consistency baselines across GSM8K, ASDiv-Aug, AQuA, StrategyQA, and Date Understanding on LLaMA-3.1-8B and Qwen3-8B, and it is orthogonal to self-consistency.","pith_inferences":["Editorial inference: the reported comparison does not equalize total inference compute. SoftCoT++'s 10 thinking chains require roughly 10 assistant forward passes, while SoftCoT-SC and CoT-SC use one assistant pass and 10 LLM decodes; a compute-matched comparison would be the cleanest test of whether latent-space scaling is inherently more efficient.","Editorial inference: the variance lemma suggests a calibration experiment. Measuring the empirical variance of soft thoughts generated by $[\\text{INI}]_i$ tokens and by random perturbation, and checking $\\mathrm{Var}[Q_1] < \\mathrm{Var}[Q_2] \\le \\mathrm{Var}[\\hat P]$, would test the mechanism directly; the paper does not report such measurements.","Editorial inference: because only the projection module is trained and the LLM stays frozen, the same machinery should transfer to larger or instruction-tuned models; the paper explicitly leaves scaling beyond 8B to future work.","Editorial inference: since thinking-stage and reasoning-stage scaling are presented as orthogonal axes, SoftCoT++ could also be combined with sequential TTS methods such as self-refinement or search, not only with parallel self-consistency; the paper does not test these combinations."],"forward_implications":["Thinking-stage scaling and reasoning-stage scaling are orthogonal: SoftCoT++ reports that 10 diverse soft thoughts followed by 10 reasoning chains per thought (100 total) beats either mode alone, so both axes can be spent simultaneously.","For frozen 8B models, the main bottleneck under expanding inference budgets can be diversity, not capacity; SoftCoT++ states that representation-level diversity unlocks reasoning that token sampling leaves untapped, with diminishing returns only on tasks where the model is already at capacity, such as StrategyQA.","The method is architecture-agnostic: it reports consistent gains on LLaMA-3.1-8B-Instruct and Qwen3-8B without architecture-specific tuning, so it transfers across pretraining corpora, tokenizers, and positional encodings.","An immediate corollary of the paper's own self-consistency analysis is that majority-vote scaling only helps when single paths are already better than chance ($p>0.5$); SoftCoT++ is framed as a way to raise $p$ through latent diversity rather than merely increasing the vote count."],"supporting_citations":[{"why":"Defines SoftCoT, the frozen-assistant latent-thought method that SoftCoT++ extends and whose official implementation and benchmarks are inherited.","marker":"[Xu et al., 2025]"},{"why":"Introduces Coconut, the continuous latent-space reasoning baseline that SoftCoT++ must beat and that it compares against as Coconut-SC.","marker":"[Hao et al., 2024]"},{"why":"Supplies self-consistency, the parallel scaling method used for all baselines and combined with SoftCoT++ in the two-axis experiments.","marker":"[Wang et al., 2023]"},{"why":"Provides the repeated-sampling scaling-law context that motivates allocating more inference compute and frames the diversity-lifts-accuracy argument.","marker":"[Brown et al., 2024]"},{"why":"Frames test-time compute budgeting, which the paper relies on when comparing 10-chain scaling settings.","marker":"[Snell et al., 2024]"},{"why":"Supplies the zero-shot CoT prompting templates used to build the Zero-Shot CoT and Zero-Shot Assist-CoT baselines.","marker":"[Sprague et al., 2024]"},{"why":"Provides GSM8K, the primary dataset for the ablation and the scaling-orthogonality experiment.","marker":"[Cobbe et al., 2021]"},{"why":"Supplies the multi-head attention analogy (same graph, different initial parameters) that motivates using distinct initial tokens.","marker":"[Vaswani et al., 2017]"}],"fun_headline_variants":["Soft thoughts outscale chain sampling","Diverse latent thoughts beat more decodes","Test-time scaling in latent space wins","SoftCoT++: Perturb, contrast, reason better","Latent diversity outperforms self-consistency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline numbers give SoftCoT++ ten assistant runs (one per soft thought) but give self-consistency baselines only one assistant run before ten LLM reasoning chains, so if those assistant runs count as compute, the comparison is not apples-to-apples and thinking-stage scaling is not actually shown to be superior to reasoning-stage scaling.","fun_headline_variants_meta":{"raw":{"variants":["Soft thoughts outscale chain sampling","Diverse latent thoughts beat more decodes","Test-time scaling in latent space wins","SoftCoT++: Perturb, contrast, reason better","Latent diversity outperforms self-consistency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000134,"raw_usage":{"total_tokens":1177,"prompt_tokens":1020,"completion_tokens":157,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":89}},"tokens_in":636,"tokens_out":157,"duration_ms":2206,"temperature":1.0,"reasoning_tokens":89,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:52:40.777277+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GSM8K with total inference compute held equal: give SoftCoT-SC $N=10$ reasoning chains from one soft thought, and give SoftCoT++ $M$ thinking chains with $M$ chosen so that $M\\cdot(\\text{assistant FLOPs}+\\text{LLM FLOPs}) = (\\text{assistant FLOPs}+10\\cdot\\text{LLM FLOPs})$; if the SoftCoT++ advantage disappears or reverses under equal compute, the headline claim is not established.","supporting_citations":[{"cited_title":"Gomez, Lukasz Kaiser, and Illia Polosukhin","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-head attention analogy (same graph, different initial parameters) that motivates using distinct initial tokens."}],"review_version":1}