REVIEW 7 cited by
SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Test-Time Scaling (TTS) refers to approaches that improve reasoning performance by allocating extra computation during inference, without altering the model's parameters. While existing TTS methods operate in a discrete token space by generating more intermediate steps, recent studies in Coconut and SoftCoT have demonstrated that thinking in the continuous latent space can further enhance the reasoning performance. Such latent thoughts encode informative thinking without the information loss associated with autoregressive token generation, sparking increased interest in continuous-space reasoning. Unlike discrete decoding, where repeated sampling enables exploring diverse reasoning paths, latent representations in continuous space are fixed for a given input, which limits diverse exploration, as all decoded paths originate from the same latent thought. To overcome this limitation, we introduce SoftCoT++ to extend SoftCoT to the Test-Time Scaling paradigm by enabling diverse exploration of thinking paths. Specifically, we perturb latent thoughts via multiple specialized initial tokens and apply contrastive learning to promote diversity among soft thought representations. Experiments across five reasoning benchmarks and two distinct LLM architectures demonstrate that SoftCoT++ significantly boosts SoftCoT and also outperforms SoftCoT with self-consistency scaling. Moreover, it shows strong compatibility with conventional scaling techniques such as self-consistency. Source code is available at https://github.com/xuyige/SoftCoT.
Forward citations
Cited by 7 Pith papers
-
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
AGS uses the ratio of attention to image tokens vs text tokens to switch multimodal LLMs between latent and explicit reasoning, improving accuracy and cutting decoding steps.
-
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
LOTUS uses a looped padded Transformer with parallel cross-entropy supervision on gold CoT tokens to match explicit CoT performance at 3B parameters while reducing thought-phase latency 2.5x-6.9x.
-
MUX: Continuous Reasoning via Multiplexed Tokens
MUX trains language models to reason with continuous latent tokens that encode spans of discrete reasoning as lossless weighted superpositions, improving accuracy and efficiency over latent-reasoning baselines.
-
LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning
LatentChem reasons in continuous latent space for chemistry, achieving a 59.88% non-tie win rate over explicit CoT on ChemCoTBench with a 10.84x average reduction in reasoning overhead.
-
Measuring Reasoning Utility in LLMs via Conditional Entropy Reduction
In an oracle study on MATH problems, conditional entropy over the answer span decreases across reasoning steps for correct LLM solutions but stays flat or rises for incorrect ones.
-
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
A large survey organizes latent-space work in language-based models by foundation, evolution, four mechanisms, seven abilities, and open challenges.
-
Implicit Reasoning in Large Language Models: A Comprehensive Survey
A survey organizing implicit (silent) reasoning in LLMs into three execution paradigms, plus evidence, benchmarks, and challenges.
Discussion (0). Sign in to comment.