Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration

Hanqi Yan; Lin Gui; Qinglin Zhu; Runcong Zhao; Yudong Chen; Yulan He

REVIEW 3 major objections 5 minor 1 cited by

Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that optimizing the first token's embedding with Gaussian noise and Bayesian search improves LLM reasoning accuracy over temperature sampling and tree search.

desk verdict A real method with solid empirical work, but the central Bayesian-optimization claim is under-tested because no baseline isolates it; still worth serious review. read the letter →

arxiv 2505.24688 v4 pith:342CXSHF submitted 2025-05-30 cs.CL

Qinglin Zhu , Runcong Zhao , Hanqi Yan , Yulan He , Yudong Chen , Lin Gui This is my paper

classification cs.CL

keywords largelanguagemodelsreasoningembeddingperturbationBayesianoptimizationdecodingverifier-guidedsearchfirst-tokensolutionspace

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Soft Reasoning, a decoding-time method that turns the embedding of the first answer token into a search variable. It injects Gaussian noise into that embedding, generates the full answer by greedy decoding, scores the result with a verifier-plus-coherence reward, and then uses Bayesian optimization to pick better perturbations. The authors claim this controlled exploration consistently beats temperature-based sampling, first-token top-k sampling, and tree search on math and commonsense benchmarks, with the largest gains in zero-shot settings. Because the search needs only the first-token logits and ordinary decoding, the method is model-agnostic, requires no parameter access, and uses the same model as its own verifier.

What carries the argument

The load-bearing mechanism is the one-to-one mapping from a perturbed first-token embedding to a complete answer: the embedding is a special token appended to the prompt, and because decoding is greedy, the output sequence $y_i$ is a deterministic function of $x_i$. This turns reasoning-path exploration into black-box optimization of the reward $f(x) = r_{\mathrm{verifier}}(y) + r_{\mathrm{coherence}}(y)$ in the embedding space. The optimization uses expected improvement under a Gaussian-process surrogate, which would be impractical in the full embedding dimension; the paper therefore projects the search onto a random 50-dimensional subspace, relying on the claim that the reward function has effective dimension at most 50.

What would settle it

On a held-out set, run the full method but replace each expected-improvement-selected perturbation with an independent Gaussian perturbation of the same magnitude, keeping the verifier and coherence scoring fixed; if accuracy and coverage are statistically indistinguishable, the Bayesian search contributes nothing beyond random exploration.

Watch

Extended reading notes

Core claim

The paper's central claim is that the solution space of an LLM reasoning problem can be navigated by varying the embedding of the very first generated token, and that this space is smooth enough for Bayesian optimization to find better starting points. Concretely, the framework samples candidate embeddings $x_i = z + \sigma \varepsilon_i$ around the greedy first-token embedding $z$, generates each full answer by greedy decoding so that one injection maps to exactly one output sequence, and scores the batch with a hybrid reward $f(x) = r_{\mathrm{verifier}}(y) + r_{\mathrm{coherence}}(y)$, where the verifier is the same model prompted to regenerate a correct solution from the candidate batch. An expected-improvement acquisition function, run in a random 50-dimensional projection of the embedding space, selects the next perturbation to evaluate. The paper reports that this scheme outperforms self-consistency sampling, FIRE, CoT-Decoding, and RAP across Llama-3.1-8B, Qwen2-7B/70B, and Mistral-7B on GSM8K, GSM-Hard, SVAMP, and StrategyQA, converging within four iterations and using a small fraction of RAP's token and time budget.

Load-bearing premise

The search only pays off if the reward function is well-approximated by a function of about 50 directions in the embedding space, so that optimizing in a random 50-dimensional projection finds genuinely better first-token embeddings than random noise; the paper does not verify this low-dimensional structure directly.

Editorial extensions

If this is right

If Soft Reasoning works as claimed, the first-token embedding becomes a cheap, model-agnostic control point: each question needs only a few embedding evaluations (four iterations at most) rather than hundreds of sampled chains.
The same pipeline transfers to any transformer-based LLM without training or parameter access, because it only requires the score of the first token and standard greedy decoding.
Coverage of the correct answer is consistently the highest among compared methods, so the framework can serve as a better candidate generator for downstream verifiers or ensembling.
A single model can be both generator and verifier: the Multi-Generate verifier (regenerating a consensus answer from the candidate batch) outperforms judge-style verifiers, so no external reward model is needed.
Computational savings are substantial: input tokens average about 6% of RAP's, output tokens about 63%, and inference time about 14%, making the search a practical drop-in for low-resource decoding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the reward landscape over first-token embeddings is genuinely smooth, other gradient-free black-box optimizers should give comparable gains to expected improvement, which would indicate that the search strategy, not the specific acquisition function, drives the result.
The neuron-activation analysis suggests that correct embeddings activate a consistent subset of MLP neurons; a testable extension is to check whether optimized embeddings reliably activate those critical neurons across tasks, which would show whether the method discovers generalizable reasoning circuits.
The paper's ablation shows that naively optimizing more than one token hurts performance; a sequential multi-token schedule that re-estimates the first-token optimum before moving to the next token might capture more of the reasoning-path space without the observed degradation.
A practical user might combine Soft Reasoning with self-consistency voting over the optimized embeddings rather than greedy decoding, since the method already improves coverage; the paper does not test this combination.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a real method with clean mechanics and honest experiments, but the central BO claim is under-tested. The novel recipe—Gaussian perturbation of the first-token embedding, greedy decoding for a deterministic mapping, verifier-plus-coherence reward, then BO in a random 50-d projection—is new relative to FIRE, CoT-Decoding, TSE, and RAP. Code is released, five runs are reported, and the ablations cover reward components, verifier strategies, token placement, reduced dimension, and acquisition function, plus efficiency comparisons against RAP. That is genuine evidence, and the accuracy gains (roughly 3–5 points in zero-shot, at a fraction of RAP's cost) are plausible.

The soft spot is the one the stress test points to: there is no baseline doing random search in the same reduced subspace. Comparing EI against PI and UCB is not the same as comparing BO against random candidate selection. Since the verifier is noisy and the reward is mostly a discrete binary signal plus log-probability coherence, the GP surrogate may be adding little; the method could reduce to 'randomly perturb, generate k candidates, verify.' The ablations keep BO, so they don't isolate the optimization component. That is a real gap, not a manufactured one. The fix is one experiment: same 50-d random projection, same number of candidates, but pick candidates by uniform random sampling instead of EI. If accuracy matches, the contribution becomes the perturbation-plus-verifier recipe, which is still useful but should be described that way.

The theoretical premise is also stated too strongly. Theorem A.2 asserts that for any effective-dimension function there exists u with f(x)=f(Au) for every x. The random-embedding BO literature gives approximate guarantees under bounded domains and low effective dimensionality; exact equality for all x is not what a random Gaussian matrix buys you. The paper should add the approximation argument or soften the claim.

The test-set tuning worry is real too: d, k, epsilon, placement, and the verifier choice all look selected on the same 200-example benchmark slices used for the headline numbers. Some sensitivity curves are reported, but a held-out validation split would make the results considerably cleaner.

Despite all that, the core empirical package is solid enough to merit careful review. The coverage numbers are consistent across models and datasets, the efficiency analysis is useful, and the method is training-free and model-agnostic. The paper would benefit from one decisive control experiment and some claim softening, but I would not desk-reject it. Send it to reviewers.

Referee Report

3 major / 5 minor

Summary. The paper proposes Soft Reasoning, a test-time decoding method that perturbs the embedding of the first generated token with Gaussian noise to create candidate answers, then uses Bayesian optimisation over a random 50-dimensional projection of the embedding space to search for perturbations that maximise a verifier-plus-coherence reward. The final answer is produced by a generation-based verifier (Multi-Generate). The authors evaluate on GSM8K, GSM-Hard, SVAMP, StrategyQA, and AIME-2024 with several open-weight LLMs, report consistent accuracy gains over CoT, Self-Consistency, FIRE, CoT-Decoding, and RAP, and provide ablations of reward components, acquisition functions, projection dimension, token placement, and sample count, along with efficiency and coverage analyses. Code is released.

Significance. If the results hold, the paper makes a useful empirical contribution: it demonstrates that first-token embedding perturbation combined with self-verification can improve reasoning accuracy at substantially lower inference cost than tree search (Table 3), across multiple models and benchmarks. The code release, the multi-model evaluation, and the ablations of reward terms and acquisition functions are strengths, and the efficiency comparison with RAP is a concrete practical contribution. However, the central mechanism claimed by the title and abstract, namely Bayesian optimisation over the embedding space, is not isolated from the verifier-based selection, and key hyperparameters are selected on the same test sets reported in the main results. These issues currently prevent the paper from supporting the strong claim that BO-driven exploration is the source of the accuracy gains.

major comments (3)

[Section 4.2, Table 4, Table 2] There is no baseline that isolates the Bayesian optimisation component. The paper compares EI with PI and UCB (Table 4) but never replaces BO with random search in the same random 50-dimensional subspace and with the same number of LLM evaluations. The ablations in Table 1 remove r_verifier or r_coherence while retaining BO, so they do not test whether the optimisation step itself helps. This is especially important because Table 2 shows that a large majority of test examples terminate after the first or second iteration, so the aggregate result may be dominated by the initial random perturbations plus the Multi-Generate verifier rather than by BO refinement. Please add a control that samples the same number of embeddings uniformly at random in the same reduced subspace and selects the final answer with the same verifier; if that control matches Soft Reasoning, the central 'optimisation' claim would need to be substantially revised.
[Section 5.1, Section 5.3, Appendix B.7] The experimental setup reports results on 200 randomly sampled test examples per dataset (Appendix B.1) but does not describe any held-out validation split. Section 5.3 and Appendix B.7 select the projection dimension d=50 (Figure 6), sample size k=5 (Figure 8), special-token placement 'Last' (Table 6), acquisition function EI (Table 4), and the noise-adaptive parameter delta=0.1 (Appendix A.4) based on accuracy and coverage on the same benchmarks whose results appear in Table 1. This selection procedure can inflate the reported numbers and weakens the generalization claim. Please evaluate these choices on a separate validation set, or report a hyperparameter sensitivity analysis on held-out data.
[Section 4.2, Eq. (1), Appendix A.5] The dimension-reduction argument assumes that the reward function has low effective dimension, but no direct evidence is provided for this premise. Theorem A.2 guarantees exact equivalence of the optimisation problem only if f has effective dimension de <= d; however f(x) = r_verifier(y) + r_coherence(y) in Eq. (1) is a discontinuous, verifier-defined function whose effective dimension over first-token embeddings of an LLM is not established. The empirical curve in Figure 6, measured on test sets, is an indirect justification at best. If the active dimension is larger than 50, the random 50-dimensional projection can miss relevant directions, and optimising g(u)=f(Au) cannot recover the global optimum. Please provide direct diagnostics, such as active-subspace estimation or a comparison of optimisation in R^d for increasing d with matched evaluation budgets, or at least qualify the theoretical claim accordingly.

minor comments (5)

[Section 4.2] The final answer selection is not fully specified: the text defines y_v and the reward f(x), but it is not stated whether the accuracy reported in Table 1 is that of y_v or of the highest-scoring candidate. Please make this explicit in the algorithm description.
[Appendix A.5] The boxed statement of Theorem A.2 omits the 'almost surely' or 'with high probability' qualifier that appears in the surrounding text; for a fixed random matrix A the stated 'for all x' property can fail when the projected active subspace is rank-deficient.
[Table 13] In the SVAMP rows for Mistral-7B-Instruct at shots 1 and 2, the RAP entry reports '0.0 ± 1.2', which appears to be a data-entry error.
[Eq. (1)] The coherence term is an unnormalised sum of log-probabilities and is therefore length-dependent; the paper does not discuss whether this biases optimisation against longer but correct reasoning chains.
[Abstract and Contributions] The claim that the method is 'model-agnostic' is supported only by experiments on three model families; a more cautious wording such as 'tested on several open-weight models' would better match the evidence.

Circularity Check

1 steps flagged · score 2.0 of 10

No circular step reduces the central accuracy claim: the only self-referential element is the openly acknowledged same-model verifier, and all reported results are measured against ground truth.

other [Section 4.2 (Optimisation Objective) and Section 6 (Conclusions)]
"Building on these insights, we propose a verifier-guided approach, where the model evaluates a batch of candidate answers and produces a refined output yv = V(y1:k). ... The verifier is the same model as the generator employed. ... Our approach relies on a verifier that may provide unreliable feedback, impacting optimisation."

The objective f(x) = r_verifier(y) + r_coherence(y) is defined entirely by outputs of the same LLM g_theta that generates the candidates, so the quantity being optimised is the model's self-agreement (a self-scored binary match plus its own token log-probabilities) rather than any external correctness measure. If the paper's own accuracy numbers were defined by this reward, the loop would be closed by construction. But the reported accuracies and coverage rates are evaluated against the ground-truth answers of GSM8K, GSM-Hard, SVAMP and StrategyQA, and the paper explicitly concedes the verifier may be unreliable. The self-reference is thus a limitation of the mechanism, not a definitional reduction of the paper's central empirical claim.

full rationale

I walked the derivation chain from embedding perturbation through Bayesian optimisation to the reported results. (1) The candidate generator is externally defined: xi = z + sigma*epsilon_i produces outputs yi via greedy decoding, so each embedding maps to a unique sequence; this is not defined in terms of the method's own outputs. (2) The BO objective (Eq. 1) is the only self-referential element: r_verifier and r_coherence are both computed from the same model that generates the candidates. This is openly acknowledged (Section 4.2: 'The verifier is the same model as the generator employed'; Section 6: 'Our approach relies on a verifier that may provide unreliable feedback'). It does not make the headline result circular because accuracy (Tables 1, 12, 13) and coverage (Table 14) are measured against ground-truth labels, not against the verifier's judgment. (3) No fitted parameter is renamed a prediction: the hyperparameters (d=50, delta=0.1, epsilon=0.01, k=5, sigma) are fixed implementation choices, and none is derived from the test outputs it is used to predict. (4) The low-dimensional-embedding machinery (Theorem A.2) is imported from external prior works (Wang et al. 2016; Nayebi et al. 2019), not from the present authors' own papers; the only self-citation (Yan et al. 2024, MIRROR, Section 2.2) is related-work context and is not load-bearing. (5) The identified weaknesses - no random-search ablation isolating the BO component, the over-strong 'for all x' phrasing of Theorem A.2, and test-benchmark-based selection of d - are correctness and experimental-design concerns, not circularity, because none of them makes a predicted quantity equal to its input by definition. Overall, the central claim is self-contained against external benchmarks, and the residual self-referentiality is minor and acknowledged.

Assumptions & free parameters 9 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard Bayesian optimization machinery, a random-embedding effective-dimensionality assumption, and the premise that the same LLM can verify its own candidate answers. Most free parameters are set by hand and several (d, k, placement) were chosen using the same test splits reported in the experiments.

free parameters (9)

sigma (perturbation scale)
Controls the magnitude of Gaussian embedding perturbations; no value is reported in the main text, so it is a hand-set hyperparameter.
d (reduced BO dimension) = 50
Selected from accuracy across reduced dimensions in Figure 6.
k (initial samples per iteration) = 5
Selected from Figure 8 where gains plateau after k=5.
K (max BO iterations) = 4
Maximum iterations set in Section 5.1; Table 2 shows most examples terminate earlier.
epsilon (convergence threshold) = 0.01
Set in Section 5.1 and Appendix A.6.
delta (noise-adaptive scaling constant) = 0.1
Set in Appendix A.4 to balance exploration and exploitation.
kernel lengthscale l
RBF kernel bandwidth in Appendix A.2; value not specified.
observation noise lambda
Noise variance in Appendix A.4; value not specified.
objective weights for r_verifier and r_coherence = 1:1
Additive combination in Equation (1) with equal weights.

assumptions (5)

standard math Gaussian process posterior update via Bayes' rule
Used to compute posterior mean and variance in Equation (2), following Rasmussen & Williams (2006).
standard math Random embeddings preserve function values for functions of effective dimension <= d
Theorem A.2 from Wang et al. (2016) and Nayebi et al. (2019) supports optimization in R^d; the assumption that f(x) has effective dimension <= 50 is a domain assumption.
domain assumption The same LLM can serve as a reliable verifier of its own candidates (Multi-Generate)
The reward uses the generator's own regenerated answer as ground truth for correctness; the authors acknowledge this limitation in the conclusions.
domain assumption Coherence score sum of log token probabilities under greedy decoding is a useful fluency measure
Used in Equation (1) to prune incoherent outputs.
domain assumption Injected special token embeddings produce valid decoding trajectories and each embedding maps to one output sequence
Required for the one-to-one mapping between perturbed embeddings and generated answers described in Section 4.1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration." pith.science (2026). https://pith.science/paper/342CXSHF

@misc{pith2026250524688,
  author       = {Pith},
  title        = {Pith review of: Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/342CXSHF}},
  note         = {Machine review of arXiv:2505.24688}
}

read the original abstract

Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution. The code is released at https://github.com/alickzhu/Soft-Reasoning.

Figures

Figures reproduced from arXiv: 2505.24688 by the authors.

**Figure 1.** Comparison of Mainstream and Proposed Approaches. identify promising directions for further exploration based on all observed injection-reward pairs by utilising Bayesian optimisation. Again, thanks to the one-to-one mapping between initial vectors and generation outcomes, this process enables an effective search for optimal vectors that enhance generation quality. A notable advantage of this approach is that it op… view at source ↗

**Figure 2.** Overview of Soft Reasoning. Starting with a natural language question prompt, the model generates initial token embeddings w (1), which, due to greedy decoding, determine the entire output. These embeddings are perturbed to create candidate embeddings x1:k, leading to outputs y1:k through greedy search, which are then evaluated for coherence and verifier feedback. A Bayesian optimisation framework updates its estima… view at source ↗

**Figure 3.** MLP layer activation rates across Transformer layers for the first five tokens of generated answers, sampled 200 times. The curves compare our method (Soft Reasoning) and the SelfConsistency (SC) baseline, further separated into correct and incorrect answers. vation rate of neurons by roughly 3–4% in nearly all layers relative to the Self-Consistency (SC) baseline, suggesting that our perturbations stochastically … view at source ↗

Figures from the paper (4 more)

**Figure 5.** Figure 5: Visualisation of the correlation matrix evolution during Bayesian optimisation across iterations [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Accuracy (solid lines) and standard deviation (shaded areas) across reduced dimensions. presented in Appendix B.4. Impact of Special Token Placement. We compare three ways of inserting the perturbed special token into the prompt: at the beginning (First), somewhere in …

**Figure 7.** Figure 7: Distribution of final accuracy across 50 random projection matrices. Performance remains stable, indicating that random projection does not introduce significant variance. The results indicate that performance remains stable across different random projections, with m…

**Figure 8.** Figure 8: Accuracy (solid lines) and standard deviation (shaded areas) across different sample sizes k. We investigate how the number of sampled embeddings k in each iteration influences final performance [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

A Survey on Latent Reasoning
cs.CL 2025-07 conditional novelty 4.0 of 10

A survey that organizes latent reasoning methods into vertical recurrence, horizontal recurrence, and infinite-depth diffusion, arguing that silent reasoning can beat explicit chain-of-thought.

Reference graph

Works this paper leans on

24 extracted references · 17 canonical work pages · cited by 1 Pith paper

[1]

The amount of salt is 20% of 2000 ml = 0.20×2000 ml = ⟨⟨0.20×2000 = 400⟩⟩400ml

Thought: The total amount of water is 2 liters = 2000 ml. The amount of salt is 20% of 2000 ml = 0.20×2000 ml = ⟨⟨0.20×2000 = 400⟩⟩400ml. Answer: 400

work page 2000
[2]

Since there are 1000 ml in 1 liter, 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml

Thought: 20% of 2 liters is 2× 20 100 =⟨⟨2×20/100 = 0.4⟩⟩0.4 liters. Since there are 1000 ml in 1 liter, 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml. Answer: 400

work page
[3]

So, 1 liter of seawater has 20%×1 liter = ⟨⟨20×0.1 = 0.2⟩⟩0.2 liters of salt

Thought: 1 liter of seawater is 20% salt. So, 1 liter of seawater has 20%×1 liter = ⟨⟨20×0.1 = 0.2⟩⟩0.2 liters of salt. Since Jack has 2 liters of seawater, he will get 0.2×2 =⟨⟨0.2×2 = 0.4⟩⟩0.4 liters of salt. Since there are 1000 ml in 1 liter, Jack will get0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml of salt. Answer: 400

work page
[4]

There are 1000 ml in 1 liter, so 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml

Thought: 20% of 2 liters is 2× 20 100 =⟨⟨2×20/100 = 0.4⟩⟩0.4 liters. There are 1000 ml in 1 liter, so 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml. Answer: 400 Correct: 1, 2, 3, 4 Question: {User Question} Your previous answers: {Previous answers} Correct: 18 Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embeddi...

work page 2000
[9]

Mockus, J

URL https://openreview.net/forum? id=FBkpCyujtS. Mockus, J. On bayesian methods for seeking the extremum. InOptimization Techniques IFIP Technical Conference, pp. 400–404. Springer, 1975. Mudgal, S., Lee, J., Ganapathy, H., Li, Y ., Wang, T., Huang, Y ., Chen, Z., Cheng, H.-T., Collins, M., Strohman, T., Chen, J., Beutel, A., and Beirami, A. Controlled de...

work page 1975
[10]

Last modified: 13 Nov 2024

URL https://openreview.net/forum? id=zHUznVsBZp. Last modified: 13 Nov 2024. Patel, A., Bhattamishra, S., and Goyal, N. Are NLP models really able to solve simple math word prob- lems? InProceedings of the 2021 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, pp. 2080–2094, Online, J...

doi:10.18653/v1/2021.naacl-main 2024
[12]

Srinivas, N., Krause, A., Kakade, S., and Seeger, M

URL https://openreview.net/forum? id=4FWAwZtd2n. Srinivas, N., Krause, A., Kakade, S., and Seeger, M. Gaus- sian process optimization in the bandit setting: no regret and experimental design. InProceedings of the 27th In- ternational Conference on International Conference on Machine Learning, ICML’10, pp. 1015–1022, Madison, WI, USA, 2010. Omnipress. ISBN...

2010
[13]

Wang, X., Jin, Y ., Schmitt, S., and Olhofer, M

URL https://openreview.net/forum? id=4Zt7S0B0Jp. Wang, X., Jin, Y ., Schmitt, S., and Olhofer, M. Recent advances in bayesian optimization.ACM Computing Surveys, 55(13s):1–36, 2023b. doi: 10.1145/3582078. Wang, X., Wei, J., Schuurmans, D., Le, Q. V ., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency improves chain of thought reasoning ...

arXiv

Show all 24 references

[15]

Answer: 1000

Thought: 1250 ml of water evaporates, leaving 1000 ml of salt. Answer: 1000
[20]

For 12 people, she needs 12× 6 8 = 9ounces of tea

Thought: 8 ounces of tea for 1 cup, so 1 ounce of tea for 1 8 of a cup. For 12 people, she needs 12× 6 8 = 9ounces of tea. Answer: 9
[21]

Since there are 12 people, 12×6 = 72 ounces of tea are needed

Thought: 6 ounces of tea is needed for each person. Since there are 12 people, 12×6 = 72 ounces of tea are needed. Since each ounce of tea is used for 1 cup, 72 ounces of tea will make 72 cups of tea. Answer: 72
[22]

For 12 people, she needs12× 3 4 = 9ounces of tea

Thought: 6 ounces of tea is 6 8 = 3 4 of an 8-ounce cup. For 12 people, she needs12× 3 4 = 9ounces of tea. Answer: 9
[23]

Since each ounce of tea is used for 1 cup, Artemis needs 72 ounces of tea

Thought: 12×6 = 72 ounces of tea needed. Since each ounce of tea is used for 1 cup, Artemis needs 72 ounces of tea. Answer: 72
[24]

So for 6 ounces of tea, she will use 6 8 = 3 4 of the amount of tea

Thought: 8 ounces of tea is used for 1 cup. So for 6 ounces of tea, she will use 6 8 = 3 4 of the amount of tea. For 12 people, she will need12× 3 4 = 9ounces of tea. Answer: 9 Analysis: Let’s think step by step. Artemis uses 1 ounce of tea for an 8-ounce cup, so for a 6-ounce...

2024
[67]

findings-acl.67

URL https://aclanthology.org/2023. findings-acl.67. Hvarfner, C., Hellsten, E. O., and Nardi, L. Vanilla bayesian optimization performs great in high dimensions. InPro- ceedings of the 41st International Conference on Machine Learning (ICML), volume 235, Vienna, Austria, 2024....

2023 arXiv
[168]

naacl-main.168

URL https://aclanthology.org/2021. naacl-main.168. Qi, X., Panda, A., Lyu, K., Ma, X., Roy, S., Beirami, A., Mittal, P., and Henderson, P. Safety alignment should be made more than just a few tokens deep. InThe Thirteenth International Conference on Learning Representations, 2...

2021
[197]

key neurons

URL https://aclanthology.org/2024. acl-long.197/. 13 Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration A. Methodology Supplement A.1. Technical Details on Optimisation Objective Although we simply sum the two component...

2024
[442]

findings-emnlp.442/

URL https://aclanthology.org/2023. findings-emnlp.442/. Lightman, H., Kosaraju, V ., Burda, Y ., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations,

2023
[507]

emnlp-main.507/

URL https://aclanthology.org/2023. emnlp-main.507/. 10 Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y . The curious case of neural text degeneration. In Intern...

2023
[589]

emnlp-main.589/

URL https://aclanthology.org/2024. emnlp-main.589/. Brown, B., Juravsky, J., Ehrlich, R., Clark, R., Le, Q. V ., R´e, C., and Mirhoseini, A. Large language monkeys: Scaling inference compute with repeated sampling, 2024. URL https://arxiv.org/abs/2407.21787. Chen, W., Zhang, Z...

2024 arXiv
[2020]

Huang, J

URL https://openreview.net/forum? id=rygGQyrFvH. Huang, J. and Chang, K. C.-C. Towards reasoning in large language models: A survey. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.),Findings of the Association for Computational Linguistics: ACL 2023, pp. 1049–1065, Toron...

2023 doi
[2022]

URL https://openreview.net/forum? id=-h6WAS6eE4. Meta. Introducing meta llama3: The most capable openly available llm to date, 2024. URL https://ai.meta. com/blog/meta-llama-3/. Miao, N., Teh, Y . W., and Rainforth, T. Selfcheck: Using llms to zero-shot check their own step-by...

2024
[2024]

Meng, K., Bau, D., Andonian, A

URL https://openreview.net/forum? id=v8L0pN6EOi. Meng, K., Bau, D., Andonian, A. J., and Belinkov, Y . Lo- cating and editing factual associations in GPT. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems,
[2025]

ISBN 979-8-89176-195-7

Association for Computational Linguistics. ISBN 979-8-89176-195-7. URL https://aclanthology. org/2025.findings-naacl.396/. Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Train...

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

The amount of salt is 20% of 2000 ml = 0.20×2000 ml = ⟨⟨0.20×2000 = 400⟩⟩400ml

Thought: The total amount of water is 2 liters = 2000 ml. The amount of salt is 20% of 2000 ml = 0.20×2000 ml = ⟨⟨0.20×2000 = 400⟩⟩400ml. Answer: 400

work page 2000

[2] [2]

Since there are 1000 ml in 1 liter, 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml

Thought: 20% of 2 liters is 2× 20 100 =⟨⟨2×20/100 = 0.4⟩⟩0.4 liters. Since there are 1000 ml in 1 liter, 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml. Answer: 400

work page

[3] [3]

So, 1 liter of seawater has 20%×1 liter = ⟨⟨20×0.1 = 0.2⟩⟩0.2 liters of salt

Thought: 1 liter of seawater is 20% salt. So, 1 liter of seawater has 20%×1 liter = ⟨⟨20×0.1 = 0.2⟩⟩0.2 liters of salt. Since Jack has 2 liters of seawater, he will get 0.2×2 =⟨⟨0.2×2 = 0.4⟩⟩0.4 liters of salt. Since there are 1000 ml in 1 liter, Jack will get0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml of salt. Answer: 400

work page

[4] [4]

There are 1000 ml in 1 liter, so 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml

Thought: 20% of 2 liters is 2× 20 100 =⟨⟨2×20/100 = 0.4⟩⟩0.4 liters. There are 1000 ml in 1 liter, so 0.4 liters is 0.4×1000 =⟨⟨0.4×1000 = 400⟩⟩400ml. Answer: 400 Correct: 1, 2, 3, 4 Question: {User Question} Your previous answers: {Previous answers} Correct: 18 Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embeddi...

work page 2000

[5] [9]

Mockus, J

URL https://openreview.net/forum? id=FBkpCyujtS. Mockus, J. On bayesian methods for seeking the extremum. InOptimization Techniques IFIP Technical Conference, pp. 400–404. Springer, 1975. Mudgal, S., Lee, J., Ganapathy, H., Li, Y ., Wang, T., Huang, Y ., Chen, Z., Cheng, H.-T., Collins, M., Strohman, T., Chen, J., Beutel, A., and Beirami, A. Controlled de...

work page 1975

[6] [10]

Last modified: 13 Nov 2024

URL https://openreview.net/forum? id=zHUznVsBZp. Last modified: 13 Nov 2024. Patel, A., Bhattamishra, S., and Goyal, N. Are NLP models really able to solve simple math word prob- lems? InProceedings of the 2021 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, pp. 2080–2094, Online, J...

doi:10.18653/v1/2021.naacl-main 2024

[7] [12]

Srinivas, N., Krause, A., Kakade, S., and Seeger, M

URL https://openreview.net/forum? id=4FWAwZtd2n. Srinivas, N., Krause, A., Kakade, S., and Seeger, M. Gaus- sian process optimization in the bandit setting: no regret and experimental design. InProceedings of the 27th In- ternational Conference on International Conference on Machine Learning, ICML’10, pp. 1015–1022, Madison, WI, USA, 2010. Omnipress. ISBN...

2010

[8] [13]

Wang, X., Jin, Y ., Schmitt, S., and Olhofer, M

URL https://openreview.net/forum? id=4Zt7S0B0Jp. Wang, X., Jin, Y ., Schmitt, S., and Olhofer, M. Recent advances in bayesian optimization.ACM Computing Surveys, 55(13s):1–36, 2023b. doi: 10.1145/3582078. Wang, X., Wei, J., Schuurmans, D., Le, Q. V ., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency improves chain of thought reasoning ...

arXiv