Pith. sign in

REVIEW 3 major objections 5 minor 43 references

Accelerating Large Language Model Reasoning via Speculative Search

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SpecSearch accelerates tree-search-based LLM reasoning by having a small model draft whole thoughts, reject low-quality ones, and correct them with token-level speculative decoding.

desk verdict A genuinely new bi-level speculative search method with consistent empirical speedups, but the advertised quality-preservation theorem has a proof error that changes the first-step bound from ~0.9995 to 0.5. read the letter →

arxiv 2505.02865 v2 pith:FRJLIEQX submitted 2025-05-03 cs.CL cs.AI

Zhihai Wang , Jie Wang , Jilai Pan , Xilin Xia , Huiling Zhen , Mingxuan Yuan , Jianye Hao , Feng Wu This is my paper · ORCID
classification cs.CLcs.AI
keywords speculativedecodingtree-searchreasoningthought-levelspeculationprocessrewardmodelrejectionsamplingLLMinferenceaccelerationbeamsearchMCTS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speculative Search (SpecSearch) claims that the primary latency of tree-search-based LLM reasoning—generating many reasoning thoughts—can be shifted onto a small model without sacrificing the big model's reasoning quality. It drafts full thoughts in parallel with a small model, scores them with a process reward model, keeps the ones above a dynamic threshold, and regenerates rejected thoughts through lossless token-level speculative decoding. The threshold is an exponential moving average of the large model's historical thought scores, so no extra forward passes are needed. The paper proves that if the threshold stays at or above the large model's mean thought quality, the speculative generator's expected quality is undegraded, and reports speedups up to 4.11x over autoregressive tree search and 2.12x over token-level speculative decoding on math benchmarks while staying within a point of baseline accuracy.

What carries the argument

The load-bearing object is the quality-preserving rejection mechanism and its step-wise threshold. The threshold updates as an exponential moving average of the large model's recent thought scores: $\hat{\beta}^{(k+1)} = \theta \hat{\beta}^{(k)} + (1-\theta)\,\Theta(V_p^{(k)})$, where $\Theta$ is a nonparametric estimator such as the sample mean or the maximum. This converts the problem of matching the large model's reasoning quality into a statistical estimation problem: estimate the large model's thought-quality distribution from history, set the rejection bar there, and draft cheaply. The key identity is that, under a truncated-normal model for accepted thoughts, the mixture of accepted small-model thoughts and corrected large-model thoughts has expected process-reward score at least the large model's, provided the threshold is at least the large model's mean quality.

What would settle it

Take a set of problems where an incorrect intermediate step is known to receive a high process-reward score (the paper's Case 3 provides an example: an erroneous step scored 0.8916). Run SpecSearch and autoregressive tree search on that set and compare final-answer accuracy; if SpecSearch retains the misleading step and its accuracy falls below the large model's, the quality-preserving guarantee is protecting the reward score rather than reasoning correctness.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that speculative execution can be lifted from tokens to reasoning thoughts without giving up quality. The generator $G_s$ operates in a draft-evaluate-reject-correct loop: the small model $G_q$ drafts $N$ thoughts, the process reward model $V$ scores them, thoughts with $V(z) \ge \hat{\beta}^{(k)}$ are accepted, and rejected ones are regenerated by a lossless token-level speculative model $G_p$. The theoretical guarantee is the undegraded quality condition $\mathbb{E}_{z \sim G_s}[V(z)] \ge \mathbb{E}_{z \sim G_p}[V(z)]$. Theorem 4.3 shows this holds whenever the threshold satisfies $\beta^{(k)} \ge \mu_p^{(k)}$, the large model's mean thought quality at step $k$, under a normality assumption on thought qualities. Theorems 4.5 and 4.6 bound the probability that the estimated threshold stays above the large model's quality, with the bound tending to 1 as the drafting width $N$ grows. Empirically, on GSM8K-100 and MATH-100 with Qwen and Llama models, SpecSearch reports speedups up to 2.12x over speculative decoding and up to 4.11x over autoregressive search, with accuracy within about one point on the main sets.

Load-bearing premise

The load-bearing premise is that the scoring model used to judge each reasoning step is a faithful proxy for actual correctness: if a wrong step can receive a high score, the rejection mechanism preserves the score, not the answer.

Editorial extensions

If this is right

  • Tree-search reasoning can run at near-small-model speed while keeping the large model's expected thought quality, as long as the threshold tracks the large model's quality.
  • SpecSearch works on both Qwen and Llama stacks, with beam search and MCTS, and with different process reward models, so the acceleration is not tied to one search algorithm or evaluator.
  • Because the probability bound improves as the drafting width N grows, wider speculative drafting is both a speed lever and a quality lever.
  • The lossless token-level correction phase means rejected thoughts are replaced by draws from the large model's own token distribution, so the final reasoning path stays anchored to the large model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the scorer's blind spots are the real limitation, SpecSearch could be paired with an outcome verifier or a correctness classifier trained on final answers; the same threshold machinery would then preserve final-answer accuracy instead of reward score.
  • The descending step-wise threshold is itself a useful diagnostic: it quantifies how much harder later reasoning steps are for the large model, which could inform where to spend extra compute.
  • The draft model can be as small as 0.5B, so a practical extension is to search over draft-model size and threshold aggressiveness to find a compute-optimal operating point for a given accuracy budget.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SpecSearch, a framework to accelerate tree-search-based LLM reasoning by combining a small draft model with a large model at two levels: the small model generates candidate reasoning thoughts in parallel, a process reward model (PRM) scores them, and a threshold-based rejection mechanism accepts thoughts whose estimated quality is at least the large model's expected quality; rejected thoughts are regenerated with token-level speculative decoding. The threshold is updated with an exponential moving average of a nonparametric estimator over historical large-model thought quality. The paper claims a theoretical guarantee of undegraded reasoning quality, defined as expected PRM score, and reports speedups up to 2.12x over speculative decoding and 4.11x over autoregressive tree search on 100-question subsets of MATH and GSM8K, with additional results on full GSM8K, AIME, Olympiad Bench, and HumanEval.

Significance. The core idea is timely and useful: treating a reasoning thought as a speculative unit, not just a token sequence, is a natural extension of speculative decoding to search-based reasoning, and the reported speedups are consistent across several models, search algorithms, and PRM choices. The paper also deserves credit for including a theoretical section, ablations of the rejection and evaluation modules, and a case study that candidly documents a failure mode of the PRM. However, the central theoretical guarantee is not established as stated, and the empirical evidence for 'comparable reasoning quality' is thin relative to the strength of the claim. If the theoretical issue is repaired or the claim is weakened, and the empirical evaluation is made more rigorous, the contribution would be solid; in its current form the advertised guarantee exceeds what the proof and experiments support.

major comments (3)
  1. [Appendix A.4, Eq. (40) and Theorem 4.6] The proof of Theorem 4.6 miscomputes the initial threshold probability. Immediately before Eq. (40), the proof uses the sample-mean estimator: Eq. (3) defines beta^(k+1) as an EMA of the sample mean, and Eq. (39) gives Var[beta^(1)] = (sigma_p^(0))^2/(N+1), which is the variance of a sample mean of N+1 Gaussian draws. For such a sample mean, P(beta^(1) <= mu_p^(0)) = 1/2 by symmetry, not 2^{-(N+1)}. The value 2^{-(N+1)} is the probability that the maximum of N+1 draws is below the mean, which is not the estimator used in the proof. Consequently, the factor (1 - 2^{-(N+1)}) in Theorem 4.6 and the numerical example in Appendix A.5.1, which reports a bound of 0.90 at step 10 under N=10, are not justified; with the correct initial survival probability of roughly 1/2, that numerical support disappears. This is an internal inconsistency in the theoretical guarantee and must be repaired or the theorem restated in weaker form.
  2. [Section 4.3 and Appendix H.3] The quality-preserving guarantee operates entirely on the PRM score V(z), but the paper's central claim is 'comparable reasoning quality' measured by final-answer accuracy. The manuscript itself shows the gap between these two notions: in Appendix H.3, Case 3, an incorrect reasoning step receives a PRM score of 0.8916, and the experiment section reports accuracy drops on AIME (16.67 to 13.33) and Olympiad Bench (63.75 to 58.75) that are labeled 'comparable' without error bars or significance tests. The rejection mechanism provably preserves expected PRM score, not expected correctness; if V is a poor proxy for correctness, the theorem does not support the accuracy claim. The authors should either validate the PRM as a correctness proxy on their tasks, or explicitly reframe the theoretical and empirical claims as being about PRM score, with accuracy reported as a secondary, exploratory outcome.
  3. [Section 5 and Appendix D.1] The main empirical evidence is based on 100-question (and in the ablation, 50-question) test sets with a single run per condition and no confidence intervals, bootstrap errors, or significance tests. Differences of 1-3 accuracy points on 100 questions are within sampling noise, and the paper's phrasing such as 'comparable reasoning quality' and 'minimal accuracy loss' is stronger than the data support. In addition, the related work discusses SEED as a token-level speculative decoding baseline for tree-search reasoning, but SEED is not included in the experimental comparison; given that the paper's claimed advantage is specifically thought-level over token-level speculation, a SEED baseline would substantially strengthen the comparison. I recommend adding uncertainty quantification and at least one token-level tree-search acceleration baseline, or tempering the claims accordingly.
minor comments (5)
  1. [Section 4.2] There is a typo: 'see Tabel 1' should be 'see Table 1'.
  2. [Appendix A.5.1] The phrase 'The bound decent rapidly' should read 'The bound decays rapidly'.
  3. [Appendix A.5.3, Eq. (49)] The probability bound in Eq. (49) seems to have a product index k=0 to K while the event is written as (k+1) variables; please check the indexing for consistency with Theorem 4.6.
  4. [Appendix H.3] The sentence 'This observation clarifies why we observed low precision loss in our SpecSearch' is unclear; 'precision loss' is not a defined metric, and the case study actually illustrates a loss in accuracy caused by PRM deception.
  5. [Section 5, Experiment 4] Figure 4 shows average reward scores at each reasoning step; it would be helpful to state explicitly that this figure measures PRM score, not final-answer accuracy, to avoid conflating the two notions of quality.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the quality-preserving theorem is a conditional lemma, the threshold estimator is not fitted to the predicted quantity, and no load-bearing self-citation is present.

full rationale

The paper's central formal claim, Theorem 4.3, is explicitly conditional: if the step threshold satisfies beta(k) >= mu_p^(k), then the mixture of above-threshold small-model thoughts and large-model corrected thoughts has expected PRM score at least mu_p^(k). The proof derives this by computing the truncated-normal conditional mean, rather than assuming the conclusion; the burden of establishing the condition beta(k) >= mu_p^(k) is deferred to Theorems 4.5 and 4.6, which give probability bounds under the stated descending-quality and bounded-variance assumptions. This is not self-definitional: the undegraded-quality inequality is not an input to the rejection mechanism, and the theorem's conclusion does not reappear as an assumption. The step-wise threshold is an EMA of historical large-model PRM scores, and the 'prediction' concerns future large-model quality under an explicit monotonicity assumption; this is an extrapolation, not a fitted parameter renamed as a prediction. The experimental speedups and accuracies are measured against external baselines and are not derived from the fitted threshold, so there is no fitted-input-called-prediction pattern. Self-citations, to SEED and OpenR, are background related work or experimental infrastructure; no uniqueness theorem, ansatz, or load-bearing claim is imported from the authors' prior work, and the statistical tools cited (truncated normal, Cantelli's inequality, EMA) are standard external results. The paper does contain a serious independent proof error in Theorem 4.6 (the initial probability P(beta-hat^(1) <= mu_p^(0)) is claimed to be 2^{-(N+1)} for what the proof treats as a sample mean, whereas a sample mean of N+1 Gaussians is at or below its mean with probability 1/2), and Appendix H.3 concedes that an incorrect reasoning step can receive a high PRM score of 0.8916. These are correctness and construct-validity concerns, not circularity: they attack whether the advertised theoretical guarantee is proven and whether the PRM score faithfully tracks reasoning correctness, but they do not show that any derivation reduces to its own inputs by construction. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on four unverified premises: Gaussian thought-quality distributions, a decreasing large-model quality profile with bounded variance, the fidelity of the PRM as a correctness proxy, and the use of accepted small-model thoughts as an upper bound on large-model quality. The first three are needed for the theoretical guarantee; the fourth is needed for the practical threshold estimator. No new physical entities are introduced. The main free parameter is the EMA weight theta, plus experimental settings N and b inherited from OpenR.

free parameters (2)
  • EMA weight theta = 0.9
    Controls the balance between historical threshold and current large-model quality estimate in the EMA update (Algorithm 1 line 14). Chosen by hand; sensitivity analysis in Table 11 shows accuracy roughly stable for theta in 0.8-0.95, with latency decreasing as theta increases.
  • Expansion width N and beam size b = N=6, b=2
    Experiment settings inherited from OpenR; they set the number of drafted thoughts per node and the beam width, and affect both latency and accuracy. Not fitted by the paper, but they are choices the central speedup numbers depend on.
assumptions (4)
  • domain assumption At each reasoning step, the qualities of thoughts generated by the large model Gp and small model Gq are independent and normally distributed with means mu_p, mu_q and mu_p >= mu_q (Section 4.4).
    Needed for the truncated-normal expectation in the proof of Theorem 4.3 and for the probability bounds in Theorems 4.5 and 4.6. Not validated against the reward distributions in Figure 7.
  • domain assumption Assumption 4.4: the large model's mean thought quality decreases across steps, mu_p^(k) <= gamma mu_p^(k-1) with gamma < 1, and variance is bounded by sigma_c^2.
    The descending-quality part is inferred from one figure (Appendix H.2, Figure 7) and is required for the EMA bias argument in Theorem 4.5. The bound requires the experimental theta=0.9 to be at least gamma, but gamma is never estimated.
  • domain assumption The process reward model V(z) is a valid measure of reasoning quality whose expectation is the right objective to preserve.
    All theoretical guarantees are about E[V], while the experiments measure final-answer accuracy. The paper's own Case 3 (Appendix H.3) shows an incorrect step scoring 0.89, indicating the PRM can be deceived.
  • ad hoc to paper Accepted small-model thoughts can be treated as an approximate upper bound on the large model's quality when forming the threshold estimate (Section 4.3).
    This assumption lets the practical estimator add small-model samples into the max in Eq. 50. It is plausible when small-model best thoughts exceed the large-model mean, but it is not proven and biases the threshold upward.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerating Large Language Model Reasoning via Speculative Search." pith.science (2026). https://pith.science/paper/FRJLIEQX

@misc{pith2026250502865,
  author       = {Pith},
  title        = {Pith review of: Accelerating Large Language Model Reasoning via Speculative Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FRJLIEQX}},
  note         = {Machine review of arXiv:2505.02865}
}
abstract

Tree-search-based reasoning methods have significantly enhanced the reasoning capability of large language models (LLMs) by facilitating the exploration of multiple intermediate reasoning steps, i.e., thoughts. However, these methods suffer from substantial inference latency, as they have to generate numerous reasoning thoughts, severely limiting LLM applicability. To address this challenge, we propose a novel Speculative Search (SpecSearch) framework that significantly accelerates LLM reasoning by optimizing thought generation. Specifically, SpecSearch utilizes a small model to strategically collaborate with a large model at both thought and token levels, efficiently generating high-quality reasoning thoughts. The major pillar of SpecSearch is a novel quality-preserving rejection mechanism, which effectively filters out thoughts whose quality falls below that of the large model's outputs. Moreover, we show that SpecSearch preserves comparable reasoning quality to the large model. Experiments on both the Qwen and Llama models demonstrate that SpecSearch significantly outperforms state-of-the-art approaches, achieving up to 2.12$\times$ speedup with comparable reasoning quality.

Figures

Figures reproduced from arXiv: 2505.02865 by the authors.

Figure 1
Figure 1. (a) The inference latency increases by several orders of magnitude with the introduction of tree-search-based reasoning methods. (b) Thought generation acts as an efficiency bottleneck of tree-search-based reasoning methods. approach (Wei et al., 2022), which decomposes problem￾solving into a sequence of intermediate reasoning steps, termed thoughts. Building upon this, TSB frameworks such as Tree-of-Thoughts (TOT) … view at source ↗
Figure 2
Figure 2. (a) Small models can generate thoughts with high re￾ward scores. (b) Simple large model engagement strategies at the thought level struggle to preserve comparable reasoning quality. is a hyperparameter) is reprocessed by the large model for refinement. However, as shown in Figure 2b, maintaining reasoning quality remains challenging when collaboration occurs at the thought level. Overview of SpecSearch Building on t… view at source ↗
Figure 3
Figure 3. Illustration of our proposed SpecSearch. SpecSearch proposes a bi-level speculative thought generator with a quality-preserving rejection mechanism, which significantly accelerates LLM reasoning while preserving comparable quality. grained collaboration, increasing the number of tokens gen￾erated by the small model throughout the search process. This, in turn, can significantly enhance the efficiency of thought gene… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: To verify that our method preserves comparable reward scores for reasoning thoughts, we visualize the average reward scores at each reasoning step during the tree search process. Sensitivity Analysis (1) The EMA Weight θ. We analyze the sensitivity of SpecSearch to the…
Figure 5
Figure 5. Figure 5: The bound decent rapidly with the reasoning steps. However, the bound remains as high as 0.90 even at the 10-th step. A.5.2. THRESHOLD ESTIMATOR WITH MAXIMUM ESTIMATION In practice, due to the limited number of samples, the accuracy of the average estimation method ten…
Figure 6
Figure 6. Figure 6: (a) Illustration of standard speculative decoding methods. (b) Illustration of the beam-search-based reasoning method. Details on Speculative Sampling Here is a detailed introduction of speculative sampling (SpS) (Leviathan et al., 2023; Chen et al., 2023a), a state-of…
Figure 7
Figure 7. Figure 7: The distribution of rewards for generated thoughts decreases step by step [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: A difficult case study showing how different reasoning steps have different levels of difficulty. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 9
Figure 9. Figure 9: A simple case study showing how different reasoning steps have different levels of difficulty. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_9.png]
Figure 10
Figure 10. Figure 10: A case study showing the PRM score on the wrong reasoning path. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 18 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    The truncated normal distribution

    Burkardt, J. The truncated normal distribution. Department of Scientific Computing Website, Florida State University, 1 0 (35): 0 58, 2014

    work page 2014

  3. [3]

    Burton, F. W. Speculative computation, parallelism, and functional programming. IEEE Trans. Computers , 34 0 (12): 0 1190--1193, 1985. doi:10.1109/TC.1985.6312218. URL https://doi.org/10.1109/TC.1985.6312218

    arXiv 1985

  4. [4]

    Accelerating large language model decoding with speculative sampling

    Chen, C., Borgeaud, S., Irving, G., Lespiau, J., Sifre, L., and Jumper, J. Accelerating large language model decoding with speculative sampling. CoRR, abs/2302.01318, 2023 a . doi:10.48550/ARXIV.2302.01318. URL https://doi.org/10.48550/arXiv.2302.01318

    arXiv doi:10.48550/arxiv.2302.01318 2023

  5. [5]

    Alphamath almost zero: process supervision without process

    Chen, G., Liao, M., Li, C., and Fan, K. Alphamath almost zero: process supervision without process. CoRR, abs/2405.03553, 2024. doi:10.48550/ARXIV.2405.03553. URL https://doi.org/10.48550/arXiv.2405.03553

    arXiv doi:10.48550/arxiv.2405.03553 2024

  6. [6]

    Chen, Z., Yang, X., Lin, J., Sun, C., Huang, J., and Chang, K. C. Cascade speculative drafting for even faster LLM inference. CoRR, abs/2312.11462, 2023 b . doi:10.48550/ARXIV.2312.11462. URL https://doi.org/10.48550/arXiv.2312.11462

    arXiv doi:10.48550/arxiv.2312.11462 2023

  7. [7]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168

    arXiv 2021

  8. [8]

    M., Kraaikamp, C., Lopuha \"a , H

    Dekking, F. M., Kraaikamp, C., Lopuha \"a , H. P., and Meester, L. E. A Modern Introduction to Probability and Statistics: Understanding why and how. Springer Science & Business Media, 2006

    work page 2006

Show all 43 references
  1. [9]

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al - Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Roz...

    2024 doi arXiv

  2. [10]

    Interpretable contrastive monte carlo tree search reasoning

    Gao, Z., Niu, B., He, X., Xu, H., Liu, H., Liu, A., Hu, X., and Wen, L. Interpretable contrastive monte carlo tree search reasoning. CoRR, abs/2410.01707, 2024. doi:10.48550/ARXIV.2410.01707. URL https://doi.org/10.48550/arXiv.2410.01707

    2024 doi arXiv

  3. [11]

    Gopinath, R. A. Maximum likelihood modeling with gaussian distributions for classification. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP'98 (Cat. No. 98CH36181), volume 2, pp.\ 661--664. IEEE, 1998

    1998

  4. [12]

    J., Wang, Z., Wang, D

    Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., and Hu, Z. Reasoning with language model is planning with world model. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023...

    2023 doi

  5. [13]

    Measuring mathematical problem solving with the MATH dataset

    Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring mathematical problem solving with the MATH dataset. In Vanschoren, J. and Yeung, S. (eds.), Proceedings of the Neural Information Processing Systems Track on Dataset...

    2021

  6. [14]

    Hennessy, J. L. and Patterson, D. A. Computer Architecture - A Quantitative Approach, 5th Edition . Morgan Kaufmann, 2012. ISBN 978-0-12-383872-8

    2012

  7. [15]

    Rot: Enhancing large language models with reflection on search trees

    Hui, W., Jiang, C., Wang, Y., and Tu, K. Rot: Enhancing large language models with reflection on search trees. CoRR, abs/2404.05449, 2024. doi:10.48550/ARXIV.2404.05449. URL https://doi.org/10.48550/arXiv.2404.05449

    2024 doi arXiv

  8. [16]

    X., Liu, Z., Yan, D., Xie, J., Wang, Z., and Wen, J

    Jiang, J., Chen, Z., Min, Y., Chen, J., Cheng, X., Wang, J., Tang, Y., Sun, H., Deng, J., Zhao, W. X., Liu, Z., Yan, D., Xie, J., Wang, Z., and Wen, J. Technical report: Enhancing LLM reasoning with reward-guided tree search. CoRR, abs/2411.11694, 2024. doi:10.48550/ARXIV.2411...

    2024 doi arXiv

  9. [17]

    Z., Chen, X., Kazemi, A., and Chen, B

    Kang, J., Li, X. Z., Chen, X., Kazemi, A., and Chen, B. Mindstar: Enhancing math reasoning in pre-trained llms at inference time. CoRR, abs/2405.16265, 2024. doi:10.48550/ARXIV.2405.16265. URL https://doi.org/10.48550/arXiv.2405.16265

    2024 doi arXiv

  10. [18]

    Exponential moving average versus moving exponential average

    Klinker, F. Exponential moving average versus moving exponential average. Mathematische Semesterberichte, 58: 0 97--107, 2011

    2011

  11. [19]

    Cllms: Consistency large language models

    Kou, S., Hu, L., He, Z., Deng, Z., and Zhang, H. Cllms: Consistency large language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=8uzBOVmh8H

    2024

  12. [20]

    H., Gonzalez, J., Zhang, H., and Stoica, I

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Flinn, J., Seltzer, M. I., Druschel, P., Kaufmann, A., and Mace, J. (eds.), Proceedings o...

    2023

  13. [21]

    Crafting papers on machine learning

    Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp.\ 1207--1216, Stanford, CA, 2000. Morgan Kaufmann

    2000

  14. [22]

    Fast inference from transformers via speculative decoding

    Leviathan, Y., Kalman, M., and Matias, Y. Fast inference from transformers via speculative decoding. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, ...

    2023

  15. [23]

    EAGLE: speculative sampling requires rethinking feature uncertainty

    Li, Y., Wei, F., Zhang, C., and Zhang, H. EAGLE: speculative sampling requires rethinking feature uncertainty. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=...

    2024

  16. [24]

    M., and Huang, C

    Lu, J., Yang, Z., Wang, Y., Liu, X., Namee, B. M., and Huang, C. Padellm-ner: Parallel decoding in large language models for named entity recognition. CoRR, abs/2402.04838, 2024. doi:10.48550/ARXIV.2402.04838. URL https://doi.org/10.48550/arXiv.2402.04838

    2024 doi arXiv

  17. [25]

    Treebon: Enhancing inference-time alignment with speculative tree-search and best-of-n sampling

    Qiu, J., Lu, Y., Zeng, Y., Guo, J., Geng, J., Wang, H., Huang, K., Wu, Y., and Wang, M. Treebon: Enhancing inference-time alignment with speculative tree-search and best-of-n sampling. arXiv preprint arXiv:2410.16033, 2024

    2024 arXiv

  18. [26]

    Qwen2.5: A party of foundation models, September 2024

    Team, Q. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  19. [27]

    M., Wen, Y., Zhang, W., and Wang, J

    Wan, Z., Feng, X., Wen, M., McAleer, S. M., Wen, Y., Zhang, W., and Wang, J. Alphazero-like tree-search can guide large language model decoding and training. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview....

    2024

  20. [28]

    Efficient large language models: A survey

    Wan, Z., Wang, X., Liu, C., Alam, S., Zheng, Y., Liu, J., Qu, Z., Yan, S., Zhu, Y., Zhang, Q., Chowdhury, M., and Zhang, M. Efficient large language models: A survey. Transactions on Machine Learning Research, 2024 b . ISSN 2835-8856. URL https://openreview.net/forum?id=bsCCJH...

    2024

  21. [29]

    M., Yang, L., Wen, Y., and Zhang, W

    Wang, J., Fang, M., Wan, Z., Wen, M., Zhu, J., Liu, A., Gong, Z., Song, Y., Chen, L., Ni, L. M., Yang, L., Wen, Y., and Zhang, W. Openr: An open source framework for advanced reasoning with large language models. CoRR, abs/2410.09671, 2024 a . doi:10.48550/ARXIV.2410.09671. UR...

    doi arXiv

  22. [30]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations

    Wang, P., Li, L., Shao, Z., Xu, R., Dai, D., Li, Y., Chen, D., Wu, Y., and Sui, Z. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Ku, L., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Co...

    2024 doi

  23. [31]

    SEED: accelerating reasoning tree construction via scheduled speculative decoding

    Wang, Z., Wu, J., Lai, Y., Zhang, C., and Zhou, D. SEED: accelerating reasoning tree construction via scheduled speculative decoding. CoRR, abs/2406.18200, 2024 c . doi:10.48550/ARXIV.2406.18200. URL https://doi.org/10.48550/arXiv.2406.18200

    2024 doi arXiv

  24. [32]

    H., Le, Q

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural I...

    2022

  25. [33]

    Cantelli's inequality --- Wikipedia , the free encyclopedia, 2024

    Wikipedia contributors . Cantelli's inequality --- Wikipedia , the free encyclopedia, 2024. URL https://en.wikipedia.org/w/index.php?title=Cantelli [Online; accessed 21-January-2025]

    2024

  26. [34]

    Scaling inference computation: Compute-optimal inference for problem-solving with language models

    Wu, Y., Sun, Z., Li, S., Welleck, S., and Yang, Y. Scaling inference computation: Compute-optimal inference for problem-solving with language models. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24, 2024. URL https://openreview.net/forum?id=j7DZWSc8qu

    2024

  27. [35]

    Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding

    Xia, H., Yang, Z., Dong, Q., Wang, P., Li, Y., Ge, T., Liu, T., Li, W., and Sui, Z. Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding. In Ku, L., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computat...

    2024 doi

  28. [36]

    Decomposition enhances reasoning via self-evaluation guided decoding

    Xie, Y., Kawaguchi, K., Zhao, Y., Zhao, X., Kan, M., He, J., and Xie, Q. Decomposition enhances reasoning via self-evaluation guided decoding. CoRR, abs/2305.00633, 2023. doi:10.48550/ARXIV.2305.00633. URL https://doi.org/10.48550/arXiv.2305.00633

    2023 doi arXiv

  29. [37]

    Multi-candidate speculative decoding

    Yang, S., Huang, S., Dai, X., and Chen, J. Multi-candidate speculative decoding. CoRR, abs/2401.06706, 2024. doi:10.48550/ARXIV.2401.06706. URL https://doi.org/10.48550/arXiv.2401.06706

    2024 doi arXiv

  30. [38]

    Tree of thoughts: Deliberate problem solving with large language models

    Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Proce...

    2023

  31. [39]

    Beyond the speculative game: A survey of speculative execution in large language models

    Zhang, C., Liu, Z., and Song, D. Beyond the speculative game: A survey of speculative execution in large language models. CoRR, abs/2404.14897, 2024 a . doi:10.48550/ARXIV.2404.14897. URL https://doi.org/10.48550/arXiv.2404.14897

    2024 doi arXiv

  32. [40]

    Rest-mcts*: LLM self-training via process reward guided tree search

    Zhang, D., Zhoubian, S., Yue, Y., Dong, Y., and Tang, J. Rest-mcts*: LLM self-training via process reward guided tree search. CoRR, abs/2406.03816, 2024 b . doi:10.48550/ARXIV.2406.03816. URL https://doi.org/10.48550/arXiv.2406.03816

    2024 doi arXiv

  33. [41]

    Gaussian distribution., 2010

    Zhang, X. Gaussian distribution., 2010

    2010

  34. [42]

    and Bharadwaj, M

    Zhong, W. and Bharadwaj, M. S3D: A simple and cost-effective self-speculative decoding scheme for low-memory gpus. CoRR, abs/2405.20314, 2024. doi:10.48550/ARXIV.2405.20314. URL https://doi.org/10.48550/arXiv.2405.20314

    2024 doi arXiv

  35. [43]

    A survey on efficient inference for large language models

    Zhou, Z., Ning, X., Hong, K., Fu, T., Xu, J., Li, S., Lou, Y., Wang, L., Yuan, Z., Li, X., Yan, S., Dai, G., Zhang, X., Dong, Y., and Wang, Y. A survey on efficient inference for large language models. CoRR, abs/2404.14294, 2024. doi:10.48550/ARXIV.2404.14294. URL https://doi....

    2024 doi arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.