Pith. sign in

REVIEW 5 major objections 5 minor 3 cited by

Tracking just the top two answer counts is enough for asymptotically optimal Bayesian stopping in LLM self-consistency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:15 UTC pith:VM6WURLJ

load-bearing objection Strong asymptotic theory for L=3 aggregation; the 'up to 50% savings' claim is not supported by the experiments. the 5 major comments →

arxiv 2602.05395 v2 pith:VM6WURLJ submitted 2026-02-05 stat.ML cs.AIcs.LG

Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answers

Jingkai Huang , Will Ma , Zhengyuan Zhou This is my paper
classification stat.ML cs.AIcs.LG MSC 60G4062L1062F15
keywords self-consistencyadaptive stoppingBayesian posterior approximationsequential hypothesis testingmode identificationLLM inference costmajority votingtest-time compute
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks how many LLM answers a system must sample before a majority vote can be trusted, and shows that a deliberately coarse Bayesian posterior—tracking only the most frequent answer, the second most frequent, and everyone else combined—is asymptotically as sample-efficient as the exact posterior when the answer distribution is known. Because the exact posterior costs factorial time in the number of distinct answers, this makes optimal stopping practical; the same L=3 rule still beats prior-free adaptive stopping when the true distribution is only known up to a candidate set. The authors prove exact asymptotic rates for expected stopping time, and demonstrate on cached real LLM responses that the rule cuts the number of generations by up to 50% with little or no loss in answer accuracy. The practical claim is that consistency-based LLM inference can be made both calibrated and cheap.

Core claim

For a query whose answers are IID draws from a fixed multinomial vector π with top probabilities p1>p2, the paper defines the optimal Bayesian stopping time n* as the first time the posterior probability of the current mode exceeds 1−δ, and an L-aggregated version n*_L that conditions only on the counts of the L−1 most frequent answers plus the lumped rest. Theorem 3.1 gives lim_{δ→0} E[n*_L]/log(1/δ) = 1/((p1−p2)·log(p1/p2)) for every L=3,...,K, identical to the exact posterior, while L=2 gives 1/D_KL(p1||p2), which is larger. Since the prior-free mode-identification rate is strictly larger than both, the paper concludes that Bayesian prior information strictly improves adaptive stopping, a

What carries the argument

The central object is the L-aggregated posterior P(H1 | C^L_n), formed by compressing the count-of-counts observation to the top L−1 observed multiplicities. It is an unbiased coarsening: the aggregated posterior equals the conditional expectation of the exact posterior given the compressed counts, so beliefs stay correct on average while reducing the computational state. The proof of asymptotic optimality works by bounding the L-aggregated stopping time between two thresholds on a log-likelihood ratio between the top two candidate labels, and showing the per-period drift converges to (p1−p2)·log(p1/p2) for any L≥3; dynamic programming over a truncated generating function computes the needed

Load-bearing premise

The load-bearing premise is that the LLM's answers are IID draws from a fixed multinomial distribution with a known gap between the top two probabilities; in live deployment the distribution is unknown and can drift, and the theory does not cover that mismatch.

What would settle it

Take a live or cached LLM whose answer distribution changes over time (for example, easy questions followed by hard ones) or whose true prior is deliberately left out of the candidate set, and compare L=3's stopping count against the exact posterior rule: if the ratio E[n*_3]/log(1/δ) no longer matches the predicted 1/((p1−p2)log(p1/p2)) or if L=3's sample count fails to beat the prior-free baseline, the core guarantee is falsified in that regime.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • With a known answer distribution, the L=3 rule matches the exact posterior's expected stopping time as δ→0, so no asymptotic sample efficiency is lost by discarding all but the top two counts.
  • Bayesian adaptive stopping strictly improves on prior-free adaptive self-consistency asymptotically, meaning prior information has provable statistical value for mode identification.
  • Under an uncertain prior drawn from a candidate set, any L≥3 still dominates the prior-free baseline; only L=2 can struggle, and only in adversarial candidate sets.
  • Because posterior computation cost scales exponentially in L rather than in K, the asymptotically optimal rule remains cheap enough for real-time inference.
  • In empirical replay on cached responses, the rule keeps answer accuracy essentially unchanged while reducing the number of LLM generations by up to 50%.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: A natural stress test is to drop the IID and fixed-support assumptions; if the answer distribution drifts across queries, the fixed multinomial prior becomes misspecified, and an online version that updates the prior from recent queries would directly test whether the 50% savings survive deployment.
  • Inference: The empirics hint that early stopping can return the runner-up answer more often, which is sometimes the correct answer; this suggests a variant that stops when the top-two combined posterior is high rather than the mode alone, trading mode-calibration for answer accuracy.
  • Inference: The theorem identifies a minimal sufficient statistic for optimal asymptotic stopping—the top-two counts—which may transfer to other sequential problems with a large label space, such as Bayesian best-arm identification.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes an adaptive Bayesian stopping rule for self-consistency in LLM decoding. Under a multinomial model for IID answer draws, it defines a posterior-threshold stopping time n* that stops when P(H1 | C_n) >= 1-delta, and then introduces an L-aggregated posterior computed from only the L-1 most frequent answer counts plus an aggregated 'other' category. The main theoretical result (Theorem 3.1) states that for L=3,...,K the asymptotic expected stopping time satisfies lim E[n*_{L}]/log(1/delta) = 1/((p1-p2) log(p1/p2)), matching the exact posterior rate, while L=2 has the slower rate 1/D_KL(p1||p2); Remark 4 concludes that prior-free adaptive stopping is strictly worse as delta->0. Theorem 4.1 extends the characterization to a finite candidate set of priors with a hyper-prior. Experiments use synthetic multinomial data and replay of cached responses from FEval-TTC, comparing L=2,3,4 against exact posterior and a prior-free ASC baseline, and report up to roughly 50% reduction in generated samples.

Significance. If Theorem 3.1 is correct and fully proved, the result is genuinely interesting: it gives a precise asymptotic rate for a coarse L=3 posterior that matches the exact posterior, and a parameter-free comparison with prior-free mode identification. The theoretical development is not built on fitted outputs; the rates are derived from the stated multinomial model. The paper also ships algorithmic details (dynamic programming for S_psi and S_tilde_psi) and code-adjacent pseudocode. However, the proof as written has significant gaps, and the empirical section does not validate the abstract's unqualified claim about reducing LLM calls on real data. The practical claim of 'up to 50% savings' rests on synthetic trajectories sampled from the very empirical distribution that the algorithm is given as the known prior, which makes the known-prior experiments in-sample and self-referential. The uncertain-prior experiments similarly do not ensure the assumption pi in Pi_M is satisfied. The core theoretical idea is promising and likely salvageable, but the current manuscript overstates both its proof completeness and its empirical support.

major comments (5)
  1. [Appendix A, Proof of Theorem 3.1, first sentence] The proof begins by assuming M(n)=K (and L(n)=L-1) 'for simplicity'. In the actual process M(n) is random and smaller than K until all K labels have appeared; the posterior sums in Section 2 and the aggregated posterior in Section 3 explicitly run over the observed M(n) distinct answers. The proof's log-likelihood ratios and the sandwich bounds A1/(2(K-1)A2) etc. presuppose that all K labels are present at all times. No argument is given that the period before the first appearance of all K labels is asymptotically negligible for E[n*_{L}]/log(1/delta). This is a load-bearing gap in the proof of Theorem 3.1, which is the central claim of the paper.
  2. [Appendix A, L=2 case, 'we omit the detailed analysis here'] After bounding P^{(−1)}(max_j r_j >= n1) by (K−1) exp(−c0 n), the proof says 'We omit the detailed analysis here and refer readers to the analysis of (11), which is exactly the same.' The analysis of (11) is itself only sketched: it invokes Chernoff bounds and a KL lower bound but does not explicitly verify that the required constants are independent of n or that the multiplicative factor (K−1) does not affect the final delta->0 limit. Similarly, the proof of Theorem 4.1 says 'We omit the similar analysis routine' for the L>=3 case. These are not merely cosmetic omissions; they are necessary to justify the claimed exact asymptotic constants.
  3. [Title, Section 2.1, Section 3.3] The manuscript calls n* and n*_L 'optimal Bayesian stopping rules', but no theorem establishes that the posterior-threshold rule minimizes expected stopping time among all rules with error probability at most delta. The results characterize the asymptotic expected stopping time of one specific rule. The word 'optimal' in the title and abstract is therefore unsupported unless a formal optimality theorem is supplied (e.g., a Bayes-optimality or asymptotic minimax statement). This is a conceptual overclaim that should be corrected or proved.
  4. [Section 5.3 and Abstract] The abstract claims the method 'can achieve similar answer accuracy while cutting the number of LLM calls ... by up to 50%.' The experiments do not measure LLM calls. For each question, the empirical distribution pi is computed from the 40 cached responses, and synthetic trajectories are generated by subsampling from this same pi. In the known-prior setting, the algorithm is handed exactly this pi, so the evaluation is circular: the same data define both the prior and the test distribution. The savings are therefore a property of the fitted multinomial replay, not of live LLM sampling. The authors should either run genuine (or held-out cached) LLM calls without using the test query's pi as the prior, or substantially qualify the empirical claim to 'simulated trajectories under a fitted multinomial model.'
  5. [Section 4.1 and Section 5.3, uncertain prior] Theorem 4.1 assumes the true prior pi belongs to the candidate set Pi_M and that the hyper-prior lambda_m is known. In the uncertain-prior experiments, the candidate set is built from the empirical distributions of a 70% training split, and the algorithm is tested on 30% test questions. Nothing guarantees that a test question's true answer distribution belongs to Pi_M; indeed, with a finite set of cached responses, it almost certainly does not coincide exactly with any training empirical distribution. The experimental section does not report coverage or membership rates, so Table 3 and Appendix D do not test the assumptions of Theorem 4.1. The comparison to ASC may still be informative, but it should be framed as a heuristic evaluation outside the theorem's assumptions, or the authors should design the split to preserve the membership condition.
minor comments (5)
  1. [Section 3.2, complexity expression] The notation 'O(K L)' is ambiguous; it should be O(K^L) (or O(K^L ar n^2_{L(n)})) to match the later expression O(K! ar n^2/(K-L)!). As written, O(KL) looks like O(K times L), which would contradict the stated factorial reduction.
  2. [Table 3 and text] The model name is written 'LaMA-3.1-405B' in Table 3 (and once in the text) instead of 'LLaMA-3.1-405B'. Also, the note 'repeated for 5 times in the testing set' is unclear: report standard errors or confidence intervals for the reported mean Num. Gen. and accuracies.
  3. [Section 4.1, null answers] The augmentation of the support with 'null' answers having probability 0 conflicts with the standing assumption in Section 2.1 that p_k > 0 for all k. This also affects identifiability of K. Please clarify whether probabilities are allowed to be zero and how the algorithms handle zero-probability labels.
  4. [Remark 4 and Remark 6] The inequalities E[n*_f] > E[n*_2] > E[n*_3] are stated only asymptotically as delta->0, but the text sometimes reads as if they hold at every delta. Add a sentence making the asymptotic qualification explicit in both remarks.
  5. [Appendix D, cutoff effect] The statement that 'Mode Acc. does not reach the threshold due to cutoff effect' is reasonable, but note that the same cutoff effect also truncates Num. Gen. values, making some reported stopping times at 1-delta=0.95 (e.g., Num. Gen.=1.00 in Table 3) uninformative about the expected stopping time of the algorithm. A more careful treatment of censored stopping times would strengthen the empirical analysis.

Circularity Check

1 steps flagged

Theorems are genuine derivations; the circularity is in the empirical component, where the ground-truth π is fit from the same cached answers used to generate the synthetic trajectories, making the reported LLM-call savings an in-sample simulation of the fitted prior.

specific steps
  1. fitted input called prediction [Section 5.3, 'Known Prior Construction' and the following paragraph]
    "For every question and every LLM, we utilize the raw 40 generations provided in the dataset to compute an empirical answer-frequency distribution. We treat this empirical distribution as the ground-truth distribution π for the specific query. In the 'known prior' setting, the algorithm has full access to this specific π. ... we generate new, synthetic generation trajectories of length 100 by subsampling (with replacement) from π."

    The distribution π is fit from the 40 cached answers, then declared the ground-truth distribution, handed to the algorithm as the known prior, and used to sample the evaluation trajectories. Hence the known-prior Num. Gen. numbers are produced under a prior that coincides with the data-generating distribution by construction. Comparing these numbers with ASC measures the value of being given the true distribution, not the value of estimating it from cached responses; the abstract's 'up to 50%' LLM-call savings on real inference is an in-sample simulation of the fitted prior rather than a prediction about real LLM calls.

full rationale

The core theoretical claims are not circular. Theorem 3.1 and Theorem 4.1 are derived from the stated IID multinomial and prior-set assumptions using explicit sandwich bounds and Sanov/Chernoff arguments (Appendix A); no fitted parameter is renamed as a prediction and no load-bearing assertion rests on a self-citation. The prior-free comparison rates are quoted from external work (Jain et al. 2022; Shah et al. 2020). The L=3 'asymptotically optimal' statement therefore has independent mathematical content. The circularity is confined to the real-world empirical demonstration. In Section 5.3, the paper computes an empirical answer-frequency distribution from the 40 cached responses, calls it the ground-truth π, gives it to the algorithm in the known-prior setting, and then evaluates by subsampling synthetic trajectories from that same π. Thus the reported savings are an in-sample fit of the prior to the data-generating distribution, not evidence about unknown, nonstationary LLM answer distributions. The uncertain-prior experiments also rely on the assumption π∈Π_M without verifying that test questions satisfy it. These issues affect the abstract's unqualified empirical savings claim while leaving the theory intact; hence a partial-circularity score.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central theory is parameter-free in the sense that asymptotic rates are derived from stated priors, but the practical claims rest on domain assumptions: IID multinomial sampling, known support/candidate set, and the equivalence between distribution mode and correct answer. The experiments add a data-fitted prior and a replay construction.

free parameters (5)
  • Known prior π (empirical ground truth) = per-query empirical answer frequencies over 40 cached responses
    Section 5.3: 'We treat this empirical distribution as the ground-truth distribution π.' The known-prior results and the claimed 50% savings depend on this fitted π.
  • Candidate prior set ΠM = empirical answer distributions from a 70% training split; size M varies
    Section 5.3: uncertain-prior tests use this data-derived set; Theorem 4.1 requires the true prior to be in ΠM, so the guarantee is conditioned on a set that is not controlled.
  • Hyper-prior λm = 1/M for all m
    Section 5.3 sets a uniform λ; the posterior mixture and the m† minimizers in Theorem 4.1 depend on these weights, which are not learned from data.
  • Support size K = maximum number of distinct answers observed; null answers padded to K
    Complexity is O(K^L), and the exact posterior (2) requires K; in live open-ended generation K is not known and can be large.
  • Aggregation parameter L = 3
    Chosen because of Theorem 3.1; it is a hand-selected design parameter, not estimated, and the 'all you need' claim is tied to this value.
axioms (5)
  • domain assumption LLM answers are IID draws from a fixed multinomial vector π with p1 > p2 ≥ ... ≥ pK > 0.
    Section 2.1: 'drawn IID following π'. All posterior updates and stopping-time rates depend on this stationarity.
  • domain assumption K and π (or a candidate set ΠM containing π) are known to the algorithm.
    Exact posterior (2) and Theorem 4.1 require K and π∈ΠM; Section 5.3 obtains them from cached data rather than live inference.
  • domain assumption The mode of the LLM's answer distribution is the answer whose identification is desired.
    Used throughout as the objective; Section 5.3 notes the runner-up is often the correct answer, which weakens this assumption.
  • ad hoc to paper Cached empirical response frequencies are faithful proxies for the true LLM distribution and can be replayed as synthetic generations.
    Section 5.3 constructs π, ΠM and 100-length trajectories by subsampling from the 40 cached responses; no live LLM calls are made.
  • standard math Standard large-deviation bounds (Hoeffding, Chernoff, Sanov) and the prior-free mode-identification rate (7) from Jain et al. (2022) are correct.
    Invoked in Appendix A/B to bound deviation terms and to compare with the prior-free baseline.
invented entities (2)
  • L-aggregated observation state C^L_n no independent evidence
    purpose: Compresses the full count-of-counts into the top L-1 counts plus an aggregated bucket, reducing posterior computation from O(K!) to O(K^L).
    A mathematical construct; its value is justified only by the paper's own theorems and simulations, with no external falsifiable handle.
  • Null answers with probability 0 no independent evidence
    purpose: Pad smaller-support priors to a common K-dimensional simplex in the uncertain-prior formulation.
    Section 4.1 introduces null answers purely as a representational device; they carry no empirical content.

pith-pipeline@v1.3.0-alltime-deepseek · 33704 in / 20825 out tokens · 202960 ms · 2026-08-03T04:15:20.327031+00:00 · methodology

0 comments
read the original abstract

A simple strategy for improving LLM accuracy, especially in math and reasoning problems, is to sample multiple responses and submit the answer most consistently reached. In this paper we leverage Bayesian prior information to save on sampling costs, stopping once sufficient consistency is reached. Although the exact posterior is computationally intractable, we further introduce an efficient "L-aggregated" stopping policy that tracks only the L-1 most frequent answer counts. Theoretically, we prove that L=3 is all you need: this coarse approximation is sufficient to achieve asymptotic optimality, and strictly dominates prior-free baselines, while having a fast posterior computation. Empirically, this identifies the most consistent (i.e., mode) LLM answer using fewer samples, and can achieve similar answer accuracy while cutting the number of LLM calls (i.e., saving on LLM inference costs) by up to 50%.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MARS: Margin-Adversarial Risk-controlled Stopping for Parallel LLM Test-time Scaling

    cs.AI 2026-06 unverdicted novelty 7.0

    MARS is a margin-adversarial stopping rule for parallel LLM test-time scaling that saves 25-47% tokens while matching full-budget majority-vote accuracy by learning trace switch probabilities and applying adversarial bounds.

  2. Online Pandora's Box for Contextual LLM Cascading

    cs.AI 2026-06 unverdicted novelty 7.0

    Introduces a parametric reservation-index policy with GMM estimation and UCB exploration for contextual LLM cascading under output-mediated feedback, claiming dimension-dependent square-root regret.

  3. ATLAS: Agentic Test-time Learning-to-Allocate Scaling

    cs.LG 2026-06 unverdicted novelty 7.0

    ATLAS introduces an LLM-orchestrated agentic framework for dynamic test-time scaling via extensible 'explore' actions, achieving higher accuracy with fewer API calls than fixed-workflow baselines on four benchmarks.

Reference graph

Works this paper leans on

23 extracted references · 7 linked inside Pith · cited by 3 Pith papers

  1. [1]

    Let's sample step by step: Adaptive-consistency for efficient reasoning and coding with llms

    Pranjal Aggarwal, Aman Madaan, Yiming Yang, et al. Let's sample step by step: Adaptive-consistency for efficient reasoning and coding with llms. arXiv preprint arXiv:2305.11860, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Cges: Confidence-guided early stopping for efficient and accurate self-consistency

    Ehsan Aghazadeh, Ahmad Ghasemi, Hedyeh Beyhaghi, and Hossein Pishro-Nik. Cges: Confidence-guided early stopping for efficient and accurate self-consistency. arXiv preprint arXiv:2511.02603, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Optimizing llm inference: Fluid-guided online scheduling with memory constraints

    Ruicheng Ao, Gan Luo, David Simchi-Levi, and Xinshang Wang. Optimizing llm inference: Fluid-guided online scheduling with memory constraints. arXiv preprint arXiv:2504.11320, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  5. [5]

    Are more llm calls all you need? towards the scaling properties of compound ai systems

    Lingjiao Chen, Jared Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. Are more llm calls all you need? towards the scaling properties of compound ai systems. Advances in Neural Information Processing Systems, 37: 0 45767--45790, 2024

    2024

  6. [6]

    Optimal self-consistency for efficient reasoning with large language models

    Austin Feng, Marius Alonso, and Ambroise Odonnat. Optimal self-consistency for efficient reasoning with large language models. arXiv preprint arXiv:2511.12309, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Bayesian data analysis

    Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995

    1995

  8. [8]

    Sample complexity and representation ability of test-time scaling paradigms

    Baihe Huang, Shanda Li, Tianhao Wu, Yiming Yang, Ameet Talwalkar, Kannan Ramchandran, Michael I Jordan, and Jiantao Jiao. Sample complexity and representation ability of test-time scaling paradigms. arXiv preprint arXiv:2506.05295, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Online scheduling for llm inference with kv cache constraints

    Patrick Jaillet, Jiashuo Jiang, Konstantina Mellou, Marco Molinaro, Chara Podimata, and Zijie Zhou. Online scheduling for llm inference with kv cache constraints. arXiv preprint arXiv:2502.07115, 2025

    arXiv 2025

  10. [10]

    Pac mode estimation using ppr martingale confidence sequences

    Shubham Anand Jain, Rohan Shah, Sanit Gupta, Denil Mehta, Inderjeet J Nair, Jian Vora, Sushil Khyalia, Sourav Das, Vinay J Ribeiro, and Shivaram Kalyanakrishnan. Pac mode estimation using ppr martingale confidence sequences. In International Conference on Artificial Intelligence and Statistics, pages 5815--5852. PMLR, 2022

    2022

  11. [11]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35: 0 22199--22213, 2022

    2022

  12. [12]

    Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning

    Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang, and Kan Li. Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. arXiv preprint arXiv:2401.10480, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Calibrating large language models with sample consistency

    Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar, Niket Tandon, Marianna Apidianaki, Mrinmaya Sachan, and Chris Callison-Burch. Calibrating large language models with sample consistency. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 19260--19268, 2025

    2025

  14. [14]

    Feval-ttc: Fair evaluation protocol for test-time compute

    Pavel Rumiantsev, Soumyasundar Pal, Yingxue Zhang, and Mark Coates. Feval-ttc: Fair evaluation protocol for test-time compute. arXiv preprint arXiv:2511.01203, 2025

    arXiv 2025

  15. [15]

    Sequential mode estimation with oracle queries

    Dhruti Shah, Tuhinangshu Choudhury, Nikhil Karamchandani, and Aditya Gopalan. Sequential mode estimation with oracle queries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5644--5651, 2020

    2020

  16. [16]

    Sequential analysis: tests and confidence intervals

    David Siegmund. Sequential analysis: tests and confidence intervals. Springer Science & Business Media, 2013

    2013

  17. [17]

    Sequential tests of statistical hypotheses

    Abraham Wald. Sequential tests of statistical hypotheses. In Breakthroughs in statistics: Foundations and basic theory, pages 256--298. Springer, 1992

    1992

  18. [18]

    Reasoning aware self-consistency: Leveraging reasoning paths for efficient llm sampling

    Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Reasoning aware self-consistency: Leveraging reasoning paths for efficient llm sampling. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3613--3635, 2025 a

    2025

  19. [19]

    Beacon: Bayesian optimal stopping for efficient llm sampling

    Guangya Wan, Zixin Stephen Xu, Sasa Zorc, Manel Baucells, Mengxuan Hu, Hao Wang, and Sheng Li. Beacon: Bayesian optimal stopping for efficient llm sampling. arXiv preprint arXiv:2510.15945, 2025 b

    arXiv 2025

  20. [20]

    Make every penny count: Difficulty-adaptive self-consistency for cost-efficient reasoning

    Xinglin Wang, Shaoxiong Feng, Yiwei Li, Peiwen Yuan, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, and Kan Li. Make every penny count: Difficulty-adaptive self-consistency for cost-efficient reasoning. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 6904--6917, 2025

    2025

  21. [21]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  23. [23]

    Distributed load balancing with workload-dependent service rates

    Wenxin Zhang, Santiago R Balseiro, Robert Kleinberg, Vahab Mirrokni, Balasubramanian Sivan, and Bartek Wydrowski. Distributed load balancing with workload-dependent service rates. In Proceedings of the 26th ACM Conference on Economics and Computation, pages 917--917, 2025

    2025