Pith. sign in

REVIEW 4 major objections 4 minor 35 references

An interpretable fuzzy controller can allocate per-prompt sampling budgets, preserving most of full-budget accuracy while using fewer samples on average.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A fuzzy controller that allocates a per-prompt sampling budget keeps LLM accuracy near a fixed full-budget baseline while reducing the average number of candidate answers on some datasets.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful idea and a clean evaluation design, but the central accuracy-compute tradeoff is undercut by an internal inconsistency between the main table and the ablation table. the 4 major comments →

arxiv 2608.03961 v1 pith:RNLNPSG3 submitted 2026-08-04 cs.AI

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Mobina Kashaniyan , Ali Jannesari This is my paper
classification cs.AI
keywords test-time scalingadaptive samplingfuzzy controllerinterpretabilityself-certaintyBorda aggregationLLM reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that an interpretable fuzzy controller can allocate a per-prompt sampling budget during LLM test-time scaling, so that easier or more confident prompts receive fewer candidate answers and harder prompts receive more. The controller uses cheap, human-readable signals—estimated complexity, confidence, entropy, prompt type, expected answer length, and a coarse history—and maps them to an integer number of samples N(x). In the main fair-alignment comparison, where decoding settings and the answer selector are held fixed, it uses 1.4 to 14.5 percent fewer samples than a fixed N=8 baseline while staying within 0.018 accuracy of it, and in one run it is slightly more accurate. The point is not that adaptive sampling beats full-budget sampling outright; it is that per-prompt compute can be made auditable and still preserve most of the accuracy. That matters because test-time scaling's compute decisions are otherwise fixed or opaque.

Core claim

The paper's central claim is that a rule-based, non-learned controller can make test-time scaling's budget decisions interpretable without sacrificing most of the accuracy of a full budget. Concretely, it reports that on MATH and GSM8K the adaptive controller, paired with a fixed self-certainty+Borda selector, matches a selector-matched fixed-N=8 control to within −0.018 to +0.005 accuracy while using 1.4 to 14.5 percent fewer samples; on SciQ it improves over Best-of-1 and self-certainty-only baselines. The authors present this as an accuracy–compute tradeoff, not an accuracy win: the controller's value is in spending less on prompts that can tolerate it and keeping more on prompts that can

What carries the argument

The central object is the hierarchical fuzzy controller: a two-stage rule base with triangular and trapezoidal membership functions over normalized signals. The first stage combines estimated complexity c and confidence γ to produce a coarse scale; the second refines it with entropy, expected answer length, prompt type, history, and three NLP sub-scores, represented as interval-valued adjustments. Defuzzification gives a scale s, an uncertainty nudge adds 0.2u, and the budget mapping N(x) = max(1, round(1 + s′(Nmax − 1))) converts the scale to an integer sample count. This mapping is what carries the argument: the same rule base both decides compute and explains the decision.

Load-bearing premise

The load-bearing premise is that the model-confidence and entropy signals are available to the controller in the exact runs that produce the headline results, but the fair-alignment protocol disables the draft pass that would produce them; if those signals are absent, the controller reduces to prompt-side features and the claimed confidence-aware mechanism is not what was evaluated.

What would settle it

Run the fair-alignment protocol while logging controller inputs. If the logged confidence γ and entropy η are constant, undefined, or have zero correlation with the selected budgets N(x), or if forcing γ to a fixed value reproduces the same budget allocations and the same accuracy–compute points, then the central claim that model confidence drives allocation is falsified. A second observation: replace the fuzzy rule base with a trivial length-only rule (for example, N from word count); if it matches the reported tradeoff, the interpretable signals beyond length do no work.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • On MATH, Phi-3-mini and Qwen2.5-1.5B retain essentially the same accuracy as fixed N=8 while using 10.8 and 14.5 percent fewer samples respectively; a deployment can convert that into a compute saving.
  • On GSM8K, the controller stays close to the full budget, showing that a conservative adaptive policy can avoid wasting accuracy; the controller does not force reductions.
  • Budget allocation and answer selection are separable: the oracle audits show candidate pools containing correct answers that the selector misses, so improving the selector is a complementary path to accuracy.
  • Fixed small budgets (N=1, 2, 3, 5) underperform, so any adaptive method must preserve enough candidate diversity for aggregation to work.
  • Because the controller's rules are fixed and inspectable, a user can audit why any prompt received N samples—something fixed budgets or learned policies do not offer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the main fair-alignment protocol disables the draft pass, the model-side confidence and entropy signals may not exist in those runs; the paper does not state how they are obtained, so the evaluated controller may in practice be a prompt-complexity allocator, and the confidence-aware mechanism claimed in the design is untested in the headline numbers.
  • Editorial inference: the same continuous-scale architecture could assign token budgets, temperature, or early-stopping thresholds per prompt, not just sample counts.
  • Editorial inference: the dataset-dependent savings (large on MATH, small on GSM8K) suggest the expected savings of such a controller could be predicted from a prompt-difficulty distribution, which would tell a deployer in advance whether adaptive control is worthwhile.
  • Editorial inference: a direct test of the mechanism would be to log the actual γ and η values in fair-alignment mode; if they are constant or unused, or if zeroing them changes no budget allocations, the paper's interpretable-signal story would need to be revised to a prompt-features-only story.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes an interpretable adaptive test-time sampling method for LLMs. A hierarchical fuzzy controller maps interpretable signals (estimated prompt complexity, model confidence, entropy, prompt type, expected answer length, and a history cache) to a per-prompt sample budget N(x). The controller is evaluated under a 'fair-alignment' protocol that fixes decoding settings and answer selection, with the claim that it preserves most of the accuracy of fixed N=8 sampling while reducing the average number of samples. Results are reported on GSM8K, a MATH subset, and SciQ using Phi-3-mini-4k-instruct and Qwen2.5-1.5B-Instruct. The paper's strongest evidence is the selector-matched comparison in Table 1, supported by appendix tables, ablations, and illustrative traces.

Significance. If the claims were reliable, the paper would make a useful contribution: a transparent, hand-designed controller that allocates test-time compute with a favorable accuracy-compute tradeoff would be a practical alternative to fixed budgets and black-box policies. The paper also has methodological strengths: it explicitly controls the answer selector, reports Wilson intervals, separates candidate-pool quality from selection quality via oracle audits, and is candid about the hand-designed nature of the controller. However, the central empirical claim is currently undermined by an unresolved internal inconsistency between core tables, an ambiguity about whether the model-side signals used by the controller were actually available in the main experiments, and the absence of paired significance testing. These issues are load-bearing, so the significance of the contribution cannot be assessed from the manuscript as written.

major comments (4)
  1. [Section 5.1, Table 1 vs Section 6, Table 3] For Phi-3-mini on GSM8K, Table 1 reports fixed N=8 accuracy as 0.717 with 95% Wilson CI [0.692, 0.741], while Table 3 reports fixed adaptive N=8 accuracy as 0.809. The adaptive rows are also inconsistent: 0.715 in Table 1 versus 0.810 in Table 3. Table 7 further reports fixed self-certainty+Borda at N=8 as 0.7172, corroborating Table 1 but not Table 3. The gap of about 0.09 is far outside the stated CI and cannot be explained by selector choice or rounding. The two tables cannot both describe the same fair-alignment protocol. Since the paper's headline tradeoff is based on Table 1, this inconsistency directly undermines the reliability of the central claim.
  2. [Section 4 vs Section 3.1, Eq. (4)] The fair-alignment protocol states 'the draft pass is disabled', but the controller's inputs gamma (confidence) and eta (entropy) are defined in Eq. (4) as being computed from a short draft answer. The paper does not specify how these model-side signals are obtained when the draft pass is disabled. If they are unavailable, the evaluated controller relies only on prompt-side features, and the mechanism described in Section 3.2 (e.g., high entropy adding compute) was not actually tested. If they are available through some other means, that procedure is not described. This needs to be resolved explicitly.
  3. [Section 5.1, Table 1] The accuracy deltas in the selector-matched comparison are small (-0.018 to +0.005) and the 95% Wilson intervals overlap. The table note correctly states that paired tests require matched per-prompt logs, but no such tests are reported. Without a paired McNemar or bootstrap test on per-prompt correctness, the differences between Adaptive and fixed N=8 are not statistically distinguished from noise. The sample-reduction claim also needs a confidence interval or significance statement. This is essential because the paper's claim is an accuracy-compute tradeoff, not just a point estimate.
  4. [Section 3.2 / Appendix C, Eqs. (9), (12)-(14), (23), (25), (27)] The controller has many hand-set parameters: membership-function boundaries, second-stage refinement weights, the uncertainty nudge alpha, the hard-prompt floor, and min-max clipping ranges estimated on a warm-up set. The paper is transparent that these are not learned, but it does not provide held-out validation of the controller design or a systematic sensitivity analysis over these parameters. Because the same controller is evaluated on the tasks and datasets that motivated the design, the reported tradeoff may reflect in-sample fitting. At minimum, the authors should show that the qualitative conclusion is stable under plausible perturbations of the membership boundaries and rule weights.
minor comments (4)
  1. [Abstract] The abstract contains a formatting artifact: 'adaptive} test-time scaling' with an unmatched brace. This should be fixed.
  2. [Appendix I.1] The illustrative trace assigns N_i=12 to a prompt, while the main fair-alignment experiments use Nmax=8. This is confusing; the example should either use Nmax=8 or explicitly state that it is illustrative of the full system with a larger cap.
  3. [Appendix G, Table 6] The text says Table 6 'excludes the selector-matched fixed-N=8 control', but the table includes a fixed N=8 row for Self-Certainty only. This wording is misleading; please clarify that the excluded row is the one with the same self-certainty+Borda selector as Adaptive.
  4. [Section 6, Table 3] The table is titled 'Fixed-budget controls on GSM8K and MATH with Phi-3-mini.' Given the inconsistency with Table 1, the caption should state the exact protocol (including whether the draft pass was enabled) and explain why the fixed N=8 accuracy differs from Table 1.

Circularity Check

0 steps flagged

No significant circularity: the fuzzy controller's budget assignment is a fixed hand-designed mapping, and the accuracy–compute tradeoff is an empirical measurement, not an input-equivalent prediction.

full rationale

The paper makes no claim to derive the controller from first principles and no self-citations are load-bearing. The mapping from signals to budget (Eq. 7, Eq. 23-27) is a deterministic hand-written policy; sample savings follow by construction from assigning N(x)<Nmax to some prompts, but the central claim—that accuracy remains close to fixed-N=8—is measured on data and could have failed. The controller's rules/weights are not fitted to the reported accuracies in any reduction exhibited by the paper; Section 7 explicitly labels them hand-designed and non-optimal, which is a limitation, not a circular step. Three flagged concerns do not change the circularity verdict: (1) Section 4 disables the draft pass while Section 3.1 defines gamma/eta from a draft, leaving unspecified how model-side signals are obtained in main experiments—a validity gap, not circularity; (2) Table 1 and Table 3 give incompatible Phi-3-mini/GSM8K fixed-N=8 accuracies (0.717 vs 0.809, with Table 7 at 0.7172), which undermines empirical reliability but is not a definitional reduction; (3) the paper openly notes the controller is not optimized, and no held-out validation of rule design is described, but absence of a tuning protocol is not evidence of circular fitting. No equation reduces to its own inputs, no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

The method rests on numerous hand-set weights, membership boundaries, and ad hoc rules, plus a hidden warm-up set for normalization. The draft-pass assumption is particularly fragile because the main evaluation protocol explicitly disables the draft pass while the controller requires its outputs.

free parameters (8)
  • NLP complexity weights = 0.15, 0.10, 0.10, 0.15, 0.20, 0.15, 0.15
    Eq. (2): hand-chosen to keep the score interpretable and stable (Appendix B); no sensitivity analysis or optimization.
  • Surface/NLP blend weights = 0.6, 0.4
    Eq. (3): weighted combination of surface and NLP complexity, chosen by hand without justification.
  • Second-stage refinement weights = 0.10, 0.08, 0.05, 0.08, 0.05, 0.04, 0.05
    Eq. (23): refinements for entropy, expected answer length, prompt type, history, semantic ambiguity, linguistic complexity, reasoning depth.
  • Uncertainty nudge alpha = 0.2
    Eq. (6) and Eq. (25): nudge upward when controller uncertainty is high.
  • Membership function boundaries = Input: L trap(0,0,0.2,0.5), M tri(0.2,0.5,0.8), H trap(0.5,0.8,1,1); output sets per Appendix C.1
    Appendix C.1: boundaries are hand-set, not learned.
  • Hard-prompt floor threshold = c > 0.6, min(6, Nmax)
    Eq. (27): ad hoc rule to prevent hard prompts from receiving very small budgets.
  • Prompt length normalization constant = 60 words
    Eq. (1): length term normalized by 60, chosen without evidence.
  • Min-max clipping ranges (a_i, b_i) = estimated on a warm-up set
    Eq. (9): normalization ranges are fit on a warm-up set, which is not described or released.
axioms (5)
  • domain assumption Prompt complexity can be estimated from hand-crafted surface and NLP features.
    Eqs. (1)-(3) and Appendix B: no evidence these features correlate with actual difficulty.
  • domain assumption Token-level confidence and entropy are reliable indicators of answer correctness.
    Eq. (4) and Section 3.1: the paper does not validate this against correctness.
  • domain assumption Average sample count is a faithful proxy for inference compute.
    Section 4 uses average N as compute proxy, ignoring token lengths and decoding cost.
  • domain assumption The warm-up set used for min-max normalization is representative and independent of evaluation.
    Appendix C.1, Eq. (9): no details on how the warm-up set was chosen.
  • ad hoc to paper In fair-alignment mode, the controller still has access to confidence and entropy despite the draft pass being disabled.
    Section 4 disables the draft pass, but Section 3.1 requires draft-derived signals; the paper does not explain how these are obtained. This assumption may be false.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpretable Adaptive Sampling for LLM Test-Time Scaling." pith.science (2026). https://pith.science/paper/RNLNPSG3

@misc{pith2026260803961,
  author       = {Pith},
  title        = {Pith review of: Interpretable Adaptive Sampling for LLM Test-Time Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RNLNPSG3}},
  note         = {Machine review of arXiv:2608.03961}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Test-time scaling improves LLM reasoning by generating and aggregating multiple candidate answers, yet many pipelines use fixed per-query budgets that spend the same compute on easy and difficult prompts. These fixed budgets are also difficult to inspect because they do not explain why a given prompt receives a particular number of samples. We propose adaptive} test-time scaling with a lightweight fuzzy controller that maps interpretable signals, including estimated prompt complexity and model confidence, to a per-query sampling budget. The controller assigns fewer samples to easier or more confident prompts and more samples to harder or less certain prompts, making inference-time compute inspectable rather than fixed or opaque. We evaluate under a fair-alignment protocol with matched decoding settings and controlled answer selection, and compare against best-of-$N$, compute-aware scaling, and self-certainty-based baselines on question-answering and mathematical reasoning tasks. Across models and datasets, adaptive fuzzy control improves over several standard baselines and remains close to a selector-matched full-budget control while reducing the average number of samples. These findings suggest that interpretable adaptive sampling is a practical direction for more efficient test-time reasoning in large language models.

Figures

Figures reproduced from arXiv: 2608.03961 by Ali Jannesari, Mobina Kashaniyan.

Figure 1
Figure 1. Figure 1: Adaptive test-time scaling pipeline. The system estimates prompt difficulty and uncertainty, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Primary fuzzy membership functions used by the controller for difficulty and confidence. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Main evaluation results under the fair-alignment setting. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Adaptive budget distributions under the fair-alignment setting. Each panel shows the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation accuracy–compute tradeoff on GSM8K and MATH with Phi-3-mini. Each point [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Trace-level diagnostics for the adaptive GSM8K run with Phi-3-medium and [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Additional run-level diagnostics for the adaptive GSM8K run with Phi-3-medium and [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Cumulative compute savings for Phi-3-medium adaptive GSM8K runs. The adaptive [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Cumulative compute savings for Qwen2.5-1.5B adaptive GSM8K runs. The adaptive [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 17 canonical work pages

  1. [1]

    Rethinking interpretability in the era of large language models

    Singh C, Inala JP, Galley M, Caruana R, Gao J. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761. 2024

    Pith/arXiv arXiv 2024

  2. [2]

    InstructZero: Efficient instruction optimization for black-box large language models

    Chen L, Chen J, Goldstein T, Huang H, Zhou T. InstructZero: Efficient instruction optimization for black-box large language models. arXiv preprint arXiv:2306.03082. 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Self-consistency improves chain of thought reasoning in language models

    Wang X, Wei J, Schuurmans D, Le Q, Chi E, Narang S, Chowdhery A, Zhou D. Self-consistency improves chain of thought reasoning in language models. ICLR, 2023

    work page 2023

  4. [4]

    Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning

    Snell C, Lee J, Xu K, Kumar A. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. ICLR, 2025

    work page 2025

  5. [5]

    Atom of thoughts for Markov LLM test-time scaling

    Teng F, Shi Q, Yu Z, Zhang J, Luo Y , Wu C, Guo Z. Atom of thoughts for Markov LLM test-time scaling. arXiv preprint arXiv:2502.12018. 2025

    arXiv 2025

  6. [6]

    Can 1B LLM surpass 405B LLM? Rethinking compute-optimal test-time scaling

    Liu R, Gao J, Zhao J, Zhang K, Li X, Qi B, Ouyang W, Zhou B. Can 1B LLM surpass 405B LLM? Rethinking compute-optimal test-time scaling. arXiv preprint arXiv:2502.06703. 2025

    Pith/arXiv arXiv 2025

  7. [7]

    MCTS-Judge: Test-time scaling in LLM-as-a-judge for code correctness evaluation

    Wang Y , Ji P, Yang C, Li K, Hu M, Li J, Sartoretti G. MCTS-Judge: Test-time scaling in LLM-as-a-judge for code correctness evaluation. arXiv preprint arXiv:2502.12468. 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Scaling LLM test-time compute optimally can be more effective than scaling model parameters

    Snell C, Lee J, Xu K, Kumar A. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314. 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Towards thinking-optimal scaling of test-time compute for LLM reasoning

    Yang W, Ma S, Lin Y , Wei F. Towards thinking-optimal scaling of test-time compute for LLM reasoning. arXiv preprint arXiv:2502.18080. 2025

    arXiv 2025

  10. [10]

    Rethinking the role of prompting strategies in LLM test-time scaling

    Liu Y , Li Z, Fang Z, Xu N, He R, Tan T. Rethinking the role of prompting strategies in LLM test-time scaling. ACL, 2025

    work page 2025

  11. [11]

    Trae Agent: An LLM-based agent for software engineering with test-time scaling

    Gao P, Tian Z, Meng X, Wang X, Hu R, Xiao Y , Liu Y , Zhang Z, Chen J, Gao C, Lin Y . Trae Agent: An LLM-based agent for software engineering with test-time scaling. arXiv preprint arXiv:2507.23370. 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Benchmark test-time scaling of general LLM agents

    Li X, Ming R, Setlur P, Paladugu A, Tang A, Kang H, Shao S, Jin R, Xiong C. Benchmark test-time scaling of general LLM agents. arXiv preprint arXiv:2602.18998. 2026

    arXiv 2026

  13. [13]

    FastTTS: Accelerating test-time scaling for edge LLM reasoning

    Chen HM, Mo Z, Lu G, Liang S, Ma L, Luk W, Fan H. FastTTS: Accelerating test-time scaling for edge LLM reasoning. ASPLOS, 2026

    work page 2026

  14. [14]

    GTA1: GUI test-time scaling agent

    Yang Y , Li D, Dai Y , Luo Z, Zhao Z, Hu Z, Huang J, Saha A, Chen Z, Xu R. GTA1: GUI test-time scaling agent. arXiv preprint arXiv:2507.05791. 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Can test-time scaling improve world foundation model? arXiv preprint arXiv:2503.24320

    Cong W, Zhu H, Wang P, Liu B, Xu D, Wang K, Pan DZ, Wang Y , Fan Z, Wang Z. Can test-time scaling improve world foundation model? arXiv preprint arXiv:2503.24320. 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Kinetics: Rethinking test-time scaling laws

    Sadhukhan R, Chen Z, Zheng H, Zhou Y , Strubell E, Chen B. Kinetics: Rethinking test-time scaling laws. arXiv preprint arXiv:2506.05333. 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Z1: Efficient test-time scaling with code

    Yu Z, Wu Y , Zhao Y , Cohan A, Zhang XP. Z1: Efficient test-time scaling with code. EMNLP Industry Track, 2025

    work page 2025

  18. [18]

    Efficient test-time scaling via self-calibration

    Huang C, Huang L, Leng J, Liu J, Huang J. Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503.00031. 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Chen Z, Wang W, Cao Y , Liu Y , Gao Z, Cui E, Zhu J, Ye S, Tian H, Liu Z, Gu L. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271. 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Revisiting the test-time scaling of o1-like models

    Zeng Z, Cheng Q, Yin Z, Zhou Y , Qiu X. Revisiting the test-time scaling of o1-like models. ACL, 2025

    work page 2025

  21. [21]

    A survey on test-time scaling in large language models

    Zhang Q, Lyu F, Sun Z, Wang L, Zhang W, Hua W, Wu H, Guo Z, Wang Y , Muennighoff N, King I. A survey on test-time scaling in large language models. arXiv preprint arXiv:2503.24235. 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting

    Turpin M, Michael J, Perez E, Bowman S. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems. 2023 Dec 15;36:74952-65

    work page 2023

  23. [23]

    Scalable best-of-n selection for large language models via self-certainty

    Kang Z, Zhao X, Song D. Scalable best-of-n selection for large language models via self-certainty. arXiv preprint arXiv:2502.18581. 2025 Feb 25

    arXiv 2025

  24. [24]

    Training verifiers to solve math word problems

    Cobbe K, Kosaraju V , Bavarian M, Chen M, Jun H, Kaiser L, Plappert M, Tworek J, Hilton J, Nakano R, Hesse C, Schulman J. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. 2021

    Pith/arXiv arXiv 2021

  25. [25]

    Measuring mathe- matical problem solving with the MATH dataset

    Hendrycks D, Burns C, Kadavath S, Arora A, Basart S, Tang E, Song D, Steinhardt J. Measuring mathe- matical problem solving with the MATH dataset. In: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. 2021

    work page 2021

  26. [26]

    Crowdsourcing multiple choice science questions

    Welbl J, Liu NF, Gardner M. Crowdsourcing multiple choice science questions. arXiv preprint arXiv:1707.06209. 2017. 10 A Appendix Overview The appendix contains details that are useful for reproducibility and interpretation. The main paper focuses on the question of whether adaptive budgeting improves the accuracy–compute tradeoff under fair comparison. T...

    Pith/arXiv arXiv 2017

  27. [27]

    Ifcis L andγis H, thensis VL

    work page

  28. [28]

    Ifcis L andγis M, thensis L

    work page

  29. [29]

    Ifcis L andγis L, thensis M. 12

    work page

  30. [30]

    Ifcis M andγis H, thensis L

    work page

  31. [31]

    Ifcis M andγis M, thensis M

    work page

  32. [32]

    Ifcis M andγis L, thensis H

    work page

  33. [33]

    Ifcis H andγis H, thensis M

    work page

  34. [34]

    Ifcis H andγis M, thensis H

    work page

  35. [35]

    Ifcis H andγis L, thensis VH. Second-stage refinements.The first-stage output is refined using entropy η, expected answer length λ, prompt type τ, history-cache score π, semantic ambiguity asem, linguistic complexity aling, and reasoning-depth scorea rea. Each refinement is small by design: ∆ = 0.10η+ 0.08λ+ 0.05τ+ 0.08(1−π) + 0.05a sem + 0.04aling + 0.05...

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.