{"id":"7ee8b015-15cb-4452-a75f-2153fe05ca77","arxiv_id":"2502.07191","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An empirical study showing that inference-time LLM reasoning methods are highly sensitive to sampling hyperparameters and reward-model choices, with recommended defaults of temperature 0.8 and top-p 0.9.","lead":"This paper benchmarks six inference-time reasoning methods (Best-of-N, self-consistency, beam search, MCTS, self-refine, step-level Best-of-N) across eight reasoning tasks and ablates prompts, temperature, top-p, self-evaluation, and reward models. It reports that simple hyperparameter choices, such as temperature near 0.8 and top-p near 0.9, can improve accuracy by a few percent, while self-evaluation often fails and reward-model gains are inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The recommended τ=0.8/top-p=0.9 default is an in-sample, one-at-a-time grid conclusion, and the paper's own combination experiments (Tables 3–4) show non-additive interactions that contradict it.","rationale":"The reader's weakest assumption concerns reward-signal validity. I agree that an unreliable verifier would change the method rankings, and this is a serious concern, but it is not the most direct vulnerability of the central claim. Even with a perfect verifier, the paper has not demonstrated that (0.8, 0.9) is a reliable joint optimum: the sweeps are one-at-a-time, single-seed, and in-sample, and Section 4.2.3 plus Tables 3–4 contain explicit evidence of non-additive interactions and task-dependent reversals. The more basic load-bearing condition is therefore that the recommended default survives a joint grid with variance estimates. A full grid with seeds and paired confidence intervals would settle whether the headline 'previously overlooked tricks' claim is real or an artifact of selection and noise. This concern does not overturn the paper's useful qualitative findings (self-evaluation is weak, reward models can mislead, prompt design matters), so the CONDITIONAL verdict remains appropriate; no verdict change is needed.","tokens_in":19288,"tokens_out":6070,"duration_ms":57916,"concrete_test":"Run a full 5×5 joint sweep over τ ∈ {0.6, 0.7, 0.8, 0.9, 1.0} and top-p ∈ {0.6, 0.7, 0.8, 0.9, 1.0} on all eight tasks for Llama-3.1-8B and Qwen-2.5-7B, with at least 10 independent seeds per cell, and report paired bootstrap 95% confidence intervals for each (τ, top-p) cell relative to the baseline (0.7, 0.9). If (0.8, 0.9) is not within the confidence-interval set of the best cell for a majority of task-model pairs, or if the claimed 2.3%–5.9% gains lie within the CI of the baseline, the default-setting recommendation fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central practical claim is that temperature τ=0.8 and top-p=0.9 are optimal defaults, improving accuracy by 2.3%–5.9% on several benchmarks. For this to hold, (0.8, 0.9) must be jointly and robustly better than nearby settings across tasks and models, not merely the best point in two separate one-dimensional sweeps. The paper does not establish this. Section 4.2.3 explicitly reports that 'improvements are not always additive,' and Tables 3–4 provide counterexamples: on Llama-3.1-8B, CoT+Majority at τ=0.7/top-p=0.8 reaches 56.4 on Bamboogle while the recommended 0.8/0.9 gives 54.8; on MATH, increasing τ from 0.7 to 0.8 at top-p=0.9 decreases accuracy from 58.4 to 57.0. Figures 3–4 show raw grid points without error bars or seed variance, and the quoted improvement ranges appear to be best-minus-worst in-sample differences. Therefore the headline default recommendation is not yet supported: the observed gains could be sampling noise or artifacts of one-at-a-time tuning, and the paper's own combination results indicate the joint optimum is task- and model-dependent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates inference-time computation methods for LLM reasoning, framing them as a proposer-verifier pipeline that generates candidate solutions and then selects the best one with a reward signal. It conducts ablations over candidate-generation choices (instruction prompt type, temperature, top-p) and selection choices (self-evaluation, reward model types), and benchmarks six inference-time methods (Best-of-N, Step-Level Best-of-N, Self-Consistency, Beam Search, MCTS, Self-Refine) on eight reasoning tasks using Llama, Qwen, and Mistral families. The central claims are that temperature tau=0.8 and top-p=0.9 improve accuracy by about 2.3% to 5.9%, that self-evaluation is largely ineffective, that reward models can inflate performance due to generalization issues, and that combining tricks does not always yield additive gains. The paper reports over 20,000 A100-80G GPU hours and more than 1,000 experiments, and it releases code.","tokens_in":19577,"tokens_out":7024,"duration_ms":58569,"significance":"If the claims were fully supported, the paper would provide useful practical guidance on default sampling settings and a broad benchmark for test-time compute. It deserves credit for the large experimental effort, the released code, the coverage of multiple model families and tasks, and the honest reporting that combinations are non-additive and reward models are inconsistent across tasks. However, the headline numerical claims are currently in-sample grid selections without uncertainty quantification, the recommended default is contradicted by the paper's own combination tables, and the 'fixed token budget' comparison is not actually a fixed-token comparison. The benchmark and practical recommendations would be valuable after these issues are addressed, but in its present form the central claims are not yet supported.","major_comments":[{"comment":"The recommended default (τ=0.8, top-p=0.9) is derived from one-dimensional sweeps in Figures 3-4 and is presented as optimal, but the paper's own combination experiments show that it is not the joint optimum. On Llama-3.1-8B, Table 3 reports CoT Majority at τ=0.7/top-p=0.8 reaching 56.4 on Bamboogle and 57.8 on MATH, while the recommended 0.8/0.9 gives 54.8 and 57.0; on Qwen-2.5-7B, Table 4 reports Reflect CoT at 0.7/0.9 giving 49.2/74.6 on Bamboogle/MATH while 0.8/0.9 gives 49.2/69.2. Since Section 4.2.3 itself states that improvements are not always additive, the headline 'optimal' settings are at best in-sample one-at-a-time optima, not robust defaults. Please provide a joint grid around the claimed optimum (with multiple seeds) or reframe the recommendation as task- and model-dependent tuning guidance.","section":"Section 4.2.1 and Tables 3-4"},{"comment":"The 'Fixed Token Budget' comparison in Table 2 is not a fixed-token comparison. The reported token counts vary widely within each model block: for Llama-3.1-8B, Self-Refine uses 287 tokens on Bamboogle while Best-of-N uses 1077, and on MATH MCTS uses 1898 tokens while Best-of-N uses 1412; for Qwen-2.5-7B, MCTS on HotpotQA uses 2585 tokens versus 1055 for Step-Level Best-of-N. If compute is not matched, the rankings in Table 2 and Figure 15 confound method quality with token consumption. Please either enforce a genuine token budget (e.g., equal total generation tokens or equal sampling budget) or rename the comparison and remove the equal-compute interpretation.","section":"Section 4.3 and Table 2"},{"comment":"All accuracy numbers are single-point estimates without error bars, multiple seeds, or significance tests, despite the paper reporting over 1,000 experiments. The claimed gains of 2.32%-5.88% in Section 4.2.1 are close to the sampling noise one would expect for evaluation sets of the sizes used here (e.g., MATH500 has 500 problems and HumanEval has 164 problems), especially with N=32 candidates. The rankings in Figure 7 and Table 2 could plausibly change under seed variance. Please report confidence intervals, seed-level variance, or at least the number of evaluation examples and a significance test for the headline comparisons.","section":"Figures 3-5, 9-14 and Tables 2-4"},{"comment":"The paper shows that reward models are inconsistent and can 'inflate' performance across tasks, yet Table 2 is built by selecting candidates with those same reward models (RLHF reward for knowledge-based tasks, QwQ-32B process reward for complex tasks). If the reward signal does not track true correctness on a given task, the observed method rankings and the 'performance inflation' interpretation both become unreliable. Please add a validation of each reward model (e.g., agreement with ground-truth labels, or an oracle-verifier upper bound) or restrict benchmark conclusions to tasks where such validation is provided.","section":"Section 4.2.2 and Figure 7"}],"minor_comments":[{"comment":"The section title 'Preliminares' should be renamed to 'Preliminaries'.","section":"Section 3"},{"comment":"There are typographical errors in Appendix B: 'Tempature' appears in the heading before Figure 10, and 'Ferver' appears in the captions of Figures 9 and 12; these should be 'Temperature' and 'FEVER'.","section":"Appendix B"},{"comment":"Table 4 lists the row 'Reflect CoT Majority 0.8 0.9 49.2 69.2' twice; the duplicate row should be removed.","section":"Table 4"},{"comment":"The reported improvements of 'approximately 2.32% to 4.83%' for temperature and '2.32%-5.88%' for top-p do not state the baseline; specify whether these are best-versus-default or best-versus-worst differences.","section":"Section 4.2.1"},{"comment":"The Conclusion mentions 'top-k sampling' even though the study varies top-p, not top-k; the terminology should be corrected.","section":"Section 5"},{"comment":"In Figures 8 and 15, some panels use a logarithmic x-axis and others use a linear axis without annotation; the axis scale should be stated in the captions.","section":"Figures 8 and 15"},{"comment":"The definition of top-p says tokens are considered when the cumulative probability is 'greater than' the top-p value; standard nucleus sampling selects the smallest set with cumulative probability at least p, so the wording should be corrected to avoid an inverted description.","section":"Section 3 (Top-p definition)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a substantial empirical resource, but the main claims and benchmark need more rigorous statistics and a genuine equal-compute setup. The in-sample grid-selection issue is the key risk: the recommended defaults are not supported by the paper's own combination tables. I would ask the authors to add multi-seed or joint-grid validation, and to correct the 'fixed token budget' wording. With those changes the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: useful empirical sweep, not a new-method paper, and the headline temperature/top-p default is not actually supported by the evidence. The qualitative findings—self-evaluation is weak, reward models give inconsistent rankings—are the most defensible part.\n\nWhat's genuinely useful: the paper does a broad, systematic comparison (six methods, eight tasks, five models) with 20k GPU hours, and it publishes code. That kind of standardized benchmark is sorely needed. The observation that reward-model scaling can hurt performance (Figure 7) is worth taking seriously, and the self-evaluation negative result is consistent with prior work. These are real contributions.\n\nThe soft spot is the central claim. The recommended τ=0.8, top-p=0.9 is the best point in two separate one-dimensional sweeps, with no error bars or significance tests, and it's selected on the same benchmarks used to report the gains. That's in-sample tuning. Worse, the paper's own combination experiments (Table 3) contradict the joint recommendation: on Llama-3.1-8B, Bamboogle is better at (0.7, 0.8) than at (0.8, 0.9), and on MATH, moving from 0.7 to 0.8 at top-p 0.9 drops accuracy from 58.4 to 57.0. So the claim that these are generally optimal defaults doesn't hold up. The authors even say improvements are not additive, which undercuts the headline.\n\nA second issue: Table 2 is labeled 'Fixed Token Budget' but token counts vary widely within a task—for MCTS on HotpotQA, Qwen-2.5-7B uses 2585 tokens while Self-Refine uses 316. That makes the comparison incommensurable. Either fix the table title or actually control the budget.\n\nMinor: the reward-model assignment differs by task, which is reasonable, but the paper's own results show reward models can mislead; the benchmark inherits that. That's acknowledged, but it should be noted in the main text as a caveat.\n\nVerdict: not a reject. The benchmark is worth having and the negative results are informative. But the actionable default recommendation needs to be reframed as 'best grid points we found, not a validated optimum,' and the fixed-token comparison needs repair. A serious referee should engage with it.\n\nRecommendation: send to peer review with a request for major revision.","headline":"A useful but flawed empirical sweep: the qualitative findings (self-evaluation weak, reward models inconsistent) are solid, but the headline τ=0.8/top-p=0.9 default is in-sample tuning contradicted by the paper's own combination tables.","tokens_in":20098,"tokens_out":2720,"would_cite":false,"duration_ms":22859,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that inference-time reasoning gains depend heavily on two under-tuned sampling parameters—temperature near 0.8 and top-p near 0.9—and uses more than 1,000 controlled experiments to show these lift accuracy by 2.3–5.9…","keywords":["inference-time computation","LLM reasoning","Best-of-N sampling","temperature tuning","top-p nucleus sampling","reward models","self-evaluation","reasoning benchmarks"],"falsifier":"Run the same temperature and top-p sweeps on the same tasks but select candidates using ground-truth correctness (which is known for GSM8K, MATH, and HumanEval test cases) instead of learned rewards; if the 2.3–5.9 point gains at $\\tau=0.8$ and $\\text{top-p}=0.9$ disappear or peak elsewhere, the recommendation is an artifact of reward-model bias rather than a property of sampling diversity.","tokens_in":19069,"feed_emoji":"🎲","tokens_out":6725,"duration_ms":53549,"temperature":0.7,"pith_summary":"Most inference-time computation for LLM reasoning works by generating many candidate solutions and then picking one with a reward signal. This paper argues that the unglamorous settings around that pipeline—what prompt is used, what temperature and top-p sample the candidates, and what reward model scores them—can shift accuracy more than the choice among search methods such as Best-of-N, beam search, or MCTS. Through more than 1,000 experiments covering five model families and eight tasks, it finds that a temperature $\\tau=0.8$ and a nucleus-sampling threshold $\\text{top-p}=0.9$ are near-optimal defaults, improving accuracy by roughly 2.3 to 5.9 percentage points compared with common settings. It also finds that self-evaluation by the model itself is unreliable, sometimes worse than random selection, and that learned reward models can inflate or degrade scores depending on the task. The result is a standardized benchmark with equal token budgets for six inference-time methods, intended as a common foundation for future work.","feed_headline":"Temperature 0.8, top-p 0.9 lift LLM reasoning by up to 5.9 percent","feed_subtitle":"A 1,000-experiment study finds these sampling defaults matter as much as the search strategy on eight reasoning tasks.","key_machinery":"The organizing mechanism is the proposer-verifier pipeline: an inference model samples $K$ candidate solutions (proposals), and a separate reward signal selects the best (verification). The paper's argument is carried by controlled ablations of that pipeline's two stages—candidate generation is varied through instruction prompt type (IO, CoT, reflection CoT), temperature $\\tau \\in \\{0.6,\\dots,1.0\\}$, and top-p $p \\in \\{0.6,\\dots,1.0\\}$; selection is varied through majority vote, random choice, self-evaluation (process and result), and four reward types (RLHF, process, proof-critical, and LLM-as-judge). The central quantities are the sampling knobs $(\\tau, p)$ and the reward-ranking quality, because the paper attributes both the performance gains and the observed performance inflation to them.","core_discovery":"On the paper's own terms, the central discovery is that inference-time computation methods are highly sensitive to experimental configuration, and several overlooked configuration choices are responsible for a large share of reported gains. Using a proposer-verifier pipeline as the organizing frame, the paper shows that chain-of-thought prompting beats plain input-output prompts, that sampling with $\\tau=0.8$ and $\\text{top-p}=0.9$ consistently outperforms the lower and higher settings tested (approximately 2.32%–5.88% accuracy gains across GSM8K, MATH500, HumanEval, and Bamboogle), and that among selection mechanisms, process-based rewards from an external judge help most on complex math and code, while self-evaluation often fails to rank solutions better than chance. The paper further claims that reward-model generalization is the reason scaling the number of candidates does not monotonically improve accuracy, and that combining individually useful tricks is not reliably additive, so configurations must be validated per model and task. As a benchmark contribution, it ranks six methods—Best-of-N, Step-Level Best-of-N, Self-Consistency, Beam Search, MCTS, and Self-Refine—under matched token budgets on eight reasoning tasks.","pith_inferences":["If the 2–6% gains survive verification with oracle ground-truth labels, the result would support a diversity mechanism the paper does not directly measure: a slightly more diverse candidate pool at $\\tau=0.8$ with moderate truncation at $\\text{top-p}=0.9$ exposes more correct trajectories for a reward model to find.","A natural testable extension is to check whether the same $(\\tau=0.8,\\text{top-p}=0.9)$ optimum transfers to newer models, non-English reasoning, or multimodal tasks, since the paper only varies architecture families and sizes within one generation of models.","The benchmark's equal-token results suggest token efficiency, not just accuracy, should be a headline metric for inference-time methods; from Table 2, Self-Consistency often reaches high accuracy cheaply while Beam Search consumes tokens without proportional gains, though the paper does not foreground this as a recommendation.","If reward-model inflation is as task-dependent as Figure 7 indicates, then reported method rankings on unverified benchmarks may partly measure verifier alignment rather than reasoning ability; researchers should report both reward-selected and oracle-selected accuracy."],"forward_implications":["Inference-time methods should be reported with their full generation settings—prompt type, temperature, and top-p—because these can change accuracy by as much as the method choice.","Practitioners should treat $\\tau=0.8$ and $\\text{top-p}=0.9$ as the default starting point for Best-of-N-style candidate generation instead of the $\\tau=0.6$–$0.7$ settings common in prior work.","Self-evaluation by the generating model should not be used as a verifier without external feedback; the paper finds it can underperform random selection.","More tokens and larger $N$ do not guarantee better answers; reward-model generalization limits the returns from scaling, and can even reverse them on hard tasks.","Combining separately effective tricks requires re-validation, because the paper finds the improvements are not additive."],"supporting_citations":[{"why":"It introduces Best-of-N sampling with learned verifiers, the core inference-time computation method the paper ablates and benchmarks.","marker":"(Cobbe et al., 2021a)"},{"why":"It supplies Self-Consistency, the selection-by-majority-vote baseline that performs strongly in the paper's benchmark.","marker":"(Wang et al., 2022)"},{"why":"It defines chain-of-thought prompting, the default instruction prompt type used throughout the ablations.","marker":"(Wei et al., 2022)"},{"why":"It is the prior result the paper extends and confirms, showing LLMs cannot reliably self-correct reasoning without external feedback.","marker":"(Huang et al., 2023)"},{"why":"It provides the process reward model and evidence that process supervision generalizes unevenly across math tasks.","marker":"(Zheng et al., 2024)"},{"why":"It documents generalization limits of process reward models that the paper invokes to explain performance inflation.","marker":"(Zhang et al., 2025)"},{"why":"It supplies the RLHF reward model used as one of the external selection signals in the experiments.","marker":"(Cai et al., 2024)"},{"why":"It provides the proof-critical critic reward used for selection in math and code reasoning tasks.","marker":"(Wu et al., 2024b)"}],"fun_headline_variants":["Temperature and top-p: the overlooked keys to LLM reasoning gains","Sampling settings, not search, drive LLM reasoning improvement","Why tuning temperature beats fancy search for LLM reasoning","LLM reasoning boosted 5% by simple sampling tweaks","Inference-time tricks: temperature and prompts matter most"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the external reward signals used for selection—the RLHF reward, the proof-critical critic, and the judge model—rank candidate solutions correctly and comparably across all eight tasks, even though the paper's own Figure 7 shows these rewards can inflate or degrade accuracy depending on the task.","fun_headline_variants_meta":{"raw":{"variants":["Temperature and top-p: the overlooked keys to LLM reasoning gains","Sampling settings, not search, drive LLM reasoning improvement","Why tuning temperature beats fancy search for LLM reasoning","LLM reasoning boosted 5% by simple sampling tweaks","Inference-time tricks: temperature and prompts matter most"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000582,"raw_usage":{"total_tokens":2821,"prompt_tokens":1108,"completion_tokens":1713,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":724,"completion_tokens_details":{"reasoning_tokens":1630}},"tokens_in":724,"tokens_out":1713,"duration_ms":13170,"temperature":1.0,"reasoning_tokens":1630,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:31:15.216748+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same temperature and top-p sweeps on the same tasks but select candidates using ground-truth correctness (which is known for GSM8K, MATH, and HumanEval test cases) instead of learned rewards; if the 2.3–5.9 point gains at $\\tau=0.8$ and $\\text{top-p}=0.9$ disappear or peak elsewhere, the recommendation is an artifact of reward-model bias rather than a property of sampling diversity.","supporting_citations":[],"review_version":1}