Pith. sign in

REVIEW 4 major objections 5 minor 54 references

Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fusing Best-of-N sampling, PRM-gated self-refinement, and tree search at the level of a single reasoning step lifts five instruction-tuned LLMs' reasoning accuracy by up to 28.6% and lets a 3B model surpass an RL-enhanced 7B model.

desk verdict Shows a plausible hybrid recipe, but the headline claim against RL rests on a Pass@16 vs single-output mismatch. read the letter →

arxiv 2507.15512 v3 pith:66SIX45X submitted 2025-07-21 cs.CL

Kaiyan Chang , Yonghao Shi , Chenglong Wang , Hang Zhou , Chi Hu , Xiaoqian Liu , Yingfeng Luo , Yuan Ge
show 2 more authors
Tong Xiao Jingbo Zhu
This is my paper
classification cs.CL
keywords test-timescalingprocessrewardmodelself-refinementbest-of-NsamplingMonteCarlotreesearchLLMreasoningstep-levelverificationtraining-freeinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that for open instruction-tuned LLMs, the best use of extra inference compute is not longer thinking or whole-solution self-critique but step-level search guided by a process reward model (PRM), a scorer that rates each reasoning step as it is produced. It builds a training-free pipeline that samples several candidate next steps, picks one with a PUCT-style score, and then, only if that step still scores below a threshold, invites the model to reflect on and rewrite just that step, accepting the rewrite only when the PRM rates it higher. On five models from 3B to 14B across MATH500, AIME24, and GPQA Diamond, the method reports consistent gains, a maximum single improvement of 28.6%, and the result that Qwen2.5-3B-Instruct reaches 51.5% on GPQA Diamond, 2.4 points above an RL-enhanced DeepSeek-R1-Distill-Qwen-7B. The paper concludes that the essence of test-time scaling is exploring and exploiting the model's latent solution space, and that an accurate step verifier can substitute for continued training.

What carries the argument

The load-bearing object is the step-level process reward model (PRM), a scorer that takes the question plus all preceding steps and outputs a 0-1 correctness score for the newest step. One trained PRM, Qwen2.5-Math-PRM-7B, does triple duty: it supplies the value term in the PUCT selection among sampled step candidates, it gates whether self-reflection runs and whether a rewrite is kept (the PRM_Cover rule), and it provides the stopping signals (score above 0.9, or an improvement gap below 0.2 over two consecutive rounds). The other named component is Deep Scaling, the replacement of the MCTS rollout phase with this multi-sample-plus-conditional-refinement procedure, so that the search expands one verified step at a time instead of simulating whole solutions. The full procedure (Algorithm 1) runs k search paths per problem, draws N candidate steps per node, and finalizes with majority vote or per-path minimum PRM score.

What would settle it

Construct a probe set of reasoning problems with independently known step correctness, compute the PRM's scores on all five models' steps, and measure how often the score ordering disagrees with true correctness; then re-run Hybrid Test-Time Scaling with the reflection gate inverted on exactly those mis-scored steps and check whether gains flip sign. A cheaper check of the headline claim is to repeat the GPQA Diamond runs over several seeds, since the paper reports a single run and documents visible fluctuation on that set, and to swap the verifier for one trained on science-domain steps, comparing the 2.4% margin over the RL-trained 7B model against the seed variance.

Watch

Extended reading notes

Core claim

The central claim is that parallel and sequential test-time scaling methods are not rivals but orthogonal tools that compose best at the granularity of a single reasoning step. The paper's conditional step-level self-refinement redesigns the critique-and-rewrite loop: reflection fires only when the PRM score of the current step is below a threshold, the rewritten step is adopted only if it out-scores the original, and iteration stops when the score passes 0.9, when two consecutive rounds gain less than 0.2, or after five rounds. Hybrid Test-Time Scaling wraps this refinement in a Monte Carlo tree search over steps, where each expansion first draws N candidate steps and selects one by a PUCT score combining the PRM value estimate, the LLM's prior, and visit counts, a 'best-of-the-best' cycle the paper calls Deep Scaling, which replaces the standard MCTS rollout. The experiments claim consistent superiority over parallel-only (MCTS+Best-of-N), sequential-only (Best-of-N plus self-refinement), and solution-level (OpenR) hybrids under matched budgets, and the authors infer that the residual gap between Pass@k and Maj@k/RM@k across their models is exactly what an improved verifier would close.

Load-bearing premise

The entire method leans on one premise: the process reward model Qwen2.5-Math-PRM-7B assigns trustworthy step-level scores for all five tested models on all three benchmarks, including GPQA Diamond science questions far outside its math training distribution.

Editorial extensions

If this is right

  • Smaller instruction-tuned models can match or beat much larger, RL-trained models on hard reasoning benchmarks without any additional training, provided a strong step-level verifier is available.
  • The gap between a model's Pass@k (what it can generate) and its Maj@k/RM@k (what it can reliably select) is the measurable target of test-time scaling; better verifiers should convert latent reasoning into realized accuracy.
  • Self-refinement must be gated by verification: unconditional critique and rewrite degrades accuracy below baseline, while PRM-gated rewriting improves steadily with iterations.
  • Parallel and sequential scaling methods are complementary at step level rather than mutually exclusive; every pairwise combination underperforms the full three-component hybrid.
  • Test-time compute and continued RL are alternative levers on the same latent reasoning resource, so training-free search can substitute for training whenever the verifier is accurate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the authors only gesture at: their Appendix D shows PRM score sums rise linearly with problem difficulty, so a difficulty-aware controller could spend refinement budget only on mid-difficulty problems and skip easy or hopeless ones, improving the efficiency figures in Appendix C.
  • Because the paper runs each configuration once and its own Appendix E shows GPQA results fluctuating under identical settings, the 3B-beats-7B-RL headline should be treated as a candidate finding until confirmed across seeds; the limitations section says as much.
  • If verifier quality is the binding constraint, the next round of gains lies in verifiers rather than generators: a domain-general or science-domain PRM is a direct replacement test, and the paper's own two-PRM comparison predicts larger gaps when the verifier is weaker.
  • The same step-verification machinery could be reused at training time, as a data-quality filter or as dense reward shaping for RL, a consequence the authors leave implicit that would compound the gains of both paradigms.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes a training-free test-time scaling method that combines parallel and sequential scaling at the step level. At each reasoning step, the algorithm samples N candidate steps, selects one with a PUCT score that blends a process reward model (PRM) value with an LLM prior, conditionally refines that step under PRM-gated acceptance rules (threshold 0.9, PRM_Cover, gap@2 < 0.2, max 5 iterations), and repeats this in an MCTS-style tree search until termination. The method is evaluated with Qwen2.5-Math-PRM-7B as the verifier on five instruction-tuned models (3B-14B) and three benchmarks (MATH500, AIME24, GPQA Diamond). The paper reports gains up to 28.6% and a headline claim that Qwen2.5-3B-Instruct outperforms DeepSeek-R1-Distill-Qwen-7B by 2.4% on GPQA Diamond.

Significance. If the empirical claims held, the paper would be significant: it would show that a fine-grained combination of training-free methods, namely Best-of-N, MCTS, and PRM-gated self-refinement, can elicit substantially more reasoning capability from small instruction-tuned models and narrow the gap with RL-trained systems in a compute-intensive regime. The paper has clear strengths: a public code release, explicit pseudocode (Algorithm 1), a multi-model and multi-benchmark sweep, a comparison between two PRMs, a computational-overhead analysis (Appendix C), and a worked case study (Appendix F). The central quantitative claims, however, depend on evaluation choices that the current manuscript does not justify: hyperparameters are selected on MATH500 and then evaluated on the same set, all experiments are run once despite observed run-to-run variance of about 3 points, and the headline comparison to DeepSeek-R1-Distill-Qwen-7B uses Pass@16 against a single-output published score while the paper itself characterizes Pass@k as a latent-capability measure.

major comments (4)
  1. [Section 5.1 and Table 2] The refinement conditions (PRM threshold 0.9, gap@2 < 0.2, maximum 5 iterations) are selected by maximizing accuracy on MATH500 using Qwen2.5-7B-Instruct, and the same MATH500 set is then used as a main evaluation set in Table 2. The MATH500 gains, including the 28.6% improvement reported for LLaMA3.1-8B, are therefore not independent evidence for the method: the experiment cannot separate the method's effect from selection on the test set. The AIME24 and GPQA results are not affected by this particular circularity, but they inherit the single-run and PRM-calibration concerns raised below. The authors should evaluate on a held-out split or use a nested selection procedure.
  2. [Table 1, Abstract, and Section 5.2] The claim that Hybrid TTS 'surpasses' DeepSeek-R1-Distill-Qwen-7B by 2.4% on GPQA Diamond compares Pass@16 of Qwen2.5-3B-Instruct (51.5 in Table 2) with the published single-output accuracy of 49.1 for the baseline. The paper's own Section 5.2 describes Pass@k as a latent-capability measure and notes that smaller models have lower realized-selection metrics; for the same configuration, Maj@16 = 39.9 and RM@16 = 41.4, both below 49.1. In addition, Table 1 labels the baseline as 'Continued Reinforcement Learning', but DeepSeek-R1-Distill-Qwen-7B is obtained by supervised distillation from R1 data, not by continued RL. The headline therefore rests on an unmatched evaluation protocol and an inaccurate baseline characterization.
  3. [Section 4, Limitations, and Appendix E] All experiments are run once, as the Limitations section acknowledges. Appendix E, Figure 7 reports repeated runs under identical settings on MATH500 with accuracy varying between 85.4 and 88.4, a spread of 3.0 points. Several differences in Table 2 are of this size or smaller, for example Qwen2.5-3B MATH500 Maj@8 = 81.4 versus Maj@16 = 83.8, and LLaMA3.1-8B GPQA Pass@8 = 45.5 versus Pass@16 = 41.4. Without multiple seeds or variance estimates, the claims of consistent improvement and of a stable ordering among configurations are not statistically supported.
  4. [Sections 2.2, 3.2, and 4] The verifier Qwen2.5-Math-PRM-7B is a math-trained PRM applied to GPQA Diamond scientific questions, and the same PRM both gates refinement decisions (threshold 0.9, PRM_Cover) and selects the final answer under RM@k (the maximum per-path minimum PRM score). The paper never checks PRM calibration on held-out steps, nor does it compare with an outcome verifier or a different selection rule. The gap on GPQA Diamond between RM@16 (41.4) and Pass@16 (51.5) in Table 2 is consistent with the PRM failing to select correct paths; without calibration evidence, the claim in Section 5.2 that high-quality verification is the source of the gains is not established.
minor comments (5)
  1. [Table 4] The 'BoN+Self-Refinement' rows report only Pass@1 under the Best-of-4/8/16 columns, and the caption explains this only in prose; the table would be clearer if the omitted metrics were shown as em-dashes or if the pass@1 values were explicitly labeled as Pass@N for the corresponding N.
  2. [Appendix E, Figure 7] Figure 7 says the runs are under 'exactly the same experimental setup' but does not list the key setup details, such as temperature, sampling seed, or the exact Best-of/num-paths configuration; these details are needed for the variance claim to be interpretable.
  3. [Table 2, AIME24] The AIME24 results are reported in increments of 3.3 points, and the paper should state explicitly that the benchmark contains about 30 problems per year; this makes many of the AIME24 differences between configurations smaller than one or two questions.
  4. [Section 3.2 and Figure 5] The stopping condition 'gap@2 < 0.2' is not fully defined: it should state whether the 2-round improvement is an absolute or relative PRM-score difference and how it is computed when refinement is skipped in some iterations.
  5. [Abstract] The 'maximum performance increase of 28.6%' and the 'surpass by 2.4%' statements are presented without the metric and experimental caveats; the abstract should at least note that these are single-run Pass@k numbers with hyperparameters selected on MATH500.

Circularity Check

1 steps flagged · score 5.0 of 10

The MATH500 gains are reported on the same dataset used to tune the refinement thresholds and stopping rules, making that headline result partially fitted rather than predicted; AIME24 and GPQA are transferred without tuning and remain independent evidence.

  1. fitted input called prediction [Section 5.1 ('Reasonable conditions for improved step-level self-refinement') and Section 5.2 ('Main Results for Hybrid Test-Time Scaling'), Table 2]
    "To further balance the performance and efficiency of step-level self-refinement, we employed Qwen2.5-7B-Instruct on the MATH500 dataset to investigate optimal refinement conditions ... we set a maximum of 5 iteration rounds as the condition ... we conservatively adopt 0.9 to flag correct steps that do not require self-reflection ... we adopted a condition of 'PRM improvement over 2 rounds < 0.2' (gap@2<0.2 ) as the criterion ... Peak improvements reach 28.6% for LLaMA3.1-8B-Instruct on MATH500"

    The conditional self-refinement hyperparameters — maximum 5 iterations, reflection threshold 0.9, and gap@2<0.2 stopping rule — were selected by maximizing accuracy on MATH500 in Figures 5(a)–5(c). The main results then report accuracy on that same MATH500 set as the evidence that Hybrid TTS improves reasoning performance (Section 5.2, Table 2). On MATH500, the reported improvement is therefore not an independent prediction: it is the value of the objective used to choose the decision rules that define the method. The circularity is dataset-specific because the AIME24 and GPQA runs use the same thresholds without tuning, so those evaluations retain independent content; the GPQA 'surpass RL' claim is not rescued by this step but is also not circular in this particular way.

full rationale

The only concrete circular reduction I can exhibit is the MATH500 tuning-and-evaluation overlap: Section 5.1 explicitly selects the refinement conditions by experiments on MATH500, and Section 5.2 reports MATH500 accuracy as the peak improvement claim. That is a fitted-input-called-prediction pattern for that benchmark. On AIME24 and GPQA the same conditions are applied without re-tuning, so the central cross-domain claims are not reduced to their inputs by construction. I find no load-bearing self-citation: the only author-overlapping citation (Chang et al. 2024) supports prompt-design choices in Appendix A and is not the basis of any result. The PRM itself (Qwen2.5-Math-PRM-7B) is adopted based on the external PRM-Bench and is not defended by a self-citation chain. There is no imported uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result as a new prediction. The RM@k metric does use the same PRM that guides generation, but RM@k counts final-answer correctness after PRM selection, so it is not equivalent to the PRM score by definition. The headline 'surpass RL by 2.4%' comparison is questionable as an evaluation protocol, but that is a correctness/experimental-design concern rather than a circularity established by the paper's own equations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The empirical evaluation rests on PRM reliability, step-boundary control, several hyperparameters tuned on the same benchmark used for evaluation, and standard tree-search assumptions. No new entities are postulated.

free parameters (4)
  • PRM threshold for skipping reflection = 0.9
    Chosen from MATH500 accuracy sweeps in Figure 5(b); also used in final evaluation on the same MATH500 set.
  • PRM improvement stop condition (gap@2) = 0.2
    Chosen from MATH500 sweeps in Figure 5(c); used in the main results on the same dataset.
  • Maximum self-refinement iterations = 5
    Chosen from MATH500 iteration sweep in Figure 5(a); used for all main results.
  • PUCT exploration constants c1 and c2 = c1=1.25, c2=19,652
    The paper states these are set in experiments (Section 3.3); c2 value is unusual and may be a typo; no search procedure reported.
assumptions (4)
  • domain assumption The PRM (Qwen2.5-Math-PRM-7B) provides reliable step-level correctness scores for all tested instruction-tuned LLMs and datasets, including GPQA Diamond.
    The method's conditional refinement, step selection, and RM@k metric all depend on PRM scores; no per-dataset calibration is reported. See Section 3.2 and Section 5.2.
  • domain assumption Instruction-tuned LLMs naturally segment reasoning with '\n\n' so that Pause-then-Continue can control step boundaries.
    Section 3.1 relies on this. It may not hold for models without strong instruction-following (they restrict to >=3B instruction-tuned models).
  • ad hoc to paper The MATH500 accuracy used for hyperparameter selection (Section 5.1) is representative of the final evaluation setting.
    Conditions are tuned directly on MATH500 accuracy, then MATH500 appears in the main results (Table 2), making the improvement partly a fitted outcome.
  • standard math The PUCT formula behaves as in prior work; no proof is provided but it is widely used in tree-search scaling.
    Section 3.3 uses the PUCT formula from Schrittwieser et al. (2020); treated as standard background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models." pith.science (2026). https://pith.science/paper/66SIX45X

@misc{pith2026250715512,
  author       = {Pith},
  title        = {Pith review of: Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/66SIX45X}},
  note         = {Machine review of arXiv:2507.15512}
}
read the original abstract

Test-Time Scaling (TTS) is a promising approach to progressively elicit the model's intelligence during inference. Recently, training-based TTS methods, such as continued reinforcement learning (RL), have further surged in popularity, while training-free TTS methods are gradually fading from prominence. However, the additional computation overhead of training amplifies the burden on test-time scaling. In this paper, we focus on training-free TTS methods for reasoning. We first design Conditional Step-level Self-refinement, a fine-grained sequential scaling method guided by process verification. On top of its effectiveness, we further combine it with other classical parallel scaling methods at the step level, to introduce a novel inference paradigm called Hybrid Test-Time Scaling. Extensive experiments on five instruction-tuned LLMs across different scales (3B-14B) and families demonstrate that hybrid strategy incorporating various training-free TTS methods at a fine granularity has considerable potential for expanding the reasoning performance boundaries of LLMs.

Figures

Figures reproduced from arXiv: 2507.15512 by the authors.

Figure 1
Figure 1. Left: The two mainstream paradigms of training-free TTS, parallel and sequential scaling. Right: Our proposed Step-level Verifier-guided Hybrid Test-Time Scaling, which incorporates representative methods from both paradigms. Specifically, it performs “Deep Scaling” on each reasoning step within an MCTS framework through a two-stage screening process: (1) Multiple candidates are generated via Best-of-N sampling and … view at source ↗
Figure 2
Figure 2. Comparisons between four common training [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Experiments on different update strategies. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (2 more)
Figure 6
Figure 6. Figure 6: The correlation between solution-level PRM [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Results of repeated trials under exactly the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 15 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and 1 others. 2024. https://doi.org/10.1609/AAAI.V38I16.29720 Graph of thoughts: Solving elaborate problems with large language models . In Thirty-Eighth AAAI Conference on Artificial Intellige...

    doi:10.1609/aaai.v38i16.29720 2024

  4. [4]

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R \'e , and Azalia Mirhoseini. 2024. https://arxiv.org/abs/2407.21787 Large language monkeys: Scaling inference compute with repeated sampling . ArXiv preprint, abs/2407.21787

    arXiv 2024

  5. [5]

    Kaiyan Chang, Songcheng Xu, Chenglong Wang, Yingfeng Luo, Xiaoqian Liu, Tong Xiao, and Jingbo Zhu. 2024. https://arxiv.org/abs/2404.01077 Efficient prompting methods for large language models: A survey . ArXiv preprint, abs/2404.01077

    arXiv 2024

  6. [6]

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. https://arxiv.org/abs/2503.09567 Towards reasoning era: A survey of long chain-of-thought for reasoning large language models . ArXiv preprint, abs/2503.09567

    arXiv 2025

  7. [7]

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2024. https://arxiv.org/abs/2412.21187 Do not think that much for 2+3=? on the overthinking of o1-like llms . ArXiv preprint, abs/2412.21187

    arXiv 2024

  8. [8]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement lea...

    arXiv 2025

Show all 54 references
  1. [9]

    Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, and 1 others. 2025. https://arxiv.org/abs/2502.16235 Dynamic parallel tree search for efficient llm reasoning . ArXiv preprint, abs/2502.16235

    2025 arXiv

  2. [10]

    Yuchun Fan, Yongyu Mu, Yilin Wang, Lei Huang, Junhao Ruan, Bei Li, Tong Xiao, Shujian Huang, Xiaocheng Feng, and Jingbo Zhu. 2025. Slam: Towards efficient multilingual reasoning via selective language alignment. In Proceedings of the 31st International Conference on Computatio...

    2025

  3. [11]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. https://arxiv.org/abs/2305.11738 Critic: Large language models can self-correct with tool-interactive critiquing . ArXiv preprint, abs/2305.11738

    2023 arXiv

  4. [12]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . ArXiv preprint, abs/2407.21783

    2024 arXiv

  5. [13]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. https://arxiv.org/abs/2103.03874 Measuring mathematical problem solving with the math dataset . ArXiv preprint, abs/2103.03874

    2021 arXiv

  6. [14]

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference on Learning Representations

    2024

  7. [15]

    Yixin Ji, Juntao Li, Hai Ye, Kaixin Wu, Jia Xu, Linjian Mo, and Min Zhang. 2025. https://arxiv.org/abs/2501.02497 Test-time computing: from system-1 thinking to system-2 thinking . ArXiv preprint, abs/2501.02497

    2025 arXiv

  8. [16]

    Jikun Kang, Xin Zhe Li, Xi Chen, Amirreza Kazemi, Qianyi Sun, Boxing Chen, Dong Li, Xu He, Quan He, Feng Wen, and 1 others. 2024. https://arxiv.org/abs/2405.16265 Mindstar: Enhancing math reasoning in pre-trained llms at inference time . ArXiv preprint, abs/2405.16265

    2024 arXiv

  9. [17]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. https://arxiv.org/abs/2001.08361 Scaling laws for neural language models . ArXiv preprint, abs/2001.08361

    2020 arXiv

  10. [18]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

    2023

  11. [19]

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, and 1 others. 2025. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419

    2025 arXiv

  12. [20]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. In The Twelfth International Conference on Learning Representations

    2023

  13. [21]

    Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. 2025. Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling. In Workshop on Reasoning and Planning for Large Language Models

    2025

  14. [22]

    MAA. 2024. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions American invitational mathematics examination . Online

    2024

  15. [23]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. http://papers.n...

    2023

  16. [24]

    Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi, and Yasuhiro Sogawa. 2024. Enhancing reasoning capabilities of llms via principled synthetic logic corpus. Advances in Neural Information Processing Systems, 37:73572--73604

    2024

  17. [25]

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. 2025. https://arxiv.org/abs/2501.19393 s1: Simple test-time scaling . ArXiv preprint, abs/2501.19393

    2025 arXiv

  18. [26]

    OpenAI. 2024. https://openai.com/index/openai-o1-system-card/ Openai o1 system card . Accessed: 2024-11-07

    2024

  19. [27]

    Zhenting Qi, MA Mingyuan, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. 2025. Mutual reasoning makes smaller llms stronger problem-solver. In The Thirteenth International Conference on Learning Representations

    2025

  20. [28]

    Gollam Rabby, Farhana Keya, Parvez Zamil, and S \"o ren Auer. 2024. https://arxiv.org/abs/2411.15645 Mc-nest--enhancing mathematical reasoning in large language models with a monte carlo nash equilibrium self-refine tree . ArXiv preprint, abs/2411.15645

    2024 arXiv

  21. [29]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling

    2024

  22. [30]

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, and Arthur Guez. 2020. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604--610

    2020

  23. [31]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingchuan Zhang, Y. K. Li, Yu Wu, and Daya Guo. 2024. https://arxiv.org/abs/2402.03300 Deepseekmath: Pushing the limits of mathematical reasoning in open language models . ArXiv preprint, abs/2402.03300

    2024 arXiv

  24. [32]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. https://arxiv.org/abs/2408.03314 Scaling llm test-time compute optimally can be more effective than scaling model parameters . ArXiv preprint, abs/2408.03314

    2024 arXiv

  25. [33]

    Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. 2025. https://arxiv.org/abs/2501.03124 Prmbench: A fine-grained and challenging benchmark for process-level reward models . ArXiv preprint, abs/2501.03124

    2025 arXiv

  26. [34]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, and 1 others. 2025. https://arxiv.org/abs/2503.19786 Gemma 3 technical report . ArXiv preprint, abs/2503.19786

    2025 arXiv

  27. [35]

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. https://arxiv.org/abs/2211.14275 Solving math word problems with process-and outcome-based feedback . ArXiv preprint, abs/2211.14275

    2022 arXiv

  28. [36]

    Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Qiaozhi He, MuRun Yang, Bei Li, Tong Xiao, Chunliang Zhang, Tongran Liu, and 1 others. 2025. Gram: A generative foundation reward model for reward generalization. In Forty-second International Conference on Machine Learning

    2025

  29. [37]

    Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M Ni, and 1 others. 2024. https://arxiv.org/abs/2410.09671 Openr: An open source framework for advanced reasoning with large language models. corr, abs/2410.09671, 2024b. ...

    2024 arXiv

  30. [38]

    Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2023 a . https://arxiv.org/abs/2312.08935 Math-shepherd: Verify and reinforce llms step-by-step without human annotations . ArXiv preprint, abs/2312.08935

    2023 arXiv

  31. [39]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 b . https://openreview.net/pdf?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confere...

    2023

  32. [40]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning...

    2022

  33. [41]

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024 a . https://arxiv.org/abs/2408.00724 Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models . ArXiv preprint, abs/2408.00724

    2024 arXiv

  34. [42]

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024 b . Scaling inference computation: Compute-optimal inference for problem-solving with language models. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS, volume 24

    2024

  35. [43]

    Tong Xiao and Jingbo Zhu. 2025. Foundations of large language models. arXiv preprint arXiv:2501.09223

    2025 arXiv

  36. [44]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. https://arxiv.org/abs/2412.15115 Qwen2. 5 technical report . ArXiv preprint, abs/2412.15115

    2024 arXiv

  37. [45]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract-Conference.html Tree of thoughts: Deliberate problem solving with large languag...

    2023

  38. [46]

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. https://arxiv.org/abs/2503.14476 Dapo: An open-source llm reinforcement learning system at scale . ArXiv preprint, abs/2503.14476

    2025 arXiv

  39. [47]

    Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. 2024 a . https://arxiv.org/abs/2406.07394 Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b . ArXiv preprint, abs/2406.07394

    2024 arXiv

  40. [48]

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, and 1 others. 2025 a . Llama-berry: Pairwise optimization for olympiad-level mathematical reasoning via o1-like monte carlo tree search. In Proceedings of t...

    2025

  41. [49]

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2025 b . Generative verifiers: Reward modeling as next-token prediction. In The Thirteenth International Conference on Learning Representations

    2025

  42. [50]

    Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. 2025 c . https://arxiv.org/abs/2503.24235 What, how, where, and how well? a survey on test-time scaling in large language models . ArXiv preprint, abs/2503.24235

    2025 arXiv

  43. [51]

    Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. 2024 b . https://arxiv.org/abs/2404.17140 Small language models need strong verifiers to self-correct reasoning . ArXiv preprint, abs/2404.17140

    2024 arXiv

  44. [52]

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025 d . https://arxiv.org/abs/2501.07301 The lessons of developing process reward models in mathematical reasoning . ArXiv preprint, abs/2501.07301

    2025 arXiv

  45. [53]

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024. https://arxiv.org/abs/2411.14405 Marco-o1: Towards open reasoning models for open-ended solutions . ArXiv preprint, abs/2411.14405

    2024 arXiv

  46. [54]

    Tong Zheng, Lichang Chen, Simeng Han, R Thomas McCoy, and Heng Huang. 2025. https://arxiv.org/abs/2505.15817 Learning to reason via mixture-of-thought for logical reasoning . ArXiv preprint, abs/2505.15817

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.