Pith. sign in

REVIEW 4 major objections 4 minor 34 cited by

Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By allocating test-time computation optimally, a 1B LLM can beat a 405B LLM on MATH-500, and a 7B LLM can beat o1 and DeepSeek-R1 on both MATH-500 and AIME24.

desk verdict Useful empirical sweep of TTS methods, but the headline small-beats-large crossings are post hoc best-of-grid results on the test set, not a deployable strategy. read the letter →

arxiv 2502.06703 v1 pith:MF7YEAFR submitted 2025-02-10 cs.CL

Runze Liu , Junqi Gao , Jian Zhao , Kaiyan Zhang , Xiu Li , Biqing Qi , Wanli Ouyang , Bowen Zhou This is my paper · ORCID
classification cs.CL
keywords test-timescalingcompute-optimalinferenceprocessrewardmodelsmathematicalreasoningsmalllanguageefficiencybest-of-nsamplingbeamsearch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Test-time scaling (TTS) spends extra computation at inference to improve a language model's answers, and this paper asks how to spend it optimally and how far that can push small models. The authors argue that the best strategy—which search method, which process reward model, and how much compute to give each problem—depends strongly on the policy model and the problem's difficulty. Under their 'compute-optimal' recipe, they report that a 1B model exceeds a 405B model on MATH-500, a 0.5B model beats GPT-4o, and a 7B model outperforms o1 and DeepSeek-R1, while using far less inference compute. The practical stakes: if correct, small models plus smart search and process-reward scoring could replace frontier models on math reasoning at a small fraction of the cost.

What carries the argument

The central object is the compute-optimal TTS strategy, a rule that selects the scaling method, compute budget, and PRM for a given policy model and problem. The paper's version is reward-aware (Eq. 3), meaning the choice depends on the reward function, and it is evaluated empirically across three TTS methods: Best-of-N (sample N answers and score them), beam search (keep the top-N/M steps per depth according to the PRM), and Diverse Verifier Tree Search (run independent beam searches in subtrees). The other load-bearing piece is the difficulty criterion: instead of Pass@1 quantiles, the authors bin problems by absolute Pass@1 accuracy into easy (50–100%), medium (10–50%), and hard (0–10%) levels, arguing that quantiles are misleading when different policy models have very different baseline abilities.

What would settle it

Take a held-out set of competition math problems, fix a small policy (e.g., Llama-3.2-3B-Instruct) and a compute budget, and choose the strategy (method, PRM, budget) using only a separate validation split. If the best-on-validation configuration fails to beat the paired large model (e.g., Llama-3.1-405B-Instruct or GPT-4o with chain-of-thought) on the held-out test set, then the claimed size-crossing results do not generalize beyond the benchmark they were selected on.

Watch

Extended reading notes

Core claim

The paper's central claim is that compute-optimal test-time scaling is reward-aware: the best way to scale inference compute cannot be chosen from the policy model and budget alone, but must also account for the process reward model (PRM) that scores reasoning steps. Concretely, the authors define the reward-aware compute-optimal strategy in Eq. (3) as the argmax over strategies of the expected probability of the correct answer given the policy, the compute budget, the problem, and the reward function. Running that selection across Llama and Qwen policy models from 0.5B to 72B, seven PRMs, and three scaling methods (Best-of-N, beam search, and Diverse Verifier Tree Search) on MATH-500 and AIME24, they find that small models with the best configuration outperform much larger models: Llama-3.2-3B-Instruct beats Llama-3.1-405B-Instruct on both benchmarks, Llama-3.2-1B-Instruct beats the 405B model on MATH-500 at budget N=512, Qwen2.5-0.5B-Instruct beats GPT-4o, and DeepSeek-R1-Distill-Qwen-7B beats o1 and DeepSeek-R1. They also report that the optimal method shifts with model size—search-based methods win for small policies while Best-of-N wins for larger ones—and with difficulty, with BoN better on easy problems and beam search better on hard ones.

Load-bearing premise

The load-bearing premise is that the best configuration on the test benchmarks can be chosen without using the test answers; the paper reports the best method, PRM, and budget found on MATH-500 and AIME24 themselves, and does not give a selection rule that would work on unseen problems.

Editorial extensions

If this is right

  • If the compute-optimal strategy is followed, small policy models (0.5B–7B) can beat frontier models on MATH-500 and AIME24 while consuming less inference FLOPS, with total FLOPS reductions of 100x–1000x.
  • The optimal TTS method is not universal: search-based methods (beam search/DVTS) help small models most, while Best-of-N is best for large models, so practitioners must match the method to the policy.
  • PRM choice matters as much as the search method: PRMs trained on different policies generalize poorly, and PRM quality (measured by process supervision ability) is positively correlated with TTS performance.
  • TTS with external search outperforms several long-CoT training recipes (rStar-Math, Eurus-2, SimpleRL, Satori) on both benchmarks, but is less effective than distilling from a strong reasoning model like DeepSeek-R1.
  • TTS gains shrink as policy models get stronger; for small models compute-optimal TTS improves accuracy over CoT by up to 154.6% and can be 256x more efficient than majority voting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper leaves implicit is that a practical deployment needs a selection rule that chooses the strategy before seeing the test answers; the reported crossings are the best configuration on the test benchmarks, so a validation-based selection rule is the natural next experiment.
  • The length-bias findings suggest that comparing TTS methods by raw accuracy without fixing token budgets can conflate search quality with PRM preferences; a token-normalized comparison would sharpen the efficiency claims.
  • The weak-to-strong result (a 7B PRM supervising a 72B policy) points toward building automated supervisors that can pick methods and budgets per problem, possibly trained on a development split, rather than assuming one global strategy.
  • The same recipe may transfer to other multi-step tasks like coding, but only if step-level reward models there are strong enough; the paper's own AIME24 results show TTS gains are smaller on harder problems, so the transfer is not automatic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents a large empirical study of external test-time scaling (TTS) methods—Best-of-N, beam search, and Diverse Verifier Tree Search—across a wide grid of policy models (0.5B to 72B), seven process reward models, several scoring/voting schemes, and compute budgets. It proposes a "reward-aware compute-optimal TTS" framework, analyzes how the best TTS method depends on policy model, verifier, and problem difficulty, and reports that compute-optimal TTS lets small models outperform much larger models: e.g., Llama-3.2-1B over Llama-3.1-405B on MATH-500, Qwen2.5-0.5B over GPT-4o, and DeepSeek-R1-Distill-7B over o1 and DeepSeek-R1 on both benchmarks. The paper also claims large FLOPS savings.

Significance. If the headline claim were supported—that a deployable, compute-optimal TTS strategy allows small models to beat frontier models on these benchmarks—the result would be practically important for efficient inference. The descriptive component of the paper is also valuable: the sweep over policy models, PRMs, and budgets is broad, and the qualitative taxonomy of PRM failure modes in Appendix C (over-criticism, error neglect, localization bias, scoring bias) is a useful contribution. However, the central comparison is not currently established because the strategy is selected on the same test benchmarks used to report the gains, and the reported crossings are often within one or two problems on a 30-problem set. The paper needs an out-of-sample selection protocol and uncertainty quantification before the small-vs-large claims can be taken as evidence.

major comments (4)
  1. [§4 and §5.1, especially Table 3] The "compute-optimal TTS strategy" is selected on the test benchmarks themselves. Equations (2)-(3) define the strategy as an argmax involving the ground-truth answer y*(x), and Section 4 sweeps multiple policy models, seven PRMs, three TTS methods, budgets {4,16,64,256} (plus N=512 in one case), and multiple scoring/voting schemes; Table 3 then reports the best configuration per small model as the "compute-optimal" result. No validation split or selection rule is specified that would allow the strategy to be chosen without access to test labels. The reported MATH-500 and AIME24 numbers are therefore selected maxima over a large configuration grid, not out-of-sample predictions. The headline claims that a 1B model exceeds a 405B model, that a 0.5B model beats GPT-4o, and that a 7B model beats o1 require a validation-based selection protocol; as written, they are not supported by the experimental design.
  2. [§3.2 and §4.3] The difficulty bins used to allocate compute are defined by Pass@1 accuracy on the same test problems that are later used to evaluate the strategy. The paper states that problems are labeled easy/medium/hard using absolute Pass@1 thresholds, but Pass@1 is estimated on MATH-500 and AIME24 with oracle labels. This means the "optimal" allocation per problem uses information that a deployable strategy would not have at inference time. The authors need either a validation-based difficulty classifier or a measurable proxy for difficulty, and the selection of thresholds must be decoupled from the test set.
  3. [Table 3 and AIME24 sample size] Several claimed crossings are within sampling noise, and no error bars or significance tests are reported. AIME24 has 30 problems: the Qwen2.5-0.5B vs GPT-4o comparison (10.0 vs 9.3) is a gap of roughly one question, and DeepSeek-R1-Distill-7B vs o1 (83.3 vs 79.2) is a gap of roughly one or two questions; on MATH-500, the 7B-vs-o1 gap (95.2 vs 94.8) is two questions. Since Table 3 reports the best of many configurations, the expected maximum over noise can produce these margins even without a real advantage. The authors should report per-configuration results, bootstrap confidence intervals, and either a multiple-testing adjustment or a pre-registered selection rule.
  4. [§5.1, Table 4] The FLOPS efficiency claim does not appear to account for the compute of the process reward model. For example, the reported inference FLOPS for Llama-3.2-3B with compute-optimal TTS is 3.07×10^17, but if the strategy uses a 72B PRM to score many steps (as suggested by the sweep in Section 4), the verifier compute should be included in a fair comparison with a large-model CoT baseline. The claim of higher inference efficiency needs to specify exactly which components (policy, verifier, sampling, scoring) are included in the FLOPS estimate.
minor comments (4)
  1. [§4.4] There is a typo: "pervious experiments" should be "previous experiments."
  2. [§5.1, footnote 5] Using Qwen2.5-32B-Instruct to extract answers for Llama-3.2-1B because the model does not emit \boxed{...} introduces a non-identical evaluation pipeline for that row; the potential effect on the reported 66.2 and 72.2 numbers should be discussed or the extraction method should be applied uniformly.
  3. [§5.1, text near Table 3] The statement "we improve the result by 487.0% (23×→ 135×)" is unclear: the comparison base and the definition of the percentage improvement over prior work should be stated precisely.
  4. [Tables and figures] Several figures (e.g., Figures 10-11) are dense and difficult to read at print resolution; providing numeric tables in the appendix or a separate supplementary file would improve verifiability of the per-configuration results that underlie Table 3.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline 'compute-optimal TTS' crossings are the test-set argmax of Eq. (3), not an out-of-sample strategy.

  1. fitted input called prediction [Eq. (3), Section 3.1; Section 4.1 setup; Table 3, Section 5.1]
    "𝜃*𝑥,𝑦*(𝑥),ℛ(𝑁) = arg max𝜃(E𝑦∼Target(𝜃,𝑁,𝑥,ℛ)[1{𝑦=𝑦*(𝑥)}]) ... With the compute-optimal TTS strategy explored in Section 4, we conduct further experiments ... From the results in Table 3, we have the following observations: (1) Llama-3.2-3B-Instruct with the compute-optimal TTS strategy outperforms Llama-3.1-405B-Instruct on MATH-500 and AIME24"

    Eq. (3) defines the compute-optimal strategy as the argmax over configurations using the ground-truth answer y*(x). Section 4 then sweeps seven PRMs, three TTS methods, budgets {4,16,64,256} (plus N=512 for Llama-3.2-1B), and multiple scoring/voting schemes, and Section 5.1 reports the best per-policy configuration in Table 3 as 'compute-optimal TTS'. No validation split or selection rule for unseen problems is given, so the reported gains are the selected maximum of a large grid evaluated on the same MATH-500/AIME24 test sets used to pick the configuration. The headline capability claim is therefore statistically forced by the test-set selection rather than demonstrated as a deployable strategy.

  2. other [Section 3.2 and Section 4.3, Figures 8-9]
    "Specifically, we define three difficulty levels based on Pass@1 accuracy: easy (50%∼ 100%), medium (10%∼ 50%), and hard (0%∼ 10%). ... The results in Figure 8 and Figure 9 show that for small policy models ... BoN is better for easy problems, while beam search works better for harder problems."

    The difficulty labels used to decide which TTS method is 'optimal' are computed from Pass@1 accuracy on the very test problems whose final performance is then reported. A compute-optimal allocation intended to generalize cannot use y*(x) or test-set Pass@1 for new problems. Conditioning the method choice on oracle difficulty labels and then presenting the resulting accuracy as the 'compute-optimal TTS' result makes the optimality statement depend on the answer key by construction.

full rationale

The central quantitative claims in Table 3 are an empirical envelope rather than an out-of-sample prediction. Eq. (3) formalizes the compute-optimal strategy as an argmax using ground-truth y*(x); Section 4 sweeps a broad grid of PRMs, methods, budgets, and scoring/voting choices; and Table 3 reports the best entries from that grid on the same MATH-500 and AIME24 sets, with no validation split or pre-specified selection rule. The difficulty-level analysis similarly uses test-set Pass@1 accuracy as an oracle label. This makes the headline 'small models surpass large models with compute-optimal TTS' a post-hoc selected maximum, not a validated deployable strategy. The underlying measurements and baselines are not themselves circular, and there is no load-bearing self-citation chain, but the load-bearing inference from Table 3 to a general compute-optimal strategy reduces to selection on the test set, so partial circularity is present.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The 'reward-aware compute-optimal TTS strategy' is a re-framing of existing TTS selection, not a new object with independent evidence.

free parameters (4)
  • Difficulty thresholds for easy/medium/hard bins = easy: Pass@1 50%-100%, medium: 10%-50%, hard: 0%-10%
    Chosen by hand in Section 3.2 to replace quantile-based bins; no principled justification, but they affect which strategy is deemed compute-optimal per difficulty.
  • Compute budget grid = N in {4, 16, 64, 256} (and 512 for Llama-3.2-1B)
    The grid over which the best TTS result is selected; the headline numbers are maxima over this grid.
  • Beam width and temperature = beam width 4; temperature 0.7 for TTS, 0.0 for CoT
    Fixed hyperparameters, not swept; they shape the search results.
  • Selected best strategy per model = Not reported as a table; implicitly the best method/PRM/budget on the test set
    This is the central free choice behind Table 3; the paper does not define a validation-based selection rule.
assumptions (3)
  • domain assumption PRM scores provide a usable reward signal for guiding solution search
    Assumed throughout Sections 4 and 5; Section 4.4 documents that PRMs are biased, which undermines but does not remove the assumption.
  • domain assumption Pass@1 accuracy on MATH-500/AIME24 is a stable measure of problem difficulty
    Used to bin problems in Section 3.2; assumes the measured accuracy is not noise, despite small datasets.
  • domain assumption The newline-delimited step format is a valid unit for process supervision
    Step division by '\n\n' is used for beam search and DVTS scoring, following prior work (Xiong et al., Zhang et al.).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling." pith.science (2026). https://pith.science/paper/MF7YEAFR

@misc{pith2026250206703,
  author       = {Pith},
  title        = {Pith review of: Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MF7YEAFR}},
  note         = {Machine review of arXiv:2502.06703}
}
read the original abstract

Test-Time Scaling (TTS) is an important method for improving the performance of Large Language Models (LLMs) by using additional computation during the inference phase. However, current studies do not systematically analyze how policy models, Process Reward Models (PRMs), and problem difficulty influence TTS. This lack of analysis limits the understanding and practical use of TTS methods. In this paper, we focus on two core questions: (1) What is the optimal approach to scale test-time computation across different policy models, PRMs, and problem difficulty levels? (2) To what extent can extended computation improve the performance of LLMs on complex tasks, and can smaller language models outperform larger ones through this approach? Through comprehensive experiments on MATH-500 and challenging AIME24 tasks, we have the following observations: (1) The compute-optimal TTS strategy is highly dependent on the choice of policy model, PRM, and problem difficulty. (2) With our compute-optimal TTS strategy, extremely small policy models can outperform larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500. Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher inference efficiency. These findings show the significance of adapting TTS strategies to the specific characteristics of each task and model and indicate that TTS is a promising approach for enhancing the reasoning abilities of LLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 34 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs' Reasoning

    cs.CL 2025-05 conditional novelty 7.0 of 10

    A checkpoint-based search and candidate augmentation method improves small LLM mathematical reasoning accuracy over existing test-time scaling baselines.

  2. Hidden Language Consistency Phenomena in Reasoning LLMs

    cs.CL 2026-08 conditional novelty 6.0 of 10

    Reasoning models often stop using the requested language as problems get harder, and this language breakdown can make accuracy look better than it is.

  3. LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Cross-model answer agreement is a free Best-of-N verifier that matches trained process reward models and is predicted by a parameter-free law from three panel statistics.

  4. Inference-Time Search Using Side Information for Diffusion-Based Image Reconstruction

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Injecting side information via inference-time particle search (GS/RFJS) improves diffusion-based inverse problem reconstructions across inpainting, super-resolution, deblurring, and MRI tasks in a training-free, plug-...

  5. SSRL: Self-Search Reinforcement Learning

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    SSRL, a training pipeline that uses an LLM's own repeated sampling as a search environment for RL, improves question answering without external tools and transfers to real search engines.

  6. Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A language model fine-tuned on knowledge-graph-path reasoning tasks (QwQ-Med-3) beats strong baselines on a same-style benchmark but shows mixed gains on external medical QA tests.

  7. Test-Time Scaling with Reflective Generative Model

    cs.LG 2025-07 conditional novelty 6.0 of 10

    MetaStone-S1 combines a shared policy and self-supervised process reward head to select high-quality reasoning traces, reaching o3-mini-level scores at 32B parameters.

  8. EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Keeping only the first and last parts of chain-of-thought training traces preserves math reasoning accuracy on MATH500 at half the sequence length, but not on harder benchmarks.

  9. PixelThink: Towards Efficient Chain-of-Pixel Reasoning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A soft token-budget penalty driven by task difficulty and model confidence roughly halves the reasoning tokens used by segmentation MLLMs while slightly improving mask accuracy on ReasonSeg-derived benchmarks.

  10. Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A hierarchical process reward model that classifies math and consistency errors before scoring steps achieves the best PRMBench result among discriminative PRMs and improves reward-guided math search.

  11. Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Guided by Gut lets an LLM guide its own search using token-level confidence and step novelty, with a RL fine-tuning step to calibrate that confidence, achieving PRM-level accuracy at far lower compute and memory.

  12. Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

    cs.AI 2025-05 conditional novelty 6.0 of 10

    RAL lets an LLM agent learn from its own play by proposing strategies, validating them in one-step state transitions, and retrieving condensed experiences, improving StarCraft II decision-making without any model training.

  13. Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A step-level reward model trained on expert-designed synthetic clinical errors detects injected note errors with 98.8% accuracy and selects physician-preferred notes with 56.2% accuracy.

  14. Test-Time Scaling for Small VLMs on Multilingual Visual MCQ

    cs.CL 2026-07 accept novelty 5.5 of 10

    On EXAMS-V, token budget and parseability dominate chain count, PRM-guided search, and selectors for small VLMs; the policy model itself yields the largest gain.

  15. Proxy Exploration and Reusable Guidance: A Modular LLM Post-Training Paradigm via Proxy-Guided Update Signals

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Relative policy-improvement signals from a weak proxy model, after simple calibration, can be transferred to improve stronger primary LLMs without re-exploring on the primary.

  16. ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanism

    cs.LG 2025-08 conditional novelty 5.0 of 10

    ETTRL improves test-time RL for LLMs by forking rollouts at high-entropy tokens and reshaping advantages with a relative entropy bonus, reporting large AIME 2024 gains at lower token cost.

  17. Causal Reward Adjustment: Mitigating Reward Hacking in External Reasoning via Backdoor Correction

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    CRA trains sparse autoencoders on PRM activations and applies backdoor adjustment to estimate true rewards, reducing reward hacking in math reasoning.

  18. Thinking Isn't an Illusion: Overcoming the Limitations of Reasoning Models via Tool Augmentations

    cs.AI 2025-07 conditional novelty 5.0 of 10

    With Python tool augmentation, DeepSeek-R1 and Qwen 3 Thinking outperform their non-thinking counterparts on River Crossing and Blocks World, but the advantage does not appear on Hanoi or Checker Jumping.

  19. Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs

    cs.AI 2025-07 conditional novelty 5.0 of 10

    The survey's L1/L2 taxonomy and benchmark show that current reasoning models waste compute on easy problems and underthink hard ones, motivating more adaptive inference.

  20. Reasoning in machine vision by learning fast and slow thinking

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A dual-process vision system improves segmentation accuracy by spending more inference-time compute, using a fast predictor and a slow self-play refiner, reporting gains on cancer localisation with only 8-16 labels.

  21. Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Ctrl-Z Sampling improves text-to-image outputs by adaptively rolling back and re-exploring when a reward model flags a quality plateau, at roughly 3 to 9 times the usual compute.

  22. DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Sampling

    cs.CL 2025-06 conditional novelty 5.0 of 10

    DynScaling improves verifier-free inference-time scaling by merging parallel and sequential sampling and allocating budget across queries with a UCB-based uncertainty rule.

  23. Scaling Test-time Compute for LLM Agents

    cs.AI 2025-06 conditional novelty 5.0 of 10

    On the GAIA benchmark, Best-of-N sampling with list-wise answer selection gives the largest agent gains, and selective (score-triggered) reflection beats reflection at every step.

  24. CyberV: Cybernetics for Test-time Scaling in Video Understanding

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A training-free test-time feedback loop, using attention drift to select key frames, improves video MLLM accuracy, with the largest gains on knowledge-heavy VideoMMMU.

  25. Can Past Experience Accelerate LLM Reasoning?

    cs.LG 2025-05 conditional novelty 5.0 of 10

    LLMs can cut reasoning compute by up to 56% on similar questions when memory of past correct solutions is combined with adaptive early stopping, though the benchmark is small and synthetic.

  26. Faster and Better LLMs via Latency-Aware Test-Time Scaling

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Test-time scaling should be tuned for wall-clock latency rather than token count; branches plus speculative decoding win under time budgets.

  27. Multilingual Test-Time Scaling via Initial Thought Transfer

    cs.CL 2025-05 reject novelty 5.0 of 10

    MITT, a prefix-tuning method for multilingual test-time scaling, is evaluated on questions whose English reasoning was used for training, confounding the reported gains.

  28. SLOT: Sample-specific Language Model Optimization at Test-time

    cs.CL 2025-05 conditional novelty 5.0 of 10

    SLOT adapts an LLM to each prompt by optimizing a lightweight final-layer vector to minimize prompt loss, boosting benchmark reasoning accuracy by a few points.

  29. A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

    cs.CR 2025-04 conditional novelty 5.0 of 10

    A large collaborative survey organizes LLM and LLM-agent safety issues into a full-stack lifecycle framework from data preparation to deployment.

  30. Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals

    cs.AI 2025-04 conditional novelty 5.0 of 10

    General-purpose LLMs show a 'reasoning floor' that extra inference compute does not overcome, simple majority voting beats expensive methods on reasoning models, and concise answers with fewer linguistic markers tend ...

  31. Interpretable Adaptive Sampling for LLM Test-Time Scaling

    cs.AI 2026-08 reject novelty 4.0 of 10

    A fuzzy controller that allocates a per-prompt sampling budget keeps LLM accuracy near a fixed full-budget baseline while reducing the average number of candidate answers on some datasets.

  32. A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law

    cs.AI 2025-05 conditional novelty 3.0 of 10

    A structured survey of reasoning LLMs that use reinforcement learning and inference-time scaling to achieve slow, deliberate thought, organized into three method categories.

  33. Generative AI Act II: Test Time Scaling Drives Cognition Engineering

    cs.CL 2025-04 conditional novelty 3.0 of 10

    Test-time scaling techniques such as long chain-of-thought, tree search, and self-correction define the paper's 'cognition engineering' paradigm, which it surveys, taxonomizes, and tutorials.

  34. Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models

    cs.CL 2025-05 conditional novelty 2.0 of 10

    A survey-style position paper claims that reinforcement fine-tuning powers reasoning in multimodal LLMs, summarizing over a hundred recent works and proposing five future research directions.

Reference graph

Works this paper leans on

77 extracted references · 31 canonical work pages · cited by 34 Pith papers

  1. [1]

    Aime 2024, 2024

    AI-MO . Aime 2024, 2024. URL https://huggingface.co/datasets/AI-MO/aimo-validation-aime

    work page 2024

  2. [2]

    Introducing Claude , 2023

    Anthropic. Introducing Claude , 2023. URL https://www.anthropic.com/index/introducing-claude/

    work page 2023

  3. [3]

    Jiang, Jia Deng, Stella Biderman, and Sean Welleck

    Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen Marcus McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics. In International Conference on Learning Representations (ICLR), 2024. URL https://openreview.net/forum?id=4WnqRR915j

    work page 2024

  4. [4]

    Scaling test-time compute with open models, 2024

    Edward Beeching, Lewis Tunstall, and Sasha Rush. Scaling test-time compute with open models, 2024. URL https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute

    2024

  5. [5]

    Large language monkeys: Scaling inference compute with repeated sampling

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R \'e , and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024

    arXiv 2024

  6. [6]

    Alphamath almost zero: Process supervision without process

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: Process supervision without process. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https://openreview.net/forum?id=VaXnxQ3UKo

    work page 2024

  7. [7]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research (TMLR), 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=YfZ4ZPt8zd

    work page 2023

  8. [8]

    Process reinforcement through implicit rewards

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, and Ning Ding. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456, 2025

    arXiv 2025

Show all 77 references
  1. [9]

    DeepSeek-AI , Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei ...

    2025 arXiv

  2. [10]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  3. [11]

    PAL : Program-aided language models

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL : Program-aided language models. In International Conference on Machine Learning (ICML), volume 202, pages 10764--10799, 2023

    2023

  4. [12]

    To RA : A tool-integrated reasoning agent for mathematical problem solving

    Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. To RA : A tool-integrated reasoning agent for mathematical problem solving. In International Conference on Learning Representations (ICLR), 2024. URL https://openreview.net/...

    2024

  5. [13]

    rStar-Math : Small llms can master math reasoning with self-evolved deep thinking

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rStar-Math : Small llms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04519, 2025

    2025 arXiv

  6. [14]

    Reinforced self-training (rest) for language modeling

    Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998, 2023

    2023 arXiv

  7. [15]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021...

    2021

  8. [16]

    V-star: Training verifiers for self-taught reasoners

    Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for self-taught reasoners. arXiv preprint arXiv:2402.06457, 2024

    2024 arXiv

  9. [17]

    O1 replication journey--part 2: Surpassing o1-preview through simple distillation, big progress or bitter lesson? arXiv preprint arXiv:2411.16489, 2024

    Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. O1 replication journey--part 2: Surpassing o1-preview through simple distillation, big progress or bitter lesson? arXiv preprint arXiv:2411.16489, 2024

    2024 arXiv

  10. [18]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  11. [19]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  12. [20]

    MindStar : Enhancing math reasoning in pre-trained llms at inference time

    Jikun Kang, Xin Zhe Li, Xi Chen, Amirreza Kazemi, Qianyi Sun, Boxing Chen, Dong Li, Xu He, Quan He, Feng Wen, et al. MindStar : Enhancing math reasoning in pre-trained llms at inference time. arXiv preprint arXiv:2405.16265, 2024

    2024 arXiv

  13. [21]

    ARGS : Alignment as reward-guided search

    Maxim Khanov, Jirayu Burapacheep, and Yixuan Li. ARGS : Alignment as reward-guided search. In International Conference on Learning Representations (ICLR), 2024. URL https://openreview.net/forum?id=shgx0eqdw6

    2024

  14. [22]

    k0-math, November 2024

    Kimi . k0-math, November 2024. URL https://kimi.moonshot.cn/

    2024

  15. [23]

    Kimi k1.5: Scaling reinforcement learning with llms

    Kimi Team , Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1.5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  16. [24]

    Training language models to self-correct via reinforcement learning

    Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917, 2024

    2024 arXiv

  17. [25]

    CoMAT : Chain of mathematically annotated thought improves mathematical reasoning

    Joshua Ong Jun Leang, Aryo Pradipta Gema, and Shay B Cohen. CoMAT : Chain of mathematically annotated thought improves mathematical reasoning. arXiv preprint arXiv:2410.10336, 2024

    2024

  18. [26]

    Process reward model with q-value rankings

    Wendi Li and Yixuan Li. Process reward model with q-value rankings. arXiv preprint arXiv:2410.11287, 2024

    2024 arXiv

  19. [27]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In International Conference on Learning Representations (ICLR), 2024. URL https://openreview.net/forum?i...

    2024

  20. [28]

    Autopsv: Automated process-supervised verifier

    Jianqiao Lu, Zhiyang Dou, WANG Hongru, Zeyu Cao, Jianbo Dai, Yunlong Feng, and Zhijiang Guo. Autopsv: Automated process-supervised verifier. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  21. [29]

    Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583, 2023

    2023 arXiv

  22. [30]

    Improve mathematical reasoning in language models by automated process supervision

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406.06592, 2024

    2024 arXiv

  23. [31]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

    2023

  24. [32]

    Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation

    Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation. arXiv preprint arXiv:2410.02725, 2024

    2024 arXiv

  25. [33]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  26. [34]

    Learning to reason with llms, 2024

    OpenAI. Learning to reason with llms, 2024. URL https://openai.com/index/learning-to-reason-with-llms/

    2024

  27. [35]

    O1 replication journey: A strategic progress report--part 1

    Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. O1 replication journey: A strategic progress report--part 1. arXiv preprint arXiv:2410.18982, 2024

    2024 arXiv

  28. [36]

    Recursive introspection: Teaching language model agents how to self-improve

    Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching language model agents how to self-improve. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https://openreview.net/forum?id=DRC9pZwBwR

    2024

  29. [37]

    Qwq: Reflect deeply on the boundaries of the unknown, November 2024

    Qwen Team . Qwq: Reflect deeply on the boundaries of the unknown, November 2024. URL https://qwenlm.github.io/blog/qwq-32b-preview/

    2024

  30. [38]

    Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold

    Amrith Setlur, Saurabh Garg, Xinyang Geng, Naman Garg, Virginia Smith, and Aviral Kumar. Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold. arXiv preprint arXiv:2406.14532, 2024 a

    2024 arXiv

  31. [39]

    Rewarding progress: Scaling automated process verifiers for llm reasoning

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146, 2024 b

    arXiv

  32. [40]

    DeepSeekMath : Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. DeepSeekMath : Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  33. [41]

    Satori: Reinforcement learning with chain-of-action-thought enhances llm reasoning via autoregressive search

    Maohao Shen, Guangtao Zeng, Zhenting Qi, Zhang-Wei Hong, Zhenfang Chen, Wei Lu, Gregory Wornell, Subhro Das, David Cox, and Chuang Gan. Satori: Reinforcement learning with chain-of-action-thought enhances llm reasoning via autoregressive search. arXiv preprint arXiv:2502.02508, 2025

    2025 arXiv

  34. [42]

    Skywork-o1, November 2024

    Skywork . Skywork-o1, November 2024. URL https://www.tiangong.cn/

    2024

  35. [43]

    Skywork-o1 open series

    Skywork o1 Team . Skywork-o1 open series. https://huggingface.co/Skywork, November 2024. URL https://huggingface.co/Skywork

    2024

  36. [44]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  37. [45]

    Prmbench: A fine-grained and challenging benchmark for process-level reward models

    Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. Prmbench: A fine-grained and challenging benchmark for process-level reward models. arXiv preprint arXiv:2501.03124, 2025

    2025 arXiv

  38. [46]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018

    2018

  39. [47]

    M ath S cale: Scaling instruction tuning for mathematical reasoning

    Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. M ath S cale: Scaling instruction tuning for mathematical reasoning. In International Conference on Machine Learning (ICML), volume 235, pages 47885--47900, 2024

    2024

  40. [48]

    DART -math: Difficulty-aware rejection tuning for mathematical problem-solving

    Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. DART -math: Difficulty-aware rejection tuning for mathematical problem-solving. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https://openreview.net/forum?id=zLU21oQjD5

    2024

  41. [49]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  42. [50]

    Reft: Reasoning with reinforced fine-tuning

    Luong Trung, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7601--7614, 2024

    2024

  43. [51]

    Solving math word problems with process-and outcome-based feedback

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022

    2022 arXiv

  44. [52]

    A lpha Z ero-like tree-search can guide large language model decoding and training

    Ziyu Wan, Xidong Feng, Muning Wen, Stephen Marcus Mcaleer, Ying Wen, Weinan Zhang, and Jun Wang. A lpha Z ero-like tree-search can guide large language model decoding and training. In International Conference on Machine Learning (ICML), volume 235, pages 49890--49920, 2024

    2024

  45. [53]

    Openr: An open source framework for advanced reasoning with large language models

    Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M Ni, et al. Openr: An open source framework for advanced reasoning with large language models. arXiv preprint arXiv:2410.09671, 2024 a

    arXiv

  46. [54]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol...

    2024

  47. [55]

    Scaling inference-time search with vision value model for improved visual comprehension

    Xiyao Wang, Zhengyuan Yang, Linjie Li, Hongjin Lu, Yuancheng Xu, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Scaling inference-time search with vision value model for improved visual comprehension. arXiv preprint arXiv:2412.03704, 2024 c

    2024 arXiv

  48. [56]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023. URL https://openrev...

    2023

  49. [57]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in neural information processing systems (NeurIPS), volume 35, pages 24824--24837, 2022

    2022

  50. [58]

    Large language models are better reasoners with self-verification

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550--2575, 2023

    2023

  51. [59]

    Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408.00724, 2024

    2024 arXiv

  52. [60]

    Self-evaluation guided beam search for reasoning

    Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min-Yen Kan, Junxian He, and Michael Xie. Self-evaluation guided beam search for reasoning. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 41618--41650, 2023

    2023

  53. [61]

    An implementation of generative prm

    Wei Xiong, Hanning Zhang, Nan Jiang, and Tong Zhang. An implementation of generative prm. https://github.com/RLHFlow/RLHF-Reward-Modeling, 2024

    2024

  54. [62]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng...

    2024 arXiv

  55. [63]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  56. [64]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024 arXiv

  57. [65]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 11809--11822, 2023

    2023

  58. [66]

    MetaMath : Bootstrap your own mathematical questions for large language models

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. MetaMath : Bootstrap your own mathematical questions for large language models. In International Conference on Learning Representations (ICLR), 2024...

    2024

  59. [67]

    Free process rewards without process labels

    Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels. arXiv preprint arXiv:2412.01981, 2024

    2024 arXiv

  60. [68]

    STaR : Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. STaR : Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 15476--15488, 2022

    2022

  61. [69]

    Quiet- ST ar: Language models can teach themselves to think before speaking

    Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Goodman. Quiet- ST ar: Language models can teach themselves to think before speaking. In Conference on Language Modeling (COLM), 2024. URL https://openreview.net/forum?id=oRXPiSOGH9

    2024

  62. [70]

    Skywork-Math : Data scaling laws for mathematical reasoning in large language models--the story goes on

    Liang Zeng, Liangjun Zhong, Liang Zhao, Tianwen Wei, Liu Yang, Jujie He, Cheng Cheng, Rui Hu, Yang Liu, Shuicheng Yan, et al. Skywork-Math : Data scaling laws for mathematical reasoning in large language models--the story goes on. arXiv preprint arXiv:2407.08348, 2024

    2024 arXiv

  63. [71]

    7b model and 8k examples: Emerging reasoning with reinforcement learning is both effective and efficient

    Weihao Zeng, Yuzhen Huang, Wei Liu, Keqing He, Qian Liu, Zejun Ma, and Junxian He. 7b model and 8k examples: Emerging reasoning with reinforcement learning is both effective and efficient. https://hkust-nlp.notion.site/simplerl-reason, 2025. Notion Blog

    2025

  64. [72]

    Re ST - MCTS *: LLM self-training via process reward guided tree search

    Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Re ST - MCTS *: LLM self-training via process reward guided tree search. In Advances in Neural Information Processing Systems (NeurIPS), 2024 a . URL https://openreview.net/forum?id=8rcFOqEud5

    2024

  65. [73]

    Entropy-regularized process reward model

    Hanning Zhang, Pengcheng Wang, Shizhe Diao, Yong Lin, Rui Pan, Hanze Dong, Dylan Zhang, Pavlo Molchanov, and Tong Zhang. Entropy-regularized process reward model. arXiv preprint arXiv:2412.11006, 2024 b

    2024

  66. [74]

    The lessons of developing process reward models in mathematical reasoning

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025

    2025 arXiv

  67. [75]

    Marco-o1: Towards open reasoning models for open-ended solutions

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-o1: Towards open reasoning models for open-ended solutions. arXiv preprint arXiv:2411.14405, 2024

    2024 arXiv

  68. [76]

    Processbench: Identifying process errors in mathematical reasoning

    Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559, 2024

    2024 arXiv

  69. [77]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 465...

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.