Pith. sign in

REVIEW 4 major objections 5 minor 27 references

A process reward model trained only from terminal outcomes can steer multi-agent search, lifting GSM8K exact match from 43.9% to 74.6% at matched compute.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 07:50 UTC pith:CCQFFXHG

load-bearing objection The clean SBS result is real but the MCTS headline is confounded; worth a serious referee, but only after major revision. the 4 major comments →

arxiv 2510.24803 v3 pith:CCQFFXHG submitted 2025-10-28 cs.MA cs.AI

MASPRM: Multi-Agent System Process Reward Model

Milad Yazdani , Mahdi Mostajabdaveh , Zirui Zhou , Ying Xiong This is my paper
classification cs.MA cs.AI
keywords process reward modelmulti-agent systemMonte Carlo tree searchinference-time searchstep-level beam searchexact matchzero-shot transfervalue-guided decoding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multi-agent systems (MAS) usually solve problems by having several language-model agents talk to each other, but deciding which agent's message actually advanced the solution is hard. MASPRM is a process reward model that learns to score each intermediate state—each message and its routing—from only the final success or failure of a whole dialogue, using Monte Carlo tree search rollouts to propagate that terminal outcome back to every visited step. The paper's central claim is that these per-agent, per-step value estimates are better aligned with multi-step multi-agent decision-making than token likelihoods or terminal-only outcome scoring. In experiments at matched compute, MASPRM-guided beam search and MCTS beat policy-likelihood baselines and outcome-only verifiers on GSM8K and MATH, and combining the process signal with a terminal outcome model gives the largest gains: up to 30.7 exact-match points over a single straight-through MAS pass on GSM8K and 22.9 points on MATH. A MASPRM trained on GSM8K transfers zero-shot to MATH, adding 8.4 exact-match points at the same budget, suggesting the learned progress signal captures reusable structure.

Core claim

The core discovery is that process-level value estimates for inter-agent states can be learned purely from terminal rewards and that using these estimates to guide search substantially improves multi-agent reasoning. MASPRM's value head V_phi maps a state—the question, the partial transcript, and the identity of the next agent—to a predicted expected terminal reward. Supervision is search-generated: multi-agent MCTS is run with only outcome rewards, and for every visited edge (s,a) the child state s' is labeled with the Monte Carlo estimate Qhat(s,a); V_phi is then fit with a bounded Huber loss. At inference the same head drives step-level beam search and MCTS, initializing leaf values and b

What carries the argument

The central mechanism is the process value head V_phi, which scores any intermediate inter-agent state as a predicted expected terminal reward. It is trained on targets y_proc(s') = Qhat(s,a) generated by MAS-MCTS: terminal rewards are backed up along visited paths, and each edge's child receives the mean return as its regression target. The head is a bounded (tanh) regressor trained with Huber loss on token embeddings, with no human step labels. At inference it plays two roles: it initializes leaf values during expansion (virtual visits) and, in the default lambda=0 setting, serves as the backpropagation target, turning a long dialogue into a sequence of value-estimated agent-level actions.

Load-bearing premise

The load-bearing premise is that V_phi accurately estimates expected terminal reward for every state the search can reach—because inference uses V_phi both to initialize leaf values and as the backpropagation target (lambda=0), any pointwise miscalibration on states outside the N=40-rollout training distribution will be compounded rather than corrected, so the reported gains rest on the value head's accuracy, not merely its ranking ability.

What would settle it

Run the same MASPRM-guided MCTS with V_phi replaced by a constant mid-range value (e.g., 0) at both leaf initialization and backpropagation, keeping the expansion policy and the ORM identical; if the exact-match gain over policy-likelihood MCTS persists, then the search tree (UCT + sampling) rather than the value predictions is producing the gains. Alternatively, measure the correlation between V_phi's scores on intermediate states and the actual terminal rewards on held-out rollouts; a near-zero correlation, especially for states after the Verifier's final message, would indicate the bootstra

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Step-level beam search scored by MASPRM improves over policy-likelihood beam search at matched agent calls: +2.2 EM on GSM8K and +4.9 on MATH at the 1/5 budget.
  • MASPRM-guided MCTS outperforms policy-likelihood MCTS at matched compute: 72.4% vs 60.8% EM on GSM8K and 47.3% vs 35.8% on MATH, using fewer agent calls and tokens.
  • Process guidance and terminal outcome scoring are complementary: combining MCTS (MASPRM) with a terminal ORM yields 74.6% EM on GSM8K and 48.0% on MATH, the best results in the paper.
  • A MASPRM trained on GSM8K transfers zero-shot to MATH, adding 8.4 EM points over policy-only MCTS at matched compute, indicating the progress signal captures reusable structure.
  • Because MASPRM only needs terminal rewards for training, it can be added to existing multi-agent systems as a drop-in controller without changing base policies or collecting step-level human annotations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: Because the value head is trained on only 40 rollouts per problem from one policy, its pointwise accuracy on out-of-distribution states—especially terminal Verifier messages—is unmeasured; a direct calibration test (e.g., Pearson R between V_phi and realized terminal reward on held-out rollouts) would show how much of the reported gain is due to ranking versus pointwise accuracy.
  • Editorial: The search budget is matched on agent calls, but MASPRM adds its own calls (MC); a natural extension is to treat these value-model evaluations as a second budget dimension and optimize the trade-off, or to distill V_phi into a cheaper surrogate so the controller adds negligible overhead.
  • Editorial: The per-agent scores could serve as dense rewards for multi-agent reinforcement learning, converting sparse terminal feedback into stepwise credit assignment; the paper lists this as future work, but the training recipe already produces exactly the signal needed for that use.
  • Editorial: The zero-shot transfer from GSM8K to MATH suggests the learned notion of 'progress' is partly domain-general; testing the same frozen head on tool-use, code, or planning domains with a similar communication graph would reveal whether the signal is tied to arithmetic text or to general inter-agent coordination.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces MASPRM, a process reward model for multi-agent systems that assigns per-agent, per-step values to intermediate inter-agent transcripts. MASPRM is trained from multi-agent MCTS rollouts using only terminal outcome rewards, without human step-level annotations. At inference, it guides step-level beam search (SBS) and MCTS over agent rollouts. Experiments on GSM8K and MATH with a Qwen2.5-1.5B-based multi-agent system report large exact-match gains: e.g., MCTS+MASPRM+ORM reaches 74.6% EM on GSM8K versus 43.9% for greedy decoding, and MCTS (MASPRM) outperforms MCTS (policy likelihood) by +11.6 and +11.5 points on GSM8K and MATH, respectively. The paper also reports zero-shot transfer of a GSM8K-trained MASPRM to MATH.

Significance. If the empirical claims hold, MASPRM would be a useful contribution: it provides a practical recipe for training process-level value models for multi-agent systems from outcome-only rewards, with detailed budget accounting (agent calls vs. model calls vs. tokens), a plug-in architecture, pseudocode, and a public code link. The SBS comparisons in Tables 1 and 2 provide some clean evidence that MASPRM scoring improves over policy-likelihood scoring at matched budgets. However, the central MCTS comparison is confounded, as detailed below, and the abstract claims results on MMLU, LogiQA, and 7B models that do not appear in the manuscript. The value-head calibration evidence promised in Appendix B.5 is not reported, and all stochastic experiments appear to be single-seed. The core idea is promising, but the current evidence is not yet sufficient to support the headline claims as stated.

major comments (4)
  1. [§4.2, Eq. (3), Algorithm 2, Tables 1–2] The central RQ1 comparison, MCTS (MASPRM) vs. MCTS (policy likelihood), is confounded. Algorithm 2 always initializes virtual visits with V_phi (line 12) and, with the default λ=0, evaluates every non-ORM leaf as v=V_phi(s) (Eq. 3, Algorithm 2 line 18). The manuscript only says f_INIT may be ψ_pol 'when applicable,' but provides no algorithm variant in which the policy-likelihood MCTS actually uses ψ_pol for leaf evaluation or backpropagation. Thus the reported +11.6/+11.5 EM gains may stem primarily from the different first-visit initializer, not from process-level versus token-level scoring during search. Please rerun MCTS (policy likelihood) with leaf values and backpropagation driven by ψ_pol (and no V_phi), or explicitly specify the exact variant used; otherwise the headline claim in §6 (Finding 1) is not supported by the MCTS experiments.
  2. [Abstract vs. §5] The abstract states that the method is evaluated on GSM8K, MATH, MMLU, and LogiQA, reports results at both 1.5B and 7B, and gives aggregate gains across all four benchmarks (e.g., '+4.1 to +14.5 at 7B'). The main text and appendices, however, report experiments only on GSM8K and MATH, with a single 1.5B MASPRM backbone. There are no MMLU or LogiQA tables, no 7B experiments, and no corresponding comparisons. This is a material mismatch between the claimed evaluation scope and the actual content. The authors should either add the missing experiments or revise the abstract and claims to match what is actually reported.
  3. [§4.1, §5.1, App. B.5] Inference-time MCTS uses V_phi in triple roles: initializer for first-visit nodes, leaf evaluator (λ=0), and backpropagation target. The correctness of this bootstrap therefore depends on V_phi being a pointwise accurate estimate of expected terminal reward, not merely a good ranker. Appendix B.5 lists Pearson, RMSE, MAE, R2, and bias as evaluation metrics, but the paper reports none of these numbers. Please report these calibration metrics on a held-out split, together with an analysis of target noise (e.g., variance over the N=40 rollouts and Cmax=3 candidate set). Without this, the risk that V_phi is miscalibrated on off-policy states (including terminal Verifier messages and search-pruned branches) cannot be assessed, and the 'pure bootstrap' search results are not fully interpretable.
  4. [§5.1, Tables 1–3] All stochastic decoding and search experiments use temperature 0.7 and top-p 0.95, yet the paper reports a single run: footnote 1 says 'We use a random seed and report single run metrics.' Differences as large as +11.6 EM could still be within sampling noise, especially for a 1,319-problem test set, and no confidence intervals or significance tests are provided. Please report results over multiple seeds (at least 3–5) with means and standard deviations, or justify why the reported single-run numbers are stable.
minor comments (5)
  1. [§5.1, footnote 1] The paper refers to 'a random seed' but never specifies its value or whether the same seed is used across all methods. Please state the exact seed(s) and the sharing protocol.
  2. [Tables 1–2] The text says comparisons are at 'matched AC budgets,' but MCTS (MASPRM) uses 89 agent calls on GSM8K while MCTS (policy likelihood) uses 96. The budget-matching convention should be clarified (e.g., same number of simulations N with different effective calls due to early termination).
  3. [App. B.6, Algorithm 2] Appendix B.6 states 'on dead ends, back up the current MASPRM leaf value,' but Algorithm 2 does not describe dead-end handling or the corresponding backpropagation rule. This should be made explicit for reproducibility.
  4. [§4.2] The phrase 'when applicable' in the discussion of f_INIT is ambiguous. Please specify exactly which experiments use ψ_pol as initializer and whether these are the same experiments labeled 'MCTS (policy likelihood)' in the tables.
  5. [Figure 1] The token-accuracy frontier figure would benefit from error bars or shaded regions reflecting the single-seed uncertainty; as presented, some point differences may be visually misleading.

Circularity Check

0 steps flagged

No significant circularity; central derivation is externally validated on held-out test sets. A non-circular MCTS-baseline confound is noted.

full rationale

The central derivation is not circular. MASPRM's regression targets yproc(s') = Qhat(s,a) are Monte Carlo backups of terminal rewards R(s_T) from MAS-MCTS rollouts on training questions only; final reported exact-match accuracies are measured on held-out test sets (GSM8K test 1,319; MATH test 2,000) and are never used as training targets. The value head is additionally checked by zero-shot transfer from GSM8K to MATH (+8.4 EM), an external, non-fitted comparison. The inference-time bootstrap (lambda=0, v_leaf=V_phi) is a standard self-consistent value update, not a definitional equivalence: downstream accuracy is not guaranteed by construction from the training labels. There are no load-bearing self-citations or imported uniqueness theorems. The only notable issue is an experimental confound rather than circularity: under Eq. (3) and Algorithm 2, leaf evaluation and backpropagation use V_phi for every non-ORM inference MCTS run, including the 'MCTS (policy likelihood)' baseline, so that MCTS comparison mainly varies the initializer rather than the scorer. The SBS comparisons (V_phi vs. psi_pol) do cleanly vary the scorer and independently support the process-value claim. This confound affects interpretation but does not make the reported gains equivalent to the training inputs by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

MASPRM introduces a learned function (a value head) but no new physical or theoretical entity. The free parameters are hand-chosen search/exploration constants (cUCT, Cmax, N, beam sizes) that shape the experimental curves but are not fit to the test-set outcome. The axioms are a mix of standard bandit background and domain assumptions about the representativeness of the benchmark configuration and the reliability of the Monte Carlo targets. The most fragile ledger entry is the untested reliability of the value targets, since the entire inference-time bootstrap leans on it.

free parameters (4)
  • cUCT = 4.0
    Exploration constant in the UCT selection rule (Eq. 2); chosen by hand with no sensitivity analysis or tuning curve. It directly affects the balance between exploitation and exploration during MCTS training and inference.
  • Candidate cap Cmax = 3
    Maximum number of children expanded per node in MCTS (training and inference); hand-chosen, not swept. Determines the breadth of the search tree and the coverage of the value targets.
  • MCTS simulation counts N = 40 (training), 20 (inference)
    Number of rollouts per problem for label generation and for inference-time search; hand-chosen budgets. The value targets' quality and the search frontier both depend on these counts.
  • Beam sizes (B1, B2) = (1,5) and (3,5)
    SBS global beam width and per-state sample count; chosen to hit matched agent-call budgets (AC=20 and AC=50). They are experimental configuration, but they shape the reported accuracy/cost trade-off.
axioms (4)
  • domain assumption Terminal reward R(s_T)=+1 for exact-match correct final answer and -1 otherwise is a sufficient proxy for reasoning progress.
    Used throughout the training and evaluation (Section 3, Eq. 3). Assumes that intermediate message quality is fully captured by the eventual correctness of the extracted final answer, ignoring partial credit or multi-step correct reasoning that ends in a formatting error.
  • standard math UCT with a constant cUCT provides a valid bandit-based search and the empirical Q-hat converges to Q* as rollouts increase.
    Standard MCTS/UCT background invoked in Section 3 Eq. (2); the paper does not re-derive this. It is a reasonable standard assumption, though the practical convergence with small N (20-40 rollouts) is unverified.
  • domain assumption The fixed 4-agent linear schedule (Reader, Planner, Solver, Verifier) and the specific communication graph are representative of multi-agent systems, and MASPRM generalizes to other topologies/schedules.
    All experiments use one schedule and one graph (App. C.2). The introduction argues MASPRM is topology/schedule-dependent and handles it via the state definition, but no experiment varies the graph or schedule, so the generalization claim is assumed, not demonstrated.
  • domain assumption Value targets y_proc(s')=Q-hat(s,a) from N=40 rollouts with Cmax=3 are reliable enough to train a value head V_phi that generalizes to unvisited states.
    The whole bootstrap (lambda=0) relies on V_phi being a good pointwise approximator of expected terminal reward. The paper lists Pearson/RMSE/R2 in App. B.5 as evaluation metrics but reports no numbers, leaving this assumption untested.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MASPRM: Multi-Agent System Process Reward Model." pith.science (2026). https://pith.science/paper/CCQFFXHG

@misc{pith2026251024803,
  author       = {Pith},
  title        = {Pith review of: MASPRM: Multi-Agent System Process Reward Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CCQFFXHG}},
  note         = {Machine review of arXiv:2510.24803}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Inference-time search over multi-agent systems (MAS) wastes compute when it cannot identify which agent's intermediate message advanced progress. We present the Multi-Agent System Process Reward Model (MASPRM), which scores routed transcripts (ordered sequences of messages between agents) and acts as an inference controller for step-level beam search (SBS) and Monte Carlo Tree Search (MCTS). MASPRM is trained from multi-agent MCTS rollouts labeled only with terminal outcome rewards, without human step-level annotations. We evaluate on GSM8K, MATH, MMLU, and LogiQA. Under matched scorer size and comparable MCTS budget, MASPRM exceeds a size-matched ORM by $+2.0$ to $+3.0$ points at 1.5B and $+4.1$ to $+14.5$ at 7B across all four benchmarks, with additional scorer-scaling gains over policy likelihood at 7B (avg $+13.4$ under MCTS). MASPRM also improves ranking quality, reducing Hit@1 to Hit@5 gaps by up to $10.3$ points, with the largest gains under stepwise search that uses intermediate decisions. Code: https://github.com/milad1378yz/MASPRM

Figures

Figures reproduced from arXiv: 2510.24803 by Mahdi Mostajabdaveh, Milad Yazdani, Ying Xiong, Zirui Zhou.

Figure 1
Figure 1. Figure 1: Token-accuracy trade-off on GSM8K. Each point reports exact match (y-axis) versus average test￾time tokens (x-axis; ×103 ). MASPRM consistently shifts the frontier upward: it improves Maj@5 and step￾wise beam search (SBS) under matched budgets, and when paired with MCTS and an ORM, it reaches 74.6% EM at ∼19k tokens, while Greedy sits near 43.9% EM at ∼1.6k tokens. work so that no single context or policy … view at source ↗
Figure 2
Figure 2. Figure 2: Search-generated supervision. Left: Extracted rollouts yield edge-level estimates Qˆ(s, a); for each child s ′ = next(s, a) we set the regression target y = Qˆ(s, a) and train the MASPRM Vϕ(s ′ ) accordingly. Right: Uses of MASPRM in multi-agent systems, including inference-time guidance (SBS and MCTS via Vϕ and Qˆ). inter-agent exchanges; it provides little information about which message, from which agen… view at source ↗
Figure 5
Figure 5. Figure 5: shows value-guided search over the unrolled tree: internal annotations (e.g., 0.5, 0.25) depict ex￾ample mean values V backed up from MASPRM leaf evaluations; white circles denote agent outputs; green/red leaves indicate terminal reward +1/ − 1 (correct/incorrect). Selection uses UCT (Eq. 2) [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Stylized transcript (example). A 4-agent pipeline produces outputs o1-o4. Green is correct (a=c=7 ⇒ 5a−3c = 14); red is incorrect (35−21 ̸= 15). A.3 MCTS rollout with MASPRM values [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 19 linked inside Pith

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024. Alphamath almost zero: process supervision without process. Advances in Neural Information Processing Systems, 37:27689--27724

    2024

  4. [4]

    Zhenfang Chen, Delin Chen, Rui Sun, Wenjun Liu, and Chuang Gan. 2025. Scaling autonomous agents via automatic reward modeling and planning. arXiv preprint arXiv:2502.12130

    Pith/arXiv arXiv 2025

  5. [5]

    Sanjiban Choudhury. 2025. Process reward models for llm agents: Practical framework and directions. arXiv preprint arXiv:2502.10325

    Pith/arXiv arXiv 2025

  6. [6]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    Pith/arXiv arXiv 2021

  7. [7]

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, and 1 others. 2025. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456

    Pith/arXiv arXiv 2025

  8. [8]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088--10115

    2023

  9. [9]

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04519

    Pith/arXiv arXiv 2025

  10. [10]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    Pith/arXiv arXiv 2021

  11. [11]

    Levente Kocsis and Csaba Szepesv \'a ri. 2006. Bandit based monte-carlo planning. In European conference on machine learning, pages 282--293. Springer

    2006

  12. [12]

    Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. arXiv preprint arXiv:1706.03872

    Pith/arXiv arXiv 2017

  13. [13]

    Junwei Liao, Muning Wen, Jun Wang, and Weinan Zhang. 2025. Marft: Multi-agent reinforcement fine-tuning. arXiv preprint arXiv:2504.16129

    Pith/arXiv arXiv 2025

  14. [14]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. In The Twelfth International Conference on Learning Representations

    2023

  15. [15]

    Muhan Lin, Shuyang Shi, Yue Guo, Vaishnav Tadiparthi, Behdad Chalaki, Ehsan Moradi Pari, Simon Stepputtis, Woojun Kim, Joseph Campbell, and Katia Sycara. 2025. Speaking the language of teamwork: Llm-guided credit assignment in multi-agent reinforcement learning. arXiv preprint arXiv:2502.03723

    Pith/arXiv arXiv 2025

  16. [16]

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, and 1 others. 2024. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406.06592

    Pith/arXiv arXiv 2024

  17. [17]

    Kartik Nagpal, Dayi Dong, Jean-Baptiste Bouvier, and Negar Mehr. 2025. Leveraging large language models for effective and explainable multi-agent credit assignment. arXiv preprint arXiv:2502.16863

    Pith/arXiv arXiv 2025

  18. [18]

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. 2024. Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146

    Pith/arXiv arXiv 2024

  19. [19]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314

    Pith/arXiv arXiv 2024

  20. [20]

    Hanlin Wang, Chak Tou Leong, Jiashuo Wang, Jian Wang, and Wenjie Li. 2025. Spa-rl: Reinforcing llm agents via stepwise progress attribution. arXiv preprint arXiv:2505.20732

    Pith/arXiv arXiv 2025

  21. [21]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    Pith/arXiv arXiv 2022

  22. [22]

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, and 1 others. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144

    Pith/arXiv arXiv 2016

  23. [23]

    Yilin Yang, Liang Huang, and Mingbo Ma. 2018. Breaking the beam search curse: A study of (re-) scoring methods and stopping criteria for neural machine translation. arXiv preprint arXiv:1808.09582

    Pith/arXiv arXiv 2018

  24. [24]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809--11822

    2023

  25. [25]

    Thomas Zeng, Shuibai Zhang, Shutong Wu, Christian Classen, Daewon Chae, Ethan Ewer, Minjae Lee, Heeju Kim, Wonjun Kang, Jackson Kunde, and 1 others. 2025. Versaprm: Multi-domain process reward model via synthetic reasoning data. arXiv preprint arXiv:2502.06737

    Pith/arXiv arXiv 2025

  26. [26]

    Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, Tianlong Chen, and Dawei Cheng. 2024. G-designer: Architecting multi-agent communication topologies via graph neural networks. arXiv preprint arXiv:2410.11782

    Pith/arXiv arXiv 2024

  27. [27]

    Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vuli \'c , Anna Korhonen, and Sercan \"O Ar k. 2025. Multi-agent design: Optimizing agents with better prompts and topologies. arXiv preprint arXiv:2502.02533

    arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.