Pith. sign in

REVIEW 5 major objections 5 minor 38 references

The paper claims that multi-agent reasoning gains factor into three independent channels—coverage, information, and aggregation—and that a system jointly optimizing all three beats existing baselines.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 03:13 UTC pith:FET5AKRY

load-bearing objection The PRISM engineering is real, but the paper's central theoretical decomposition is an empty shell; send it out only if the authors are willing to drop or substantially rewrite the theory. the 5 major comments →

arxiv 2602.08586 v3 pith:FET5AKRY submitted 2026-02-09 cs.AI

DIANOIA: Diagnostic Decomposition and Joint Optimization for Multi-Agent Reasoning

Yiming Yang , Zhuoyuan Li , Fanxiang Zeng , Hao Fu , Yue Liu This is my paper
classification cs.AI
keywords multi-agent reasoninggain decompositioncoverageinformation feedbackaggregationtest-time scalinglarge language modelsdiagnostic framework
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to explain why multi-agent LLM systems outperform single agents and to give practitioners a diagnostic tool for designing better systems. It proposes that all multi-agent gains come from three separable mechanisms: exploring a broader solution space, obtaining higher-fidelity feedback, and aggregating proposals wisely. The expected quality of a multi-agent system is written as coverage probability times selection accuracy, which lets each channel be measured and optimized independently. If the decomposition holds, designers could diagnose which channel is the bottleneck for a new task and invest compute where it matters instead of scaling agents uniformly. The paper instantiates this in a system called PRISM and reports state-of-the-art results on math, code, and function-calling benchmarks.

Core claim

The central claim is Theorem 3.1: the expected quality of a multi-agent system equals C_K · η(f, s), where C_K is the probability that at least one proposal is correct and η(f, s) is the probability that the aggregator picks a correct proposal given one exists. From this multiplicative identity, the gain over the single-agent baseline p is bounded by the sum of three non-negative terms: exploration gain (C_K − p), information gain from execution feedback over textual feedback, and aggregation gain from principled synthesis over a baseline aggregator. The paper argues these dimensions are conceptually orthogonal at design time but interact multiplicatively at run time, so joint optimization i

What carries the argument

The central object is the multiplicative identity E[Q(τ_MAS)] = C_K · η(f, s) — coverage probability times selection accuracy — which decomposes total performance into exploration, information, and aggregation channels. This identity generates the three gain terms and implies that improving any single channel has diminishing returns unless the other channels are improved too. PRISM operationalizes the decomposition through role-diverse proposers (Minimalist, Skeptic, Explorer) for coverage, execution-grounded feedback with evidence-based cross-evaluation for information, and iterative synthesis with closed-loop validation for aggregation; a potential-game argument is invoked to support conve

Load-bearing premise

The proof that better feedback yields better selection accuracy relies on Lemma F.5, which states that Fano's inequality implies the Bayes-optimal error P*_e is bounded above by the conditional entropy H(Q|S); standard Fano for a binary Q gives H(Q|S) ≤ H_b(P*_e), which does not yield that upper bound, so the information channel's positive contribution is unsupported unless a valid argument is supplied.

What would settle it

On a deterministic-verification benchmark such as MBPP, compute the Bayes-optimal selection accuracy over generated proposals using (a) execution feedback and (b) textual self-critique as the signal; if execution feedback does not strictly improve selection accuracy, the information channel's positivity fails. Alternatively, search for any binary channel where the minimum-error probability exceeds H(Q|S), which would directly refute the lemma as stated.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the decomposition holds, practitioners can diagnose the bottleneck channel for any task by measuring C_K and η, then allocate tokens to the binding dimension rather than scaling agents or rounds uniformly.
  • The Pareto-frontier experiments imply joint optimization is more compute-efficient than scaling any single dimension or enlarging the base model: PRISM with a 30B model reportedly beats an 8x larger single model and reaches a high-accuracy ceiling at roughly five times fewer tokens than a strong baseline.
  • The framework predicts majority voting fails when correct answers lack natural clustering or the answer space fragments across equally valid solutions, explaining Self-Consistency's negative gain on AIME-2025 and negligible gains on code tasks.
  • On tasks without deterministic feedback, information gain vanishes, so the framework reduces to exploration plus aggregation; the paper explicitly states its theoretical guarantees weaken in that regime.
  • The decomposition reframes multi-agent design as channel-aware resource allocation, which could enable automated per-task selection of how many proposers, reviewers, and synthesis iterations to run.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One practical extension is a diagnostic toolkit: sample K proposals, estimate C_K with oracle labels, and estimate η for each candidate aggregation rule; the channel with the largest gap would indicate where compute should be spent.
  • The multiplicative structure suggests a marginal-value rule: when C_K is already high, improving selection accuracy yields more than adding proposers, while at low C_K the reverse holds; this could be turned into a token-budgeting rule.
  • The subadditivity coefficient (γ≈0.88 in the paper's MBPP experiment) might depend on task difficulty and feedback fidelity; if modeled, it could predict how much of the independent single-channel gains are actually realized jointly.
  • Because the theory relies on a verifiable quality signal, the framework's diagnostic value likely collapses in open-ended domains—a limitation the paper itself acknowledges, and a boundary worth testing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript—titled PRISM in the body and DIANOIA in the abstract—proposes a three-channel decomposition of multi-agent reasoning gains into exploration (coverage), information (feedback fidelity), and aggregation (selection/synthesis). It claims a multiplicative identity E[Q(τ_MAS)] = C_K·η(f,s) and a subadditive bound E[Q]−p ≤ G_explore + G_info + G_aggr, from which it derives design principles (role diversity, execution-grounded feedback, evidence-based synthesis) instantiated in the PRISM system, together with a potential-game convergence guarantee and a performance lower bound. Empirically, PRISM is evaluated on GSM8K, AIME-2025, MBPP, and BFCL-SP against Self-Consistency, MoA, Two Heads, and ReConcile, reporting accuracy gains at matched token budgets and Pareto dominance on MBPP. The paper further claims that the protocol identifies each benchmark's bottleneck channel and that the synergy coefficient γ=0.88 on MBPP validates the predicted subadditivity. The experimental protocol is careful and reproducible; however, the central theoretical claims, which constitute the stated main contribution, are not supported by the proofs as written.

Significance. If Theorem 3.1's decomposition were genuine, the paper would fill a real gap: a measurable, diagnostic account of when and why multi-agent systems beat single agents would be a valuable contribution. Strengths to credit explicitly: the multiplicative identity E[Q]=C_K·η is clean, correct, and near-parameter-free; the experiments use matched token budgets, bootstrap confidence intervals, fixed seeds, and a Pareto-frontier comparison (Figure 3), which is a fair and strong way to compare methods; and the authors release code, adapters, and diagnostic metrics. These strengths are real and make the empirical portion of the paper credible on its own. However, the framing contribution—the three-channel bound, the claimed subadditivity, the information-theoretic optimality of execution feedback, the aggregation guarantee, and the convergence theorem—is not established by the supplied proofs, and the paper's own limitations sections (D.7.5, H) concede several of the needed idealizations. The empirical system alone, while competent, is an incremental addition to the multi-agent LLM literature.

major comments (5)
  1. [Theorem 3.1 / Appendix D.1, Part III] The three-channel bound is vacuous. In Appendix D.1 Part III, the proof derives E[Q]−p = (C_K−p)+C_K(η(f,s)−1) ≤ C_K−p = G_explore, and then adds G_info and G_aggr simply because they are non-negative; their definitions are never used. The inequality holds even in the degenerate world where feedback is uninformative and aggregation matches the baseline. The claimed subadditivity (Remark 3.1) is therefore asserted, not derived: no result relates the realized gain to the sum of individually optimized channel gains. Table 4's γ=0.88 and the '+9.8% linear-additivity upper bound' are empirical ablation sums, not implications of Theorem 3.1. The only non-trivial content is the multiplicative identity E[Q]=C_K·η, which is correct.
  2. [Lemma F.5 and Corollary, Appendix F.1] Lemma F.5 misstates Fano's inequality. For binary Q, Fano gives H(Q|S) ≤ H_b(P_e*); the proof's auxiliary claims are false ('H_b(p) ≤ 2p for p ≤ 0.5' fails at p=0.4), and the step 'H_b(P_e*) ≥ H(Q|S) and H_b(p) ≤ 1 ⇒ P_e* ≤ H(Q|S)' is a non sequitur. The Corollary's inference 'H(Q|e) < H(Q|σ) ⇒ P_e*(e) < P_e*(σ)' does not follow from an upper bound of the form P_e* ≤ H(Q|S); smaller conditional entropy does not imply smaller Bayes error in general. The conclusion η*(e) > η*(σ) actually holds in the deterministic regime for a different reason (e determines Q, so η*(e)=1), but that argument is not given, and for the pseudo-verification regime (GSM8K/AIME) the asserted η*(σ_v) > η*(σ) (Remark 3.2, Definition A.7) remains without proof or measurement.
  3. [Proposition 3.4(b) / Theorem 3.5(c), Appendix D.8] The bound η(f_PRISM) ≥ 1−ε0^{K−1} is not established. The proof (D.8) explicitly sets aside its own Scenario 2; with several incorrect proposals, the probability that at least one receives unanimous false endorsements scales as (K−1)·ε0^{K−1}, and the synthesizer's selection behavior is unmodeled. The paper's Caveat (Section 3) concedes that synthesis error is excluded, yet Theorem 3.5(c) inherits this bound as a performance guarantee. The regime of the 'general' bound is also ambiguous: with the paper's illustrative ε0=0.2 and GSM8K's p≈0.84 (§5.5), it gives ≥95.6%, above the observed 91.1% (Table 2); ε0 is never measured for the pseudo-verification regime, so the bound is not falsifiable as stated.
  4. [Proposition 3.2 / Appendix D.3] Inequality (4) is not derived. The proof compares Bonferroni lower bounds: the diverse-case lower bound exceeds the IID lower bound by −(K choose 2)·ρ̄·p(1−p) when ρ̄<0, but a shift in a lower bound does not establish a larger actual coverage probability. The proposition states G_diverse − G_iid ≈ ... > 0, which is stronger than anything proved. The measured negative correlations in Appendix C.1 support the intended claim empirically, so the point is repairable, but the proposition as stated overclaims.
  5. [Theorem 3.5(b) / Algorithm 1] The convergence guarantee applies to a potential game with utilities u_k = max_j Q(τ^(j)) + λR_k(τ^(k)) and best-response dynamics. Algorithm 1 is a single-synthesizer loop (S → execute → refine) with no players, no role-preference functions R_k, and no best-response updates; the algorithm does not play the game. Appendix D.7.5 concedes this idealization and Section H calls the guarantee 'a qualitative prediction,' but the theorem as stated ('PRISM satisfies ... finite-step convergence to a pure-strategy Nash equilibrium') overclaims for the actual system.
minor comments (5)
  1. [Title / Abstract vs. Section 1] The front matter refers to 'DIANOIA' (including a release URL .../DIANOIA4MAS and a Claude Code skill), while the full text is titled 'PRISM: A Principled Framework for Multi-Agent Reasoning via Gain Decomposition' and describes PRISM throughout; Sections 1–6 never mention DIANOIA. The manuscript is internally inconsistent about its own identity.
  2. [Table 4 / §5.3] Calling the sum of the single-dimension ablation gains (+5.2%, +3.8%, +0.8%) a 'theoretical linear-additivity upper bound of +9.8%' is a misnomer: the theory provides no additive bound on these ablation gains, even setting aside the vacuousness of Eq. (3).
  3. [Appendix F / Assumption A2] Typographical issues: 'Ineqality' (Lemmas F.4, F.5), 'Eqilibrium' (Lemma F.10). Conceptually, Proposition 3.2 uses Assumption A2's single p for role-specialized agents, although A2 defines p 'in the absence of role specialization'; the marginals under roles are never modeled.
  4. [Figure 3 / Abstract] The 'Baseline ceiling (84.2%)' label in Figure 3 refers to MoA's ceiling, not a single-model baseline; please clarify. Also, the abstract's claim that 'on every benchmark, the protocol picks the right bottleneck channels' is not operationalized—no quantitative criterion for 'right' is given, which makes the claim difficult to falsify.
  5. [Definition A.7 / Remark 3.2] The inequalities I(Q;σ_v) ≤ I(Q;e) and I(Q;σ_v) > I(Q;σ) are asserted as if they followed from the data processing inequality, but no Markov chain Q→e→σ_v (or Q→σ_v→σ) is established; these claims need proof or empirical measurement.

Circularity Check

4 steps flagged

Theorem 3.1's three-channel bound is vacuous by construction; the positive information channel rests on an inverted Fano inequality, and Theorem 3.5(a) is tautological.

specific steps
  1. self definitional [Theorem 3.1 (Eq. 3); Appendix D.1, Part III, Eq. (26)-(27)]
    "Since η(f,s)≤1, the second term is non-positive: E[Q(τ_MAS)]−p≤G_explore≤G_explore+G_info+G_aggr (27) where the second inequality follows from non-negativity of Ginfo,Gaggr≥0."

    The proof first obtains E−p = G_explore + C_K(η−1) ≤ G_explore, since η≤1. This already gives E−p≤C_K−p without using the definitions of G_info or G_aggr. Appending two non-negative terms preserves the inequality for any definitions, so Eq. (3) is not a derived decomposition or a testable prediction; it is a tautological consequence of G_explore alone. The claimed subadditivity (Remark 3.1) is never proved: Part IV only rewrites C_K(1−η) as an algebraic sum of 'information loss' and 'aggregation loss' and asserts interactions.

  2. self definitional [Definition A.5; Theorem 3.5(a); Remark 3.2 / Principle 2]
    "For code generation, quality is typically: Q(τ):=1[e.success ∧ e.tests = 1]. ... When quality Q is deterministically computable from execution feedback e (e.g., code with test suites, tool calls with return values), e is a sufficient statistic for solution quality: I(Q;e)=H(Q)."

    Q is defined as a deterministic function of e, so H(Q|e)=0 and I(Q;e)=H(Q) by construction. Calling this 'information-theoretic optimality' restates the definition of Q; it does not compare execution feedback with textual feedback. The paper then uses this tautology, together with Lemma F.5, to argue that G_info>0, making the information channel's positive contribution essentially definitional rather than derived.

  3. other [Appendix F.1, Lemma F.5 and its Corollary]
    "For the upper bound: P*_e≤H(Q|S) follows from H_b(P*_e)≥H(Q|S) and H_b(p)≤1 for all p. ... When I(Q;e)>I(Q;σ) (Proposition 3.3), Fano's inequality guarantees H(Q|e)<H(Q|σ), hence P*_e(e)<P*_e(σ), yielding η*(e)>η*(σ)."

    Standard Fano for binary Q gives H(Q|S) ≤ H_b(P*_e), which lower-bounds the conditional entropy in terms of the error probability; it does not upper-bound P*_e by H(Q|S). The paper inverts the bound and then uses that inversion to conclude that more mutual information implies higher Bayes-optimal selection accuracy. Without this step, η*(e)>η*(σ) and hence G_info>0 are unsupported, so the only non-tautological channel claim in Theorem 3.1 is not actually derived.

  4. fitted input called prediction [Section 5.3, Table 4 and surrounding text]
    "Individual dimension contributions are: Explore-only (+5.2%), Info-only (+3.8%), Aggr-only (+0.8%), yielding a theoretical linear-additivity upper bound of +9.8%. PRISM-full achieves +8.6% (76.0%→84.6%), corresponding to a synergy coefficient γ=8.6/9.8=0.88. The coefficient γ<1 indicates subadditivity ... This validates our framework (Remark 3.1)."

    The 'theoretical linear-additivity upper bound' is just the sum of measured single-dimension gains from the same MBPP evaluation, and γ is computed from those same numbers. It is then presented as confirming the subadditivity predicted by Remark 3.1. This is a post-hoc description of the identical dataset, not an independent prediction; the coefficient γ=0.88 is fitted to the data it is said to validate.

full rationale

The paper's central theoretical claim is not self-contained: Theorem 3.1's three-channel bound reduces to the already-trivial coverage bound E−p≤C_K−p, with G_info and G_aggr attached only because they are defined to be non-negative. The one substantive channel claim, G_info>0, depends on Lemma F.5, which misstates Fano's inequality and therefore does not establish η*(e)>η*(σ). Theorem 3.5(a) is tautological because Q is defined as a deterministic function of execution feedback. The subadditivity coefficient γ=0.88 is computed from the same MBPP results it is used to confirm, so the empirical 'validation' is circular in that narrow sense. I did not find load-bearing self-citation: the references cited are external and the circularity lies in the derivation itself, not in a self-citation chain. The empirical Pareto and benchmark comparisons may be valuable and are not themselves circular, but the advertised gain decomposition and diagnostic protocol are not supported as derived predictions.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central claim rests on standard conditioning (valid), on assumptions A1/A4/A5/A6 (approximate), and on a mis-stated Fano lemma that is load-bearing for the information channel. No new physical or ontological entities are introduced; the 'channels' are conceptual labels for standard design axes.

free parameters (4)
  • baseline success probability p = 0.40 (theory examples); 0.76 (MBPP); 0.84 (GSM8K)
    Task-specific input; not fitted by the method, but appears in bounds and examples and is estimated from data when used for explanations.
  • reviewer error rate ε0 = 0.2 (examples)
    Chosen for numerical examples; no empirical estimate; drives Prop 3.4(b) and Theorem 3.5(c) bounds.
  • role preference weight λ = not specified
    Appears in the potential-game utility; never set or tuned; the convergence theorem only requires λ>0.
  • synergy coefficient γ = 0.88
    Computed from MBPP Table 4 as the ratio of PRISM-full gain to the sum of individual-dimension gains; used to claim subadditivity but is a post-hoc summary of the same data.
axioms (5)
  • domain assumption Assumption A1: conditional independence of proposers given input
    Stated in Appendix B; unrealistic for same-base-model agents. The paper relaxes it in Prop 3.2, but the relaxation only establishes a lower-bound difference.
  • domain assumption Assumption A4: deterministic execution feedback
    Holds for MBPP/BFCL-SP, not for GSM8K/AIME; Theorem 3.5(a) and the information-tightened bound depend on it.
  • domain assumption Assumptions A5/A6: reviewer error rate ε0<0.5 and conditional independence of reviewers
    No calibration evidence; Prop 3.4(b)'s exponential bound depends on both; A6 is acknowledged as approximate.
  • ad hoc to paper Lemma F.5: P*_e ≤ H(Q|S) from Fano's inequality
    Stated in Appendix F.1 and used to prove G_info>0. Fano gives H(Q|S) ≤ H_b(P*_e), not an upper bound on P*_e; as stated the lemma is false.
  • standard math Law of total probability in Theorem 3.1
    Used to write E[Q]=C_K·η; valid but tautological.

pith-pipeline@v1.3.0-alltime-deepseek · 31855 in / 18413 out tokens · 193265 ms · 2026-08-03T03:13:21.399315+00:00 · methodology

0 comments
read the original abstract

Multi-agent LLM systems consistently outperform single-agent baselines, yet practitioners still cannot predict which design works for a new task or diagnose why one fails. We argue this gap persists largely because the field lacks a diagnostic framework with measurable primitives and testable predictions. We introduce \textbf{DIANOIA}, a three-channel decomposition of multi-agent reasoning gain into coverage, fidelity, and synthesis, each of which is empirically measurable. From this decomposition, we derive a diagnostic protocol that identifies the bottleneck channels for any given task. We instantiate the protocol as a multi-agent system whose three components mirror the channels: role-diverse proposers for coverage, execution-grounded verification for fidelity, and iterative synthesis. On GSM8K, AIME-2025, MBPP, and BFCL-SP, our method outperforms strong multi-agent baselines under matched token budgets, dominating the Pareto frontier on MBPP at $\sim$$5{\times}$ token savings and reaching $+4.6$pp at matched cost. On every benchmark, the protocol picks the right bottleneck channels; the system we built around it leads across models. We release code, adapters, diagnostic metrics, and a Claude Code skill at https://anonymous.4open.science/r/DIANOIA4MAS. DIANOIA reframes multi-agent design as channel-aware resource allocation: diagnose which channel is the bottleneck for your task, then invest tokens accordingly.

Figures

Figures reproduced from arXiv: 2602.08586 by Fanxiang Zeng, Hao Fu, Yiming Yang, Yue Liu, Zhuoyuan Li.

Figure 1
Figure 1. Figure 1: The gain decomposition theory. Like a prism decom [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the PRISM framework. PRISM implements a four-phase workflow to jointly maximize all three gain [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Equal-budget efficiency frontier on MBPP. Each [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 12 linked inside Pith

  1. [1]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732

    Pith/arXiv arXiv 2021

  2. [2]

    Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2024. ReConcile: Round-Table Conference Improves Reasoning via Consensus Among Diverse PRISM: A Principled Framework for Multi-Agent Reasoning via Gain Decomposition LLMs. InProceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics. Association for Computational Lingu...

    2024

  3. [3]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168

    Pith/arXiv arXiv 2021

  4. [4]

    2006.Elements of Information Theory(2nd ed.)

    Thomas M Cover and Joy A Thomas. 2006.Elements of Information Theory(2nd ed.). Wiley-Interscience, Hoboken, NJ, USA

    2006

  5. [5]

    1785.Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix

    Marquis de Condorcet. 1785.Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix. De l’Imprimerie Royale, Paris

  6. [6]

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mor- datch. 2024. Improving Factuality and Reasoning in Language Models through Multiagent Debate. InForty-first International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 11733–11763

    2024

  7. [7]

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large Language Model based Multi-Agents: A Survey of Progress and Challenges. arXiv preprint arXiv:2402.01680

    Pith/arXiv arXiv 2024

  8. [8]

    Lu Hong and Scott E Page. 2004. Groups of Diverse Problem Solvers Can Out- perform Groups of High-Ability Problem Solvers.Proceedings of the National Academy of Sciences101, 46 (2004), 16385–16389

    2004

  9. [9]

    Dong Huang, Jie Bu, Jie Zhang, Michael Luck, and Zhi Wen. 2024. AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation. arXiv preprint arXiv:2312.13010

    Pith/arXiv arXiv 2024

  10. [10]

    Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. 2024. Map- Coder: Multi-Agent Code Generation for Competitive Problem Solving. arXiv preprint arXiv:2405.11403

    Pith/arXiv arXiv 2024

  11. [11]

    Chengsen Jin, Haozhen Peng, Qi Zhang, Yuqing Tang, Dimitris N Metaxas, and Mengdi Wang. 2025. Two Heads Are Better Than One: Test-Time Scaling of Multi-Agent Collaborative Reasoning. arXiv preprint arXiv:2504.09772

    Pith/arXiv arXiv 2025

  12. [12]

    Weiya Jin, Haifeng Du, Baoxiang Zhao, Xiaobo Tian, Bo Shi, and Guang Yang. 2025. A Comprehensive Survey on Multi-Agent Cooperative Decision- Making: Scenarios, Approaches, Challenges and Perspectives. arXiv preprint arXiv:2503.13415

    Pith/arXiv arXiv 2025

  13. [13]

    Zhangyue Ke, Fei Jiao, Yichuan Ming, Xuan-Phi Nguyen, Anbang Xu, Shulin Li, Zhenwen Wang, Xuan-Son Huang, Shunyu Yao, and Shafiq Joty. 2025. A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems. arXiv preprint arXiv:2504.09037

    arXiv 2025

  14. [14]

    Elliot Kim, Avi Garg, Kenny Peng, and Nikhil Garg. 2025. Correlated Errors in Large Language Models. InProceedings of the 42nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267). PMLR, Vancouver, Canada. arXiv:2506.07962

    Pith/arXiv arXiv 2025

  15. [15]

    Anders Krogh and Jesper Vedelsby. 1995. Neural Network Ensembles, Cross Validation, and Active Learning. InAdvances in Neural Information Processing Systems, Vol. 7. MIT Press, Denver, CO, USA, 231–238

    1995

  16. [16]

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2024. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Miami, Florida, USA, 17...

    2024

  17. [17]

    Zhenwen Liang, Ye Liu, Tao Niu, Xiangliang Zhang, Yingbo Zhou, and Yong Chen. 2024. Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification. arXiv preprint arXiv:2410.05318

    Pith/arXiv arXiv 2024

  18. [18]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s Verify Step by Step. InThe Twelfth International Conference on Learning Representations. OpenReview.net, Vienna, Austria

    2024

  19. [19]

    Mathematical Association of America. 2025. The 43rd American Invita- tional Mathematics Examination (AIME) I & II. https://www.maa.org/math- competitions/aime. Accessed: 2026-02-06

    2025

  20. [20]

    Dov Monderer and Lloyd S Shapley. 1996. Potential Games.Games and Economic Behavior14, 1 (1996), 124–143

    1996

  21. [21]

    Ranjita Naik, Varun Chandrasekaran, Mert Yuksekgonul, Hamid Palangi, and Besmira Nushi. 2023. Diversity of Thought Improves Reasoning Abilities of Large Language Models. InNeurIPS 2023 Workshop on Instruction Tuning and Instruction Following. NeurIPS, New Orleans, LA, USA

    2023

  22. [22]

    2007.The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies

    Scott E Page. 2007.The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press, Princeton, NJ, USA

    2007

  23. [23]

    Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E

    Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267...

    2025

  24. [24]

    Priya Pitre, Naren Ramakrishnan, and Xuan Wang. 2025. CONSENSAGENT: Towards Efficient and Effective Consensus in Multi-Agent LLM Interactions through Sycophancy Mitigation. InFindings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics, Vienna, Austria, 22112–22133

    2025

  25. [25]

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. 2024. Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning. arXiv preprint arXiv:2410.08146

    Pith/arXiv arXiv 2024

  26. [26]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems, Vol. 36. Curran Associates, Inc., New Orleans, LA, USA, 8634–8652

    2023

  27. [27]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Pa- rameters. InThe Thirteenth International Conference on Learning Representations. OpenReview.net, Singapore

    2025

  28. [28]

    Haoran Sun, Yusen Wu, Yukun Cheng, and Xu Chu. 2025. Game Theory Meets Large Language Models: A Systematic Survey. InProceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. International Joint Conferences on Artificial Intelligence Organization, Montreal, Canada, 10669–10677. doi:10.24963/ijcai.2025/1184 Survey Track

    doi:10.24963/ijcai.2025/1184 2025

  29. [29]

    Khanh-Tung Tran, Duy Dao, Minh-Duong Nguyen, Quoc-Viet Pham, and Minh- Triet Ngo. 2025. Multi-Agent Collaboration Mechanisms: A Survey of LLMs. arXiv preprint arXiv:2501.06322

    Pith/arXiv arXiv 2025

  30. [30]

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. 2024. Mixture-of-Agents Enhances Large Language Model Capabilities. arXiv preprint arXiv:2406.04692

    Pith/arXiv arXiv 2024

  31. [31]

    Tianle Wang, Zhen Liu, Yuxin Chen, John Light, Hongyi Chen, and Jian Li

  32. [32]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. InThe Eleventh International Conference on Learning Representations. OpenReview.net, Kigali, Rwanda

    2023

  33. [33]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Rea- soning in Large Language Models. InAdvances in Neural Information Processing Systems, Vol. 35. Curran Associates, Inc., New Orleans, LA, USA, 24824–24837

    2022

  34. [34]

    Danny Wood, Tingting Mu, Andrew Webb, Henry Reeve, Mikel Lujan, and Gavin Brown. 2023. A Unified Theory of Diversity in Ensemble Learning.Journal of Machine Learning Research24, 359 (2023), 1–49

    2023

  35. [35]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. InAdvances in Neural Information Processing Systems, Vol. 36. Curran Associates, Inc., New Orleans, LA, USA, 11809–11822

    2023

  36. [36]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InThe Eleventh International Conference on Learning Representations. OpenReview.net, Kigali, Rwanda

    2023

  37. [37]

    correct”,“incorrect

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu- Xiong Wang. 2024. Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 62138–62160. A Formal Definitions T...

    2024

  38. [2025]

    arXiv preprint arXiv:2502.11027

    Diversified Sampling Improves Scaling LLM Inference. arXiv preprint arXiv:2502.11027

    arXiv