Literature Review Of Multi-Agent Debate For Problem-Solving

Arne Tillmann

REVIEW 4 major objections 6 minor 2 cited by

Literature Review Of Multi-Agent Debate For Problem-Solving

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This literature review synthesizes evidence that multi-agent LLM systems can outperform single-agent models on problem-solving tasks, but that scaling up the number of agents or debate rounds initially helps and then degrades performance…

desk verdict A useful but methodologically loose survey; the scaling synthesis needs an auditable corpus and a reconciliation of the Becker negative result before the 'often outperform' claim is credible. read the letter →

arxiv 2506.00066 v1 pith:WHKONAOK submitted 2025-05-29 cs.MA cs.AI

Arne Tillmann This is my paper

classification cs.MAcs.AI

keywords multi-agentdebateLLMproblem-solvingscalingdecision-makingcontextexplosionproblemdriftliteraturereview

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The review argues that multi-agent large language models (MA-LLMs) often beat single-agent LLMs on reasoning and problem-solving benchmarks by pooling diverse perspectives and iterative feedback. However, the gains are bounded: adding agents or debate rounds initially improves accuracy, then performance peaks and declines, typically around 3 to 4 agents and 2 to 4 rounds depending on task and topology. The paper also imports a classification scheme from traditional multi-agent systems to organize agent profiles, communication topologies, and decision-making mechanisms, and it identifies context explosion as a major scaling cost. A sympathetic reader would take the central message as a roadmap: multi-agent debate is promising but must be managed carefully because its benefits saturate and can reverse.

What carries the argument

The central organizing device is a three-part classification scheme adapted from traditional multi-agent systems: agent profiles (pre-defined, model-generated, data-derived, with prompt-engineered and model-family subcategories), communication topology (hierarchical vs. non-hierarchical, fix vs. dynamic, homogeneous vs. heterogeneous, holonic), and decision-making processes (majority voting, judge, consensus). The scaling argument is carried by two named mechanisms: context explosion, the quadratic growth in token pressure with agent count in certain topologies, and problem drift, the systematic decay in performance as agents diverge from the original task across debate rounds.

What would settle it

A single large controlled study that varies the number of agents (from 1 to, say, 20) and debate rounds (from 1 to 10) on several tasks with consistent evaluation and standard errors would settle the claim: if accuracy keeps rising monotonically or shows no consistent peak across tasks, the 3-to-4-agent and 2-to-4-round optimum would fail to generalize.

Watch

Extended reading notes

Core claim

The paper's central synthesis, stated in the abstract and repeated in the discussion, is that multi-agent approaches can yield superior results compared to single-agent LLMs, yet they also face elevated computational costs and challenges unique to MA-LLMs. Concretely, the review finds that increasing the number of agents or debate rounds consistently boosts performance at first, but both factors can eventually cause degradation due to problem drift, interference, or redundancy. The optimal configuration is task- and topology-dependent, with frequent sweet spots of 3 to 4 agents and 2 to 4 rounds. The review further claims that no universally optimal decision-making process exists, drawing on Arrow's impossibility theorem, and that current research lacks rigorous statistical comparisons and standard-error reporting.

Load-bearing premise

The review's quantitative conclusions presume that the corpus found through Elicit and informal searches is representative and that benchmark results from different papers can be compared as if they came from a single protocol, even though many studies do not report standard errors.

Editorial extensions

If this is right

If the synthesis holds, system designers should treat agent count and debate rounds as tunable hyperparameters with an optimum, not as monotonically beneficial resources.
Adopting dynamic or pruned communication topologies, such as agent deactivation or sparse message passing, could push the performance peak to larger team sizes than static fully-connected debate.
Decision-making mechanisms should be chosen with the same care as agent prompts, since majority voting, judge, and consensus each have distinct failure modes and no universal optimum exists.
The field needs standardized reporting of standard errors and statistical significance before cross-paper claims about optimal agent counts or debate rounds can be trusted.
The classification scheme itself gives researchers a common vocabulary for comparing MA-LLM systems directly rather than only against single-agent baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The recurring 3-to-4 agent and 2-to-4 round optima may be an artifact of current LLM context windows and instruction-following limits, so they could shift as models improve or as memory management matures.
A testable extension would be to replace fixed debate rounds with an adaptive stopping rule based on measured inter-agent agreement, which could reduce token costs and avoid problem drift on easy tasks.
Group-level biases, beyond the named problem drift and degeneration-of-thought, likely also degrade scaling and deserve systematic study similar to individual positional and demographic biases.
If the cost-effectiveness framing is taken seriously, the practical recommendation is not necessarily to maximize accuracy but to find the smallest agent-and-round budget that reaches a target accuracy, a trade-off the author notes is under-quantified.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a competent but methodologically loose survey of MA-LLM debate. The taxonomy transfer from traditional MAS is genuinely useful, and the review does a good job mapping decision-making mechanisms and the scaling literature through early 2025. But the quantitative conclusions about optimal agent counts and debate rounds are not supported by an auditable corpus, and the review never reconciles a negative result that undercuts its own abstract.

The paper's best contributions: it imports Maldonado et al.'s four-way topology classification (hierarchical/non-hierarchical, fixed/dynamic, homogeneous/heterogeneous, holonic) into the MA-LLM discussion, which is a helpful lens. The suggestion to add 'instruction' as a fourth category alongside agent profiles, topology, and decision-making is small but sensible. The survey also clearly surfaces the field's two real weaknesses: missing standard errors and the near-total absence of head-to-head MA-LLM comparisons. Those are not novel observations, but they are accurate and worth stating.

The soft spots are where the reader and stress-test point. The methodology section describes Elicit and keyword searches but gives no inclusion criteria, no paper count, no PRISMA-style flow, and no final corpus list. That makes the selection non-auditable and means the 'task- and topology-dependent' caveat absorbs every contradiction in the data. The scaling narrative cites results from 2-4 agents (Liang, Wu, Chan, Zhang) up to 7, 8, 20, 40, and 1000+ agents (Du, Liu, Zhang, Li, Qian) and treats them as consistent evidence of 'initially boosts, then degrades.' That is only coherent if the heterogeneity is front and center, yet the abstract's 'often outperform single-agent LLMs' is exactly the kind of unqualified claim the evidence does not support.

The most serious specific problem is the Becker et al. result in Section 6.2.1: only about 0.5% of discussions improve over the first-turn draft and about 0.8% degrade. The review reports this without reconciling it with its 'multi-agent approaches can yield superior results' framing. A reader is entitled to ask why the modal effect is near zero. That is not a minor omission.

The citation pattern is fine: yes, Becker and Kaesberg are from the same Göttingen group, but the central claims rest on many external papers. Self-citation is not the issue here.

Who this is for: a newcomer to MA-LLM debate who wants a structured overview and a set of pointers. It is not a reliable source for quantitative guidance. It deserves a serious referee, because a revised version with an auditable corpus and a sober treatment of the negative result could be a genuinely useful review. But as is, the scaling synthesis is not ready.

Recommendation: send to peer review with a request for major revision, and make the corpus table and the Becker reconciliation conditions for resubmission.

Referee Report

4 major / 6 minor

Summary. The paper is a literature review of multi-agent large language models (MA-LLMs) for problem-solving. It proposes a taxonomy of agent profiles, communication topologies, and decision-making processes, and it attempts to synthesize scaling behavior with respect to the number of agents and debate rounds. The central claims are that MA-LLMs often outperform single-agent LLMs, that performance initially improves with more agents/rounds but then degrades due to problem drift, interference, or redundancy, and that optimal configurations are typically 3-4 agents and 2-4 rounds, depending on task and topology. The review also identifies challenges such as context explosion, computational cost, and weak statistical reporting in the field.

Significance. If the synthesis were supported by a rigorous and auditable methodology, this review would be a valuable resource for researchers navigating the rapidly growing MA-LLM literature. The paper provides a useful classification scheme adapted from traditional multi-agent systems, a structured description of decision-making mechanisms (majority voting, judge, consensus), and a clear catalog of scaling-related phenomena such as context explosion and problem drift. It also explicitly acknowledges several field-wide weaknesses, including missing standard errors and the tendency to compare against single-agent baselines rather than other MA-LLM systems. However, the significance is currently undercut by the lack of a reproducible corpus and by the unresolved tension between the reported negative result from Becker et al. and the review's overall positive synthesis.

major comments (4)

[Section 3.2 and Section 3.1] The corpus assembled for this review has no reproducible inclusion criteria, no final paper list, and an admitted arbitrary inclusion rule ('because the definition is arbitrary, I also include several noteworthy papers'). The Elicit-based search is described qualitatively with no search strings, dates, screening steps, or handling of duplicates. This matters because every quantitative claim in Section 6 is a comparison across papers from this non-auditable corpus; without a defined sampling frame, the central synthesis cannot be independently verified or updated.
[Section 6.2.1 and abstract] The review reports Becker et al. (2025) as finding that only roughly 0.5% of analyzed discussions benefit from multi-agent debate relative to the first-turn draft, while roughly 0.8% suffer a performance drop. This directly undercuts the abstract's claim that MA-LLMs are 'outperforming single-agent large language models' and Section 7.1's statement that adding agents or rounds 'consistently boosts performance' initially. The review never reconciles this negative result with the overall positive synthesis, nor does it explain why this finding should not dominate the qualitative conclusion. The authors should either provide a concrete explanation (e.g., task domain, baseline definition, measurement window) or substantially soften the central claim.
[Sections 6.2.2 and 6.3] The scaling conclusions are based on an unweighted aggregation of studies that differ in tasks, base models, communication topologies, baseline choices (single-agent vs. other MA-LLM systems), and evaluation metrics. Section 6.3 itself admits that 'many studies in this area do not consistently report standard errors or other measures of statistical significance,' and Section 6.2.2 notes that many studies only report up to three or four rounds. Yet the review still presents specific optimal ranges (3-4 agents, 2-4 rounds) and a general 'initially boosts, then degrades' narrative. The 'task- and topology-dependent' caveat in Section 7.1 absorbs all contradictory findings, making the central scaling claim unfalsifiable in its current form. A formal meta-analysis or, failing that, an explicit presentation of the heterogeneity and a justification for why a central tendency is still meaningful is required.
[Section 3.3 and Section 6.2.2] The reading process described in Section 3.3 is a three-step filter (First Impression, Bird's Eye View, Full Read) but no inter-rater reliability, no dual screening, and no explicit criteria for what counts as 'relevant enough for full reading' are provided. Combined with the absence of a corpus list, this means the reader cannot assess selection bias. This is particularly problematic for the review's performance-scaling claims, where the selection of which results to report (e.g., which studies are cited for 'peak at four agents' vs. 'continuous improvement to 40 agents') could materially change the synthesis.

minor comments (6)

[Section 3.2] There is a typo in 'I utilizeElicit' (missing space) and the sentence 'Elicit’s ability to summarize key findings and highlight methodologies helps me quickly assess the relevance and quality of each source' could be more precise about how quality is assessed.
[Section 4.2.2] The word 'schafolding' appears to be a misspelling of 'scaffolding'; please correct.
[Section 5.3] The phrase 'consensus a priori by Xu et al. [2023]' is grammatically unclear; consider revising to 'consensus imposed a priori, as in Xu et al. [2023]'.
[Section 7.1] The sentence 'It appears to that perspectives are tailored primarily in fields such as software development and text summarization' contains a grammatical error ('appears to that'); should read 'It appears that perspectives are tailored primarily...'
[Throughout] The manuscript inconsistently uses the first-person singular ('I') and plural ('we'), sometimes within the same section (e.g., Section 3.2 vs. Section 7.1). Please standardize the voice.
[Section 1] The statement 'The reference to the latest OpenAI model serves as evidence for increasingly powerful Large Language Models' is vague and does not name the specific model or the nature of the evidence; consider clarifying or removing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the review's synthesis is an inductive summary of cited empirical studies, not a derivation from fitted inputs or a self-citation chain.

full rationale

This paper is a literature review and performs no formal derivation, fitting, or prediction. Its central claims—that MA-LLMs often outperform single-agent LLMs but face computational costs and scaling-related degradation—are inductive summaries of the cited empirical literature. The scaling conclusions in Sections 6.2 and 6.3 rest on a heterogeneous set of external studies (Du, Liu, Zhang, Li, Qian, Liang, Wu, Chan, Xu, Ye, etc.), not on parameters fitted in this paper. The self-citations to Becker et al. (2025) and Kaesberg et al. (2025), both from the same Göttingen group as the review, are used as two empirical data points among many and are not the sole support for any central claim; the 'problem drift' concept and the 0.5%/0.8% result are reported as findings from those papers, not derived here. Arrow's impossibility theorem is cited as an external mathematical result and is used only to motivate the non-existence of a universally optimal decision mechanism, which is not circular. The review's acknowledged methodological limitations—heterogeneous benchmarks, missing standard errors, and a non-auditable Elicit-based corpus—undermine the strength of its quantitative synthesis, but they are correctness and generalizability risks, not circularity. No equation, fitted input, or self-citation chain is equivalent to the paper's conclusions by construction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities are introduced. The review's load-bearing premises are about corpus representativeness and cross-paper comparability rather than any mathematical or structural assumption.

assumptions (3)

domain assumption The Elicit-assisted search and informal keyword queries yield a representative sample of the MA-LLM literature.
Sections 3.2 and 3.3 describe the search but provide no inclusion and exclusion criteria or exhaustive corpus list; all scaling conclusions depend on this sample.
domain assumption Benchmark results from different papers can be compared across tasks and models without meta-analytic adjustment.
Sections 6.2 and 6.3 aggregate findings from diverse benchmarks while admitting that many studies omit standard errors.
domain assumption Arrow's impossibility theorem, originally about preference aggregation, applies to MA-LLM decision-making viewed as preference aggregation.
Section 5 invokes Arrow's theorem to argue that no universally optimal decision process exists; this is a reasonable analogy but not formally established for MA-LLMs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Literature Review Of Multi-Agent Debate For Problem-Solving." pith.science (2026). https://pith.science/paper/WHKONAOK

@misc{pith2026250600066,
  author       = {Pith},
  title        = {Pith review of: Literature Review Of Multi-Agent Debate For Problem-Solving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WHKONAOK}},
  note         = {Machine review of arXiv:2506.00066}
}

read the original abstract

Multi-agent large language models (MA-LLMs) are a rapidly growing research area that leverages multiple interacting language agents to tackle complex tasks, outperforming single-agent large language models. This literature review synthesizes the latest research on agent profiles, communication structures, and decision-making processes, drawing insights from both traditional multi-agent systems and state-of-the-art MA-LLM studies. In doing so, it aims to address the lack of direct comparisons in the field, illustrating how factors like scalability, communication structure, and decision-making processes influence MA-LLM performance. By examining frequent practices and outlining current challenges, the review reveals that multi-agent approaches can yield superior results but also face elevated computational costs and under-explored challenges unique to MA-LLM. Overall, these findings provide researchers and practitioners with a roadmap for developing robust and efficient multi-agent AI solutions.

Figures

Figures reproduced from arXiv: 2506.00066 by the authors.

**Figure 2.** Scaling in MA-LLMs with Citations a critical limitation of extended interactions, as the benefit of additional exchanges can diminish or reverse entirely. The optimal number of debate rounds varies depending on task complexity. Liang et al. [2024] observe that complex tasks benefit from multiple rounds of debate, as nuanced information accumulates gradually, whereas more straightforward tasks typically reach peak … view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges
cs.SE 2026-07 accept novelty 6.0 of 10

A systematic review of 141 papers derives a three-axis taxonomy of multi-agent debate design (participants, interaction, agreement) and shows the field has converged on a narrow default pattern.
Scaling LLM-Driven Multi-Agent Systems: Design Principles and Architectural Scalability Analysis
cs.MA 2026-07 conditional novelty 5.0 of 10

Architecturally scaling LLM multi-agent systems raises accuracy at near-linear cost only above a model-capability threshold, peaks at intermediate complexity, and never fixes poor run-to-run consistency.

Reference graph

Works this paper leans on

27 extracted references · 6 canonical work pages · cited by 2 Pith papers

[5]

In Proceedings of the 62nd Annual Meeting of the ACL (Volume 1: Long Papers)

ReConcile: Round- Table Conference Improves Reasoning via Consensus among Diverse LLMs. In Proceedings of the 62nd Annual Meeting of the ACL (Volume 1: Long Papers). Associa- tion for Computational Linguistics, Online, 7066–7085. Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025.Optima: Optimizing Effectiveness and Efficie...

arXiv doi:10.48550/arxiv.2410.08115 2025
[6]

arXiv:1706.03741 [stat.ML] https://arxiv.org/abs/1706.03741 Ben Cottier, Ben Snodin, David Owen, and Tom Adamczewski

Deep reinforcement learning from human preferences. arXiv:1706.03741 [stat.ML] https://arxiv.org/abs/1706.03741 Ben Cottier, Ben Snodin, David Owen, and Tom Adamczewski

arXiv
[7]

https://epoch.ai/data- insights/llm-inference-price-trends Accessed: 2025-03-28

LLM inference prices have fallen rapidly but unequally across tasks. https://epoch.ai/data- insights/llm-inference-price-trends Accessed: 2025-03-28. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch

work page 2025
[8]

https://doi.org/10.48550/arXiv.2305.14325 arXiv:2305.14325 Elicit

Improving Factuality and Reasoning in Language Models through Multiagent Debate. https://doi.org/10.48550/arXiv.2305.14325 arXiv:2305.14325 Elicit. 2023.Elicit: The AI Research Assistant. Ought.org. https://elicit.com Epoch AI

arXiv doi:10.48550/arxiv.2305.14325 2023
[9]

https://epoch.ai/data/notable-ai-models Accessed: 2024-11-25

Data on Notable AI Models. https://epoch.ai/data/notable-ai-models Accessed: 2024-11-25. Ege Erdil and Tamay Besiroglu

work page 2024
[10]

Algorithmic Progress in Computer Vision. https://doi.org/10.48550/arXiv.2212.05153 arXiv:2212.05153 Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli Järviniemi, Matthew Barnett, Robert Sandler, Matej Vrzala, Jaime Sevilla, Qiuyu Ren...

arXiv doi:10.48550/arxiv.2212.05153
[11]

https://doi.org/10.48550/arXiv.2411.04872 arXiv:2411.04872 Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V

FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI. https://doi.org/10.48550/arXiv.2411.04872 arXiv:2411.04872 Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang

arXiv doi:10.48550/arxiv.2411.04872
[12]

InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Kate Larson (Ed.)

Large Language Model Based Multi- agents: A Survey of Progress and Challenges. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Kate Larson (Ed.). International Joint Conferences on Artificial Intelligence Organization, Jeju, South Korea, 8048–8057. https://doi.org/10.24963/ijcai.2024/890 Survey Track....

doi:10.24963/ijcai.2024/890

Show all 27 references

[13]

InProceedings of the 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Discussion via Argumentation Schemes. InProceedings of the 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, Lisbon, Portugal, 5486–5493. Tianjie Ju, Yiting Wang, Xinbei Ma, Pengzh...

2024 doi arXiv
[14]

arXiv:2502.18969 [cs.LG] https://arxiv.org/abs/2502.18969 Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ie

(Mis)Fitting: A Survey of Scaling Laws. arXiv:2502.18969 [cs.LG] https://arxiv.org/abs/2502.18969 Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ie. 2024a. Improving Multi-Agent Debate with Sparse Communication Topology. InFindings of the ...

2024 arXiv
[15]

InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Miami, FL, USA, 17889–17904. Tongxuan Liu, Xingyu Wa...

2024 doi
[16]

https://doi.org/10.1109/ACCESS.2024.3409051 METR

Multi-Agent Systems: A Survey About Its Components, Framework and Workflow.IEEE Access12 (2024), 80950–80975. https://doi.org/10.1109/ACCESS.2024.3409051 METR

2024
[17]

https://metr.github.io/autonomy-evals-guide/elicitation-gap//#3.-results

Measuring the impact of post-training enhancements — metr.github.io. https://metr.github.io/autonomy-evals-guide/elicitation-gap//#3.-results. [Ac- cessed 25-11-2024]. Evan Miller. 2024.Adding Error Bars to Evals: A Statistical Approach to Lan- guage Model Evaluations. Anthrop...

2024 doi arXiv
[18]

ChatDev: Communicative Agents for Software Development. arXiv:2307.07924 [cs.SE] https://arxiv.org/abs/2307.07924 Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025.Scal- ing La...

2025 arXiv
[19]

arXiv:2402.16713 [cs.MA] https://arxiv.org/abs/ 2402.16713 Ciaran Regan, Alexandre Gournail, and Mizuki Oka

Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs. arXiv:2402.16713 [cs.MA] https://arxiv.org/abs/ 2402.16713 Ciaran Regan, Alexandre Gournail, and Mizuki Oka. 2024.Problem-Solving in Language Model Networks. University of Tsukuba, Japan. https://doi.or...

2024 arXiv
[20]

https://doi.org/10.48550/arXiv.2311.12022 arXiv:2311.12022 Mirac Suzgun and Adam Tauman Kalai

GPQA: A Graduate- Level Google-Proof Q&A Benchmark. https://doi.org/10.48550/arXiv.2311.12022 arXiv:2311.12022 Mirac Suzgun and Adam Tauman Kalai

doi arXiv
[21]

arXiv:2401.12954 [cs.CL] https://arxiv

Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding. arXiv:2401.12954 [cs.CL] https://arxiv. org/abs/2401.12954 Yashar Talebirad and Amirhossein Nadiri

arXiv
[22]

arXiv:2306.03314 [cs.AI] https://arxiv.org/ abs/2306.03314 Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui

Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents. arXiv:2306.03314 [cs.AI] https://arxiv.org/ abs/2306.03314 Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023a.Large Language Models...

arXiv
[23]

arXiv:2201.11903 [cs.CL] https://arxiv.org/ abs/2201.11903 Ning Wu, Ming Gong, Linjun Shou, Shining Liang, and Daxin Jiang

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 [cs.CL] https://arxiv.org/ abs/2201.11903 Ning Wu, Ming Gong, Linjun Shou, Shining Liang, and Daxin Jiang

arXiv
[24]

arXiv:2303.15078 [cs.CL] https://arxiv.org/abs/2303.15078 Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin

Large Language Models are Diverse Role-Players for Summarization Evaluation. arXiv:2303.15078 [cs.CL] https://arxiv.org/abs/2303.15078 Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin

arXiv
[25]

InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.)

Examining Inter- Consistency of Large Language Models Collaboration: An In-depth Analysis via De- bate. InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguis- tics, Singa...

2023 doi
[26]

arXiv:2311.08152 [cs.CL] https://arxiv.org/abs/2311.08152 Hai Ye, Mingbao Lin, Hwee Tou Ng, and Shuicheng Yan

Towards Reasoning in Large Language Models via Multi-Agent Peer Literature Review Of Multi-Agent Debate For Problem-Solving Conference’17, July 2017, Washington, DC, USA Review Collaboration. arXiv:2311.08152 [cs.CL] https://arxiv.org/abs/2311.08152 Hai Ye, Mingbao Lin, Hwee T...

2017 arXiv
[27]

InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.)

Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication. InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Lingu...

2023 doi arXiv
[1950]

https://doi.org/10.1086/256963 Leopold Aschenbrenner

A Difficulty in the Concept of Social Welfare.Journal of Political Economy58, 4 (1950), 328–328. https://doi.org/10.1086/256963 Leopold Aschenbrenner

1950 doi
[2023]

arXiv:2308.07201 [cs.CL] https://arxiv.org/abs/2308.07201 Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal

ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate. arXiv:2308.07201 [cs.CL] https://arxiv.org/abs/2308.07201 Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal

arXiv
[2024]

https://www.alignmentforum.org/posts/s9zd6f9eZ8qN2jrcu/auto-enhance- developing-a-meta-benchmark-to-measure-llm

Auto- Enhance: Developing a meta-benchmark to measure LLM agents’ ability to improve other agents — AI Alignment Forum — alignmentforum.org. https://www.alignmentforum.org/posts/s9zd6f9eZ8qN2jrcu/auto-enhance- developing-a-meta-benchmark-to-measure-llm. [Accessed 25-11-2024]. ...

2024
[2025]

arXiv:2502.19559 [cs.CL] https://arxiv.org/abs/2502.19559 Sam Brown, Basil Labib, Codruta Lugoj, and Sai Sasank Y

Stay Focused: Problem Drift in Multi-Agent Debate. arXiv:2502.19559 [cs.CL] https://arxiv.org/abs/2502.19559 Sam Brown, Basil Labib, Codruta Lugoj, and Sai Sasank Y

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [5]

In Proceedings of the 62nd Annual Meeting of the ACL (Volume 1: Long Papers)

ReConcile: Round- Table Conference Improves Reasoning via Consensus among Diverse LLMs. In Proceedings of the 62nd Annual Meeting of the ACL (Volume 1: Long Papers). Associa- tion for Computational Linguistics, Online, 7066–7085. Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025.Optima: Optimizing Effectiveness and Efficie...

arXiv doi:10.48550/arxiv.2410.08115 2025

[2] [6]

arXiv:1706.03741 [stat.ML] https://arxiv.org/abs/1706.03741 Ben Cottier, Ben Snodin, David Owen, and Tom Adamczewski

Deep reinforcement learning from human preferences. arXiv:1706.03741 [stat.ML] https://arxiv.org/abs/1706.03741 Ben Cottier, Ben Snodin, David Owen, and Tom Adamczewski

arXiv

[3] [7]

https://epoch.ai/data- insights/llm-inference-price-trends Accessed: 2025-03-28

LLM inference prices have fallen rapidly but unequally across tasks. https://epoch.ai/data- insights/llm-inference-price-trends Accessed: 2025-03-28. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch

work page 2025

[4] [8]

https://doi.org/10.48550/arXiv.2305.14325 arXiv:2305.14325 Elicit

Improving Factuality and Reasoning in Language Models through Multiagent Debate. https://doi.org/10.48550/arXiv.2305.14325 arXiv:2305.14325 Elicit. 2023.Elicit: The AI Research Assistant. Ought.org. https://elicit.com Epoch AI

arXiv doi:10.48550/arxiv.2305.14325 2023

[5] [9]

https://epoch.ai/data/notable-ai-models Accessed: 2024-11-25

Data on Notable AI Models. https://epoch.ai/data/notable-ai-models Accessed: 2024-11-25. Ege Erdil and Tamay Besiroglu

work page 2024

[6] [10]

Algorithmic Progress in Computer Vision. https://doi.org/10.48550/arXiv.2212.05153 arXiv:2212.05153 Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli Järviniemi, Matthew Barnett, Robert Sandler, Matej Vrzala, Jaime Sevilla, Qiuyu Ren...

arXiv doi:10.48550/arxiv.2212.05153

[7] [11]

https://doi.org/10.48550/arXiv.2411.04872 arXiv:2411.04872 Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V

FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI. https://doi.org/10.48550/arXiv.2411.04872 arXiv:2411.04872 Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang

arXiv doi:10.48550/arxiv.2411.04872

[8] [12]

InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Kate Larson (Ed.)

Large Language Model Based Multi- agents: A Survey of Progress and Challenges. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Kate Larson (Ed.). International Joint Conferences on Artificial Intelligence Organization, Jeju, South Korea, 8048–8057. https://doi.org/10.24963/ijcai.2024/890 Survey Track....

doi:10.24963/ijcai.2024/890