Pith. sign in

REVIEW 2 major objections 6 minor 56 references

CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a sparse, dynamically pruned debate graph, in which each LLM agent hears only from opponents scored as helpful by a McKinsey Trust Formula weight, improves accuracy and shortens contexts at once.

desk verdict Sparse-debate idea is worth a look, but the algorithm divides by zero in round 1 and the headline numbers rest on an unstated convention. read the letter →

arxiv 2507.03928 v1 pith:YXCA6UIJ submitted 2025-07-05 cs.AI cs.MA

Yiliu Sun , Zicheng Zhao , Sheng Wan , Chen Gong This is my paper
classification cs.AIcs.MA
keywords multi-agentdebatesparsedebatinggraphMcKinseyTrustFormulaLLMreasoningcontextlengthreductionoverconfidencemitigationdynamicpruningmajorityvoting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the standard multi-agent debate design—every LLM agent reading every other agent's answer—is the wrong default. CortexDebate instead builds a directed sparse debate graph and re-weights its edges each round using a 'McKinsey-based Debate Matter' module that scores how much one agent's input helps another. The reported result is that this pruned debate beats all baselines on all eight datasets, with gains of 2.33 to 5.33 absolute points over the best baseline per dataset, while cutting per-agent input context by up to 70.79 percent. If the result holds, the bottleneck in multi-agent debate is not too little talking but too much: context overload and overconfident dominance are what drag accuracy down.

What carries the argument

The load-bearing object is the McKinsey-based Debate Matter (MDM) module, an artificial stand-in for the brain's white matter that scores every directed edge with the McKinsey Trust Formula $W^d_{i\to j} = C^d \times R^d \times I^d / S^d$. The critical operation is the average-incoming-weight threshold of Equation (11): an agent's debate opponents in a round are exactly the incoming edges whose trust weight is at least the average of all its incoming weights, so the graph is sparse and re-shaped every round. The intimacy factor $I^d$ and the self-orientation factor $S^d$ carry the paper's claim of 'equal' debate, because they make an edge depend on viewpoint collision and participation rather than on the source agent's self-reported confidence alone.

What would settle it

Run Algorithm 1 literally on MATH with Equations (8) through (11): at round 1 every edge weight is $C\times R\times I / 0$, i.e. undefined, so the stated algorithm cannot produce the reported result unless a special-case initialization is silently added; recording that single fact settles whether the published equations describe the evaluated system.

Watch

Extended reading notes

Core claim

CortexDebate's central discovery is that a dynamically pruned, directed debate graph yields both higher result accuracy and shorter contexts than debating with everyone. Each edge from agent $i$ to agent $j$ carries a weight $W^d_{i\to j} = C^d \times R^d \times I^d / S^d$, where $C^d$ is the source agent's credibility estimated from a scaling-law loss, $R^d$ is its average historical self-confidence, $I^d$ is the average viewpoint difference ($1$ minus cosine similarity of their past answers), and $S^d$ is its debate participation count. In every round, the edges pointing into each agent whose weight falls below that agent's average incoming weight are deleted, and the agent reads only the surviving answers. The paper reports that this 'debate with whoever helps you' rule outperforms the best baseline on each of eight datasets, e.g. 79.33% versus 77.00% on GSM-IC, 36.33% versus 32.67% on GPQA, and 60.31% versus 55.97% on LongBench, while reducing average per-agent input length to between 2,413 and 11,966 tokens across datasets, a maximum reduction of 70.79% against full-debate baselines.

Load-bearing premise

The load-bearing assumption is that the edge-weight equations are computable in every round, but Equation (9) divides by $S_d=(d-1)(n-1)-P_d$, which is zero at $d=1$, so the paper never actually specifies how the first-round graph is built.

Editorial extensions

If this is right

  • If CortexDebate is right, fully connected multi-agent debate is not merely expensive but counterproductive: pruning the lowest-trust incoming edges improves accuracy on every one of the eight datasets while shortening contexts.
  • A maximum context reduction of 70.79% (and at least 17.62% over part-debate baselines) means the method can scale to more agents and more rounds before the 'lost in the middle' context overload that motivated the sparse design sets in.
  • The per-agent score gains and the shortening of context hold across math, world-knowledge QA, reasoning, and long-context tasks, so the benefit is not tied to a single benchmark family.
  • Increasing the number of agents and rounds continues to improve accuracy under the sparse graph, suggesting the approach is compatible with large-scale debates rather than being a small-ensemble trick.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the paper only compares the average threshold to top-3, bottom-3, and median pruning on MATH; a different task mix could prefer a different pruning rule, so the average threshold is a design choice, not a demonstrated optimum.
  • Inference: if the intimacy term is what gives 'equal' debate its power, then the method should show the largest advantage on tasks where answers are verbose and partially correct, and the smallest advantage on single-answer verification tasks; that task-by-task prediction is not tested in the paper.
  • Inference: because the trust weights are updated from debate history, freezing the graph after the first round would separate the benefit of sparsity from the benefit of adaptation; the paper does not run this control.
  • Inference: the McKinsey formula supplies the shape of the edge weight, but nothing in the paper proves these four factors are the minimal sufficient statistics of helpfulness; a simpler weight such as source accuracy times viewpoint diversity might reproduce the results at lower engineering cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The manuscript proposes CortexDebate, a multi-agent debate method that maintains a sparse directed graph among LLM agents. Edge weights are computed with a McKinsey Trust Formula analog based on credibility, reliability, intimacy, and self-orientation; edges below the per-receiver average are pruned each round to reduce input context and avoid domination by overconfident agents. The authors report experiments on eight datasets across four task types, claiming the highest accuracy and the largest context-length reduction versus full-debate and static-partial-debate baselines, with ablations isolating the contributions of the sparse graph and the MDM module.

Significance. If the algorithm were fully specified, the paper would make a useful contribution to multi-agent debate: the dynamic sparsification idea is well motivated, the reported context-length reductions are substantial (up to 70.79% relative to PRD on ARC-C), and the main table shows consistent gains over the best baseline on every dataset (e.g., +5.33 on MMLU-pro, +4.34 on LongBench). The 1000-example experiments in Appendix C reproduce the ranking of Table 1, the ablation study in Table 2 attributes clear value to both the sparse graph and the full MDM factors, and Table 4 provides direct evidence that retained edges are helpful to individual agents. The paper also supplies prompts for all methods. However, the central algorithm is formally undefined as written because the edge-weight formula divides by zero in round 1, and one free parameter is selected on the evaluation datasets; both issues must be resolved before the empirical claims can be accepted.

major comments (2)
  1. [Section 4.2, Eqs. (8)-(9), and Algorithm 1] The edge-weight computation is undefined in round 1. Algorithm 1 initializes P_i^0 to 0 (line 5) and never updates P_i in any later line. With S_d = (d-1)(n-1) - P_d from Eq. (8), at d=1 we obtain S_1=0, and Eq. (9) divides by zero for every directed edge. No epsilon, additive constant, or special case for round 1 is stated, so the graph construction in Eqs. (10)-(12) has no values in the very round that determines the initial sparse graph. Because Algorithm 1 does not update P_i, S_d is also unspecified for d>1, and S_d=0 recurs whenever P_d=(d-1)(n-1), which the pruning rule in Eq. (11) can produce when all incoming weights tie. The reported accuracy and context-length results therefore depend on an unstated convention, and the algorithm as written cannot be executed.
  2. [Section 5.3 and Appendix D.3 (Table 11)] The confidence recalibration thresholds in Eq. (2) are selected on the evaluation datasets. Appendix D.3 reports average scores on the eight adopted datasets for six threshold configurations and identifies [0.8, 0.6, 0.3] as "Ours" because it achieves the best average score. This is test-set hyperparameter selection for a component of the method, which weakens the claim in Section 5.2 that CortexDebate "performs stably on all adopted datasets." The authors should either use a held-out validation split for choosing thresholds or present the sensitivity analysis as a post-hoc robustness check without treating the winning configuration as the method's default.
minor comments (6)
  1. [Algorithm 1, lines 30-31] The assignment on line 30 uses O^d_i and H^d_i on the left-hand side while the call is A_j(Q, Others^d_j); it should be O^d_j and H^d_j, and the recalibration on line 31 should also use H^d_j.
  2. [Appendix B and Appendix C] The text refers to a baseline "MPRC" ("MLD, RECONCILE, ChatEval, and MPRC") that is not defined anywhere; the intended name appears to be PRD (Peer Review Debate).
  3. [Section 3.1] The phrase "with A_j as the tail node" is inconsistent with standard directed-edge terminology: for an edge E_{i->j}, A_j is the head, not the tail.
  4. [Figure 5] In the provided manuscript, Figure 5 contains uninterpretable "/uni..." glyph sequences in place of axis labels; the figure must be regenerated with readable text before publication.
  5. [Eq. (2)] Equation (2) uses H^0_i on both sides for the raw and recalibrated confidence values; a distinct notation for the recalibrated value would avoid confusion, especially since the recalibration is applied again in Algorithm 1.
  6. [Eq. (8)] The notation P_d is used for an individual agent's participation count, yet the subscript d also denotes the debate round; the paper should use a per-agent symbol such as P_i^d throughout, and Algorithm 1 should show how P_i^d is updated after each debate round.

Circularity Check

0 steps flagged · score 2.0 of 10

Derivation is empirical, not circular: trust weights and pruning are computed from agent outputs, accuracy and context results are external benchmark claims; the single self-citation is a metric reference, and the round-1 division by zero in Eq. (9) is a correctness gap, not circularity.

full rationale

The central derivation chain—Eqs. (3)-(12) in Section 4.2—computes edge weights W^d_{i->j} from the agents' own credibility, reliability, intimacy, and self-orientation, then prunes edges below the per-receiver average; the resulting sparse graph is an algorithmic mechanism, not a restatement of the reported accuracies or context lengths. Tables 1 and 7 compare CortexDebate against external baselines on standard datasets, so the main claims are empirical and falsifiable rather than consequences of a definition. The only author-overlapping citation, Sun et al. (2025), is used in Section 5.1 merely as one of several references for the Result Accuracy metric; it is not load-bearing. A genuine issue exists in the manuscript's own equations: at d=1, Eq. (8) gives S_1=0 given P_1=0 from Algorithm 1, and Eq. (9) then divides by zero for every edge, with no stated initialization or special case; this is an omitted-definition and reproducibility flaw, not a circular reduction, so it is recorded here but does not raise the circularity score. Similarly, the recalibration thresholds in Table 11 are selected by comparing configurations on the adopted datasets, which is a potential test-set-informed design concern rather than a circular derivation.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The central claim rests on borrowed constants (the Chinchilla scaling-law coefficients from Hoffmann et al. (2022) and the McKinsey trust formula structure) and on several behavioral assumptions about LLM agents that are asserted rather than independently validated: that recalibrated self-confidence tracks correctness, that answer-text cosine similarity captures productive viewpoint difference, that participation level reflects self-orientation, and that agents actually update from peer answers. The one quantity visibly fitted to the evaluation data is the recalibration threshold triple, selected on the benchmark test sets reported in Table 11. The round-1 degeneracy of the self-orientation term means an unstated implementation detail is also load-bearing. No physical entities are postulated; the only invented component is the MDM module, which is fully specified by equations and auditable.

free parameters (2)
  • Confidence recalibration thresholds = [0.8, 0.6, 0.3]
    Equation (2) maps raw confidence scores into three bins. Appendix D.3 (Table 11) selects this triple as best-performing among six configurations evaluated on the same eight benchmark test sets used for the headline results, so the final reported numbers include test-set-informed tuning.
  • Per-model pretraining token count M in credibility term = unspecified
    Equation (3) requires M for each backbone to compute credibility C_d = 1/L(N,M). The paper never states the token counts used for Qwen-2.5-7B, Mistral-7B, Typhoon-1.5-8B, Llama-3.1-8B, or Gemma-2-9B; C_d is a fixed per-agent constant that weights every edge, and without M the credibility component cannot be verified or reproduced.
assumptions (6)
  • domain assumption The Chinchilla scaling law (Hoffmann et al., 2022) with coefficients 406.4, 410.7, and 1.69 estimates the relative competence of the five instruct-tuned backbones from parameter count and pretraining tokens.
    Section 4.2 Step 1 uses this law in Equations (3)-(4) to define credibility; the transfer of coefficients from the original training runs to chat-tuned checkpoints is assumed without validation.
  • domain assumption The McKinsey trust formula T = C x R x I / S, designed for trust between people in organizations, transfers to LLM agents as a measure of debate usefulness.
    Section 3.2 presents the formula and Section 4.2 maps its four factors onto agent properties; the mapping and the transfer are asserted, and no validation against external trust judgments is given.
  • domain assumption Cosine similarity between two agents' answer texts is a valid measure of viewpoint difference, with larger differences making debate more productive.
    Equations (6)-(7) define intimacy as one minus the cosine similarity of answers; the assumption that text similarity corresponds to debate-relevant viewpoint difference is stated without external support.
  • domain assumption Self-reported confidence scores, after the recalibration in Equation (2), are informative about answer correctness and stable enough to serve as a reliability signal over rounds.
    Equations (2) and (5) build the reliability term from recalibrated self-confidence; the recalibration practice is cited to Chen et al. (2023), but the specific mapping and its stability are assumed.
  • domain assumption Agents follow the answer-regeneration prompt of Equation (13), read the supplied peer answers, and update their answers accordingly.
    Phase 2 Step 3 (Section 4.2) and Appendix F assume instruction-following behavior; if agents ignore peer answers, the graph structure has no causal effect on the outcome.
  • ad hoc to paper S_d in Equation (8) is nonzero in every round so that the edge weight in Equation (9) is finite.
    At d=1, Equation (8) gives S_1 = 0 with the initialization P_1 = 0 in Algorithm 1, so Equation (9) divides by zero; the paper relies on an unstated initialization, epsilon, or special case without which the algorithm is undefined in round 1.
invented entities (1)
  • MDM (McKinsey-based Debate Matter) independent evidence
    purpose: Edge-weight evaluation module that assigns a trust score to each directed debate link and guides the per-round pruning of the debate graph.
    MDM is an algorithmic component fully specified by Equations (3)-(9), not a physical postulate; its outputs are computable and auditable, and the paper's ablations (Tables 2, 3, 9, 10, and 11) provide falsifiable checks on its behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate." pith.science (2026). https://pith.science/paper/YXCA6UIJ

@misc{pith2026250703928,
  author       = {Pith},
  title        = {Pith review of: CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YXCA6UIJ}},
  note         = {Machine review of arXiv:2507.03928}
}
read the original abstract

Nowadays, single Large Language Model (LLM) struggles with critical issues such as hallucination and inadequate reasoning abilities. To mitigate these issues, Multi-Agent Debate (MAD) has emerged as an effective strategy, where LLM agents engage in in-depth debates with others on tasks. However, existing MAD methods face two major issues: (a) too lengthy input contexts, which causes LLM agents to get lost in plenty of input information and experiences performance drop; and (b) the overconfidence dilemma, where self-assured LLM agents dominate the debate, leading to low debating effectiveness. To address these limitations, we propose a novel MAD method called "CortexDebate". Inspired by the human brain's tendency to establish a sparse and dynamically optimized network among cortical areas governed by white matter, CortexDebate constructs a sparse debating graph among LLM agents, where each LLM agent only debates with the ones that are helpful to it. To optimize the graph, we propose a module named McKinsey-based Debate Matter (MDM), which acts as an artificial analog to white matter. By integrating the McKinsey Trust Formula, a well-established measure of trustworthiness from sociology, MDM enables credible evaluations that guide graph optimization. The effectiveness of our CortexDebate has been well demonstrated by extensive experimental results across eight datasets from four task types.

Figures

Figures reproduced from arXiv: 2507.03928 by the authors.

Figure 1
Figure 1. Shortcomings of the existing MAD methods: (a) Debating with all others causes lengthy contexts input [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed CortexDebate, which is inspired by the working mode of human brain cortex [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison results of average length of context input to the LLM agents on eight datasets. We reflect the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Task performance of CortexDebate under different LLM agent numbers and debating rounds. edge weights of the debating graph, including self￾evaluation (Chen et al., 2023), peer evaluation (Xu et al., 2023b), MDM (w/o Id and Sd in Equa￾tion (9)), and MDM (see Appendix E …
Figure 6
Figure 6. Figure 6: Prompt of Multi-LLM Debate used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Debating summary prompt of ChatEval used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Prompt of RECONCILE used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Prompt of Peer Review Debate used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Prompt of GroupDebate Debate used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Prompt of Neighbor Debate used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 35 canonical work pages

  1. [1]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. 2023. Longbench: A bilingual, multitask benchmark for long context understanding. In Annual Meeting of the Association for Computational Linguistics

    work page 2023

  2. [2]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In AAAI Conference on Artificial Intelligence

    work page 2024

  3. [3]

    Jonah Brown-Cohen, Geoffrey Irving, and Georgios Piliouras. 2023. Scalable ai safety via doubly-efficient debate. In International Conference on Machine Learning

    work page 2023

  4. [4]

    Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. In International Conference on Learning Representations

    work page 2023

  5. [5]

    Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2023. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. arXiv preprint arXiv:2309.13007

    arXiv 2023

  6. [6]

    Steffi Chern, Ethan Chern, Graham Neubig, and Pengfei Liu. 2024. Can large language models be trusted for evaluation? scalable meta-evaluation of llms as evaluators via agent debate. arXiv preprint arXiv:2401.16788

    arXiv 2024

  7. [7]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    arXiv 2018

  8. [8]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

Show all 56 references
  1. [9]

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. In International Conference on Machine Learning

    2023

  2. [10]

    Zhihua Duan and Jialin Wang. 2024. Enhancing multi-agent consensus through third-party llm integration: Analyzing uncertainty and mitigating hallucinations in large language models. arXiv preprint arXiv:2411.16189

    2024 arXiv

  3. [11]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  4. [12]

    Yong Guan, Hao Peng, Lei Hou, and Juanzi Li. 2025. Mmd-ere: Multi-agent multi-sided debate for event relation extraction. In Proceedings of the 31st International Conference on Computational Linguistics

    2025

  5. [13]

    Zhitao He, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Jiexin Xu, Huaijun Li, Kang Liu, and Jun Zhao. 2024. Agentscourt: Building judicial decision-making agents with court debate simulation and legal knowledge augmentation. In Findings of the Association for Computatio...

    2024

  6. [14]

    Mahmood Hegazy. 2024. Diversity of thought elicits stronger reasoning capabilities in multi-agent debate frameworks. arXiv preprint arXiv:2410.12853

    2024

  7. [15]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. In International Conference on Learning Representations

    2020

  8. [16]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. Advances in Neural Information Processing Systems

    2021

  9. [17]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. Advances in Neural Information Processing Systems

    2022

  10. [18]

    Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. 2024. Routerbench: A benchmark for multi-llm routing system. arXiv preprint arXiv:2403.12031

    2024 arXiv

  11. [19]

    Zhe Hu, Hou Pong Chan, Jing Li, and Yu Yin. 2025. Debate-to-write: A persona-driven multi-agent framework for diverse argument generation. In International Conference on Computational Linguistics

    2025

  12. [20]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  13. [21]

    Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R Bowman, Tim Rockt \"a schel, and Ethan Perez. 2024. Debating with more persuasive llms leads to more truthful answers. In International Conference on Machine Learning

    2024

  14. [22]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems

    2022

  15. [23]

    Eric Lamarre, T Mansour, and J Tetrault. 2012. Mckinsey on cooperatives

    2012

  16. [24]

    Renhao Li, Minghuan Tan, Derek F Wong, and Min Yang. 2024 a . Coevol: Constructing better responses for instruction finetuning through multi-agent cooperation. In Conference on Empirical Methods in Natural Language Processing

    2024

  17. [25]

    Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ie. 2024 b . Improving multi-agent debate with sparse communication topology. In Findings of the Association for Computational Linguistics: EMNLP 2024

    2024

  18. [26]

    Jingcong Liang, Rong Ye, Meng Han, Ruofei Lai, Xinyu Zhang, Xuanjing Huang, and Zhongyu Wei. 2024. Debatrix: Multi-dimensinal debate judge with iterative chronological analysis based on llm. In Findings of the Association for Computational Linguistics: ACL 2024

    2024

  19. [27]

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. In Annual Meeting Of The Association For Computational Linguistics

    2023

  20. [28]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024 a . Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157--173

    2024

  21. [29]

    Tongxuan Liu, Xingyu Wang, Weizhe Huang, Wenjiang Xu, Yuting Zeng, Lei Jiang, Hailong Yang, and Jing Li. 2024 b . Groupdebate: Enhancing the efficiency of multi-agent debate using group discussion. arXiv preprint arXiv:2409.14051

    2024

  22. [30]

    Linhao Luo, Zicheng Zhao, Chen Gong, Gholamreza Haffari, and Shirui Pan. 2024. Graph-constrained reasoning: Faithful reasoning on knowledge graphs with large language models. arXiv preprint arXiv:2410.13080

    2024 arXiv

  23. [31]

    Julian Michael, Salsabila Mahdi, David Rein, Jackson Petty, Julien Dirani, Vishakh Padmakumar, and Samuel R Bowman. 2023. Debate helps supervise unreliable experts. arXiv preprint arXiv:2311.08702

    2023 arXiv

  24. [32]

    Chau Pham, Boyi Liu, Yingxiang Yang, Zhengyu Chen, Tianyi Liu, Jianbo Yuan, Bryan A Plummer, Zhaoran Wang, and Hongxia Yang. 2023. Let models speak ciphers: Multiagent debate through embeddings. In International Conference on Learning Representations

    2023

  25. [33]

    Kunat Pipatanakul, Phatrasek Jirabovonvisut, Potsawee Manakul, Sittipong Sripaisarnmongkol, Ruangsak Patomwong, Pathomporn Chokchainant, and Kasima Tharnpipitchai. 2023. Typhoon: Thai large language models. arXiv preprint arXiv:2312.13951

    2023 arXiv

  26. [34]

    P Rajpurkar. 2016. Squad: 100,000+ questions for machine comprehension of text. In Conference on Empirical Methods in Natural Language Processing

    2016

  27. [35]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022

    2023 arXiv

  28. [36]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Sch \"a rli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning

    2023

  29. [37]

    Qiushi Sun, Zhangyue Yin, Xiang Li, Zhiyong Wu, Xipeng Qiu, and Lingpeng Kong. 2024 a . Corex: Pushing the boundaries of complex reasoning through multi-model collaboration. In ICLR 2024 Workshop on Large Language Model (LLM) Agents

    2024

  30. [38]

    Xiaoxi Sun, Jinpeng Li, Yan Zhong, Dongyan Zhao, and Rui Yan. 2024 b . Towards detecting llms hallucination via markov chain-based multi-agent debate framework. arXiv preprint arXiv:2406.03075

    2024 arXiv

  31. [39]

    Yiliu Sun, Yanfang Zhang, Zicheng Zhao, Sheng Wan, Dacheng Tao, and Chen Gong. 2025. Fast-slow-thinking: Complex task solving with large language models. arXiv preprint arXiv:2504.08690

    2025 arXiv

  32. [40]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  33. [41]

    Michel Thiebaut de Schotten and Stephanie J Forkel. 2022. The emergent properties of the connected brain. Science, 378(6619):505--510

    2022

  34. [42]

    Lifu Tu, Semih Yavuz, Jin Qu, Jiacheng Xu, Rui Meng, Caiming Xiong, and Yingbo Zhou. 2023. Unlocking anticipatory text generation: A constrained approach for faithful decoding with large language models. In Conference on Empirical Methods in Natural Language Processing

    2023

  35. [43]

    Fanqi Wan, Longguang Zhong, Ziyi Yang, Ruijun Chen, and Xiaojun Quan. 2024. Fusechat: Knowledge fusion of chat models. arXiv preprint arXiv:2408.07990

    2024 arXiv

  36. [44]

    Haotian Wang, Xiyuan Du, Weijiang Yu, Qianglong Chen, Kun Zhu, Zheng Chu, Lian Yan, and Yi Guan. 2025. Learning to break: Knowledge-enhanced reasoning in multi-agent debate system. Neurocomputing, 618:129063

    2025

  37. [45]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations

    2022

  38. [46]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems

    2024

  39. [47]

    Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin. 2023. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Findings of the Association for Computational Linguistics: EMNLP 2023

    2023

  40. [48]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023 a . Recomp: Improving retrieval-augmented lms with compression and selective augmentation. In International Conference on Learning Representations

    2023

  41. [49]

    Zhenran Xu, Senbao Shi, Baotian Hu, Jindi Yu, Dongfang Li, Min Zhang, and Yuxiang Wu. 2023 b . Towards reasoning in large language models via multi-agent peer review collaboration. arXiv preprint arXiv:2311.08152

    2023 arXiv

  42. [50]

    Ziyi Yang, Fanqi Wan, Longguang Zhong, Tianyuan Shi, and Xiaojun Quan. 2024. Weighted-reward preference optimization for implicit model fusion. arXiv preprint arXiv:2412.03187

    2024 arXiv

  43. [51]

    Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. Exchange-of-thought: Enhancing large language model capabilities through cross-model communication. In Conference on Empirical Methods in Natural Language Processing

    2023

  44. [52]

    Luke Yoffe, Alfonso Amayuelas, and William Yang Wang. 2024. Debunc: mitigating hallucinations in large language model agent communication with uncertainty estimations. arXiv preprint arXiv:2407.06426

    2024 arXiv

  45. [53]

    Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2024 a . Exploring collaboration mechanisms for llm agents: A social psychology view. In ICLR 2024 Workshop on Large Language Model (LLM) Agents

    2024

  46. [54]

    Mingqing Zhang, Haisong Gong, Qiang Liu, Shu Wu, and Liang Wang. 2024 b . Breaking event rumor detection via stance-separated multi-agent debate. arXiv preprint arXiv:2412.04859

    2024 arXiv

  47. [55]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.