Pith. sign in

REVIEW 3 major objections 4 minor 61 references

Multi-agent systems outperform single-agent systems exactly when relay compression removes more irrelevant context than it loses task-relevant information, a condition the paper formalizes as G = H(M|m) − βΔ.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:13 UTC pith:KKGPIBKE

load-bearing objection Smart IB framing and a clean experimental protocol, but the headline 'prediction' is a definitional identity and the capability parameter β is never measured. the 3 major comments →

arxiv 2607.16133 v1 pith:KKGPIBKE submitted 2026-07-17 cs.LG cs.AI

When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

Wendi Yu , Lianhao Zhou , Xiangjue Dong , Sai Sudarshan Barath , Declan Staunton , Byung-Jun Yoon , Xiaoning Qian , James Caverlee
show 1 more author
Shuiwang Ji
This is my paper
classification cs.LG cs.AI
keywords multi-agent systemssingle-agent systemsinformation bottleneckrelay compressionLLM agentsmodel capabilitycontext reductionMAS gain
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the inconsistent, often contradictory results on whether multi-agent systems beat single-agent systems are not random: they follow from a single trade-off. Because a MAS passes only compressed relay messages between isolated workers, while a SAS keeps one shared context, the MAS advantage is determined by whether relay compression removes more upstream noise than it discards downstream-relevant information. The authors formalize this with an information bottleneck objective, giving a per-stage gain G_i = H(M_i|m_i) − βΔ_i(m_i). The effective parameter β represents downstream model capability: stronger models gain less from context reduction and suffer more from information loss. This yields testable predictions—MAS helps with near-sufficient relays, hurts or flips with insufficient relays—which the paper validates across five benchmarks and three model scales.

Core claim

The central claim is that multi-agent vs. single-agent performance differences reduce to a single information-bottleneck identity. Under unbounded relay bandwidth a MAS can simulate any SAS exactly, so the only structural difference is bounded relay compression. For each relay interface, MAS gain decomposes into H(M_i | m_i) − β Δ_i(m_i): the upstream context the relay removes, minus the capability-weighted downstream information it loses. Consequently MAS helps exactly when context reduction exceeds the weighted relay loss, and because β is assumed to grow with model capability, the same relay can help a weak model but hurt a strong one. The paper supports this with 18 controlled experiment

What carries the argument

The relay message m_i, an information bottleneck: each inter-worker relay compresses the upstream context M_i to a bounded message while trying to preserve the downstream target Y_{i+1}, the classical trade-off between compression and prediction. The paper attaches to this an effective parameter β that is interpreted as downstream model capability, converting the IB objective into a stage-wise gain formula; the Markov relation m_i ↔ M_i ↔ Y_{i+1} | X_{i+1} ensures the loss term is nonnegative and well-defined. The identity G_i = H(M_i|m_i) − βΔ_i(m_i) is what carries the argument. Its two terms—how much context the relay removes and how much task-relevant information it drops—and the β weigh

Load-bearing premise

The prediction that MAS gains shrink or reverse as models get stronger rests on the unmeasured assumption—flagged by the authors in Section 7—that the effective information-bottleneck weight β increases monotonically with model capability; if stronger models do not actually value preserving predictive information more, the capability-dependent reversal does not follow.

What would settle it

Run the same MAS-vs-SAS-contextflow comparison on a deliberately lossy relay (relay drops a known downstream-required field, as in the ablation) across a capability ladder; if the strongest model still shows a positive MAS gain, or if a near-sufficient relay shows a negative gain for the weakest model, the β-monotonicity and gain decomposition prediction would be contradicted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • For tasks whose natural decomposition admits a near-sufficient relay (e.g., locating an object, independent subqueries), MAS should reliably beat a single-agent baseline, with the gain largest for weak models.
  • For tasks where correct output depends on exact upstream details (e.g., dates, IDs, prior command outputs), bounded relays are predicted to hurt; the paper's WorkBench results show negative gains at all three model scales.
  • The framework predicts a capability-dependent reversal: the same relay can help a weak model and hurt a strong one, because β weights the information-loss term more heavily.
  • Decomposition alone is not the source of MAS gains: SAS-contextflow and SAS-Plan baselines show that removing context isolation removes the advantage, confirming relay compression is the operative mechanism.
  • MAS design becomes an optimization problem: relays should be built to drop upstream-irrelevant content while preserving downstream-relevant information; the paper suggests joint optimization of relay encoder and downstream task.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the β assumption: measure or estimate the effective β across model scales (e.g., by fitting the gain formula to data); if β does not increase monotonically with capability, the predicted reversal pattern would need revision.
  • The framework predicts that adaptive relays—learned to minimize the IB objective for a given downstream task—should recover positive MAS gains even in the WorkBench regime where fixed relays fail; this is a testable design consequence the paper only gestures at.
  • The same lens may explain other reported MAS phenomena, such as performance degrading with agent count: more hops mean more relay compressions, so cumulative βΔ loss can dominate any context-reduction benefit.
  • One could estimate the relay-complexity regime δ quantitatively (e.g., by measuring minimal description length of sufficient relays), turning the paper's qualitative benchmark grouping into a predictive metric.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops an information-bottleneck account of when multi-agent LLM systems outperform single-agent systems. It models a MAS as a chain of workers with isolated contexts connected by compressed relay messages, while a SAS accumulates a single shared context. Proposition 3.1 shows that with unbounded relay bandwidth any SAS can be simulated by a MAS. Under bounded relays, the authors define relay information loss Δ_i and derive a local MAS gain G_i = H(M_i|m_i) − βΔ_i(m_i) (Theorem 4.1), so MAS helps exactly when upstream context reduction exceeds capability-weighted relay loss. They assign each benchmark to a relay-complexity regime δ (Table 1) and compare SAS, SAS-contextflow, and MAS across five benchmarks and three model scales, reporting that MAS gains are consistently positive for near-sufficient relays and shrink or reverse for insufficient relays, especially for stronger models. Ablations show that decomposition alone (SAS-Plan) does not reproduce MAS gains and that deliberately degrading relay sufficiency hurts performance.

Significance. If validated, the framework would provide a mechanistic explanation for inconsistent MAS/SAS results and practical relay-design guidance. The formal part is clean and useful: the proof of Proposition 3.1 is sound under the finite-entropy assumption, and Theorem 4.1 is a correct algebraic decomposition. The controlled three-prototype design and the relay-sufficiency ablation (Table 4) are genuine strengths, and the paper is unusually transparent about its limitations. However, the headline prediction is currently underdetermined: β is not estimated and δ is qualitatively assigned. The experiments therefore demonstrate consistency with the framework rather than testing it. As a conceptual contribution the paper is valuable, but the strength of the empirical validation is overstated.

major comments (3)
  1. [§4.2, Eq. (5)–(8); §7] The central condition G_i > 0 ⇔ H(M_i|m_i) > βΔ_i(m_i) is a direct restatement of the definition of G_i: substituting Eq. (5) into Eq. (6) makes the equality hold for any β. The empirical content therefore resides entirely in (i) the assumption that β increases monotonically with base-model capability and (ii) the hand-assigned δ regimes in Table 1. The paper states in §7 that β is not estimated and δ is assigned by task-structure analysis. As a result, the predicted capability-dependent reversal can be rationalized post hoc for any observed pattern by adjusting β, and the experiments cannot falsify Eq. (8). To make the central claim testable, the authors should either provide an independent estimate of β (e.g., by fitting the IB objective to per-model behavior) or experimentally vary β/δ independently and show the predicted sign change.
  2. [§5.2, Table 2 vs. Table 6 and Figure 3] The ALFWorld gains in Table 2 (+0.194, +0.157, +0.023) match the few-shot columns in Table 6, not the zero-shot values plotted in Figure 3, which give +0.373, +0.179, and +0.007. The text in §5.2 cites Figure 3 in the same paragraph while reporting the few-shot numbers. This is a discrepancy that must be resolved: if Table 2 uses few-shot ALFWorld, the caption and text should say so; if Figure 3 is the referenced evidence, the gains should be the zero-shot ones. Without clarification, the consistency claim is weakened and raises the appearance of metric selection after seeing the results.
  3. [§5.2, Table 6; RQ1/RQ2] The empirical validation lacks error bars, confidence intervals, or significance tests, yet the text repeatedly uses 'consistently' and 'support the prediction.' Several gains are tiny relative to what the sample sizes can support: TravelPlanner-CS on Qwen2.5-7B is +0.011, TravelPlanner-HC on Qwen2.5-7B is +0.017, and WebShop on Qwen3.5-27B is −0.003. Moreover, the claimed monotonic shrinkage is not actually clean: TravelPlanner-CS gains are +0.011, +0.183, +0.028 across the capability axis, which is not a monotonic decrease. The authors should report uncertainty and per-task variance, and temper claims of 'consistent' support where the effects are within noise.
minor comments (4)
  1. [§3; §C.3; §D.1] Typos: 'empircal' (§3), 'capabiltiy' (§C.3), 'decompostion' (§D.1), and 'Workbench gpt-5 decompostion prompt' should be cleaned. Also standardize capitalization of 'WorkBench' vs 'Workbench.'
  2. [Eq. (10), Table 1] The relay-complexity δ is defined via a minimum description length L(·), but L(·) is never instantiated. Since δ is load-bearing for the benchmark categorization, give at least a concrete description-length proxy (e.g., token counts of sufficient relays) or state clearly that Table 1 is a qualitative judgment.
  3. [§5.1, Models] The capability axis mixes API models (GPT-4o-mini) with locally served models, and Qwen3.5-27B is AWQ 4-bit quantized. Please state whether quantization could affect the β interpretation, and ideally include a non-quantized strong baseline or a sensitivity check.
  4. [§6, Table 4] The relay-sufficiency ablation is a good causal check, but the magnitude of the effect varies widely (e.g., WebShop GPT-4o-mini −0.078 vs. ALFWorld −0.015, −0.015). A short discussion of why the cost of relay loss is benchmark-dependent would strengthen the connection to Theorem 4.1.

Circularity Check

2 steps flagged

The central MAS-gain prediction is an algebraic identity given the paper's definitions, and the capability-dependent reversal rests on an unmeasured β assumption.

specific steps
  1. self definitional [Theorem 4.1, Eqs. (3)-(8), Section 4.2]
    "GMAS i ≜L SAS i − LMAS i .(6) ... Theorem 4.1: ... GMAS i =H(M i |m i) −β ∆i(mi) ... Therefore, MAS is locally beneficial exactly when GMAS i >0⇐ ⇒H(M i |m i)>β∆ i(mi). (8)"

    Substituting Eq. (3), L_SAS_i(β)=H(M_i)−βI(M_i;Y_{i+1}|X_{i+1}), and Eq. (5), L_MAS_i(m_i;β)=I(M_i;m_i)+βΔ_i(m_i)−βI(M_i;Y_{i+1}|X_{i+1}), into definition (6) yields Eq. (7) using I(M_i;m_i)=H(M_i)−H(M_i|m_i). No assumption beyond the definitions is needed. The headline condition G_i>0 ⇔ H(M_i|m_i)>βΔ_i(m_i) therefore restates the definition of G_i as L_SAS−L_MAS. Because H(M_i|m_i), Δ_i, and β are not estimated empirically, the inequality is not independently testable; it is true by construction.

  2. fitted input called prediction [Section 4.1 'Effective β and LLM Capability'; Section 7 Limitations]
    "Weaker models are more sensitive to noisy context and thus favor stronger compression (smaller effectiveβ), whereas stronger models can better utilize rich context and therefore place higher value on preserving predictive information (larger effectiveβ). ... We useβ as a qualitative capability parameter rather than a directly estimated quantity."

    The paper's predicted capability-dependent reversal—the same relay helps weak models but hurts strong models—follows entirely from asserting that β grows monotonically with model capability. Since β is never measured or independently derived, the 'prediction' is a restatement of the asserted β-capability coupling, not a consequence of the IB framework. The empirical gain trends in Table 2 are then interpreted with this assumed β direction and with hand-assigned δ regimes, so the match does not provide an independent test of the trade-off in Theorem 4.1.

full rationale

The formal derivation of Theorem 4.1 is internally correct, but it is a tautology: Eq. (7) follows by substituting the definitions (3)-(6), and Eq. (8) is the same comparison rewritten. A purely algebraic decomposition is not itself circular; however, the paper presents Eq. (8) as the explanatory prediction and validates it with experiments in which the right-hand-side terms are not measured. The relay-loss regimes δ are hand-assigned by task-structure analysis (Table 1, §5.1) and β is explicitly stated to be qualitative (§7). Thus the empirical tables can only show that observed MAS-vs-SAS-contextflow differences are consistent with labels chosen for δ and a direction chosen for β; they cannot provide an independent check of the claimed trade-off. The controlled SAS-contextflow comparison and the relay-sufficiency ablation (§6) are genuine empirical contributions and support the qualitative claim that relay compression matters, but they do not identify H(M_i|m_i), Δ_i, or β. Because the headline prediction reduces to the definition of G_i and the capability dependence is loaded into an unmeasured β, I score this as partial circularity (6), not full circularity: the experimental setup and ablations contain independent content, and there is no reliance on self-citations or imported uniqueness theorems.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 1 invented entities

The paper's central theorem is a definition-level identity; the empirical content rests on hand-assigned δ regimes and an assumed monotone β-capability relationship. No numeric parameters are fitted to data, so the ledger captures the qualitative constructs that carry the explanatory burden.

free parameters (2)
  • β (effective capability parameter) = not estimated
    Controls the IB trade-off in Eq. (2) and Eq. (8); interpreted as downstream LLM capability, assumed larger for stronger models (§4.1). No quantitative estimator is provided; the paper admits it is qualitative (§7).
  • δ (benchmark relay complexity) = qualitative: ≈0, >0, ≫0
    Defined in Eq. (10) as the minimum description length of a sufficient relay, but assigned by task-structure analysis in Table 1 rather than computed from data; the paper admits δ is not quantitatively estimated (§7).
axioms (5)
  • domain assumption Agent contexts, relays, and downstream targets are stochastic variables with well-defined entropies and mutual informations.
    Needed throughout §4 for the IB formulation; presupposes LLM reasoning can be treated as an information source. Invoked without discussion of estimation.
  • standard math The relay m_i is a (possibly randomized) function of M_i, giving the Markov relation m_i ↔ M_i ↔ Y_{i+1} | X_{i+1}.
    Justified in Appendix A.2 via p(y|m,M,X)=p(y|M,X); this is the data-processing assumption that makes Δ_i ≥ 0.
  • ad hoc to paper The effective β increases monotonically with base model capability; stronger models 'can better utilize rich context'.
    Stated in §4.1 but never measured or derived; it is the load-bearing mechanism for the capability-dependent reversal predicted in Eq. (8).
  • ad hoc to paper Benchmark relay complexity δ takes qualitative values (≈0, >0, ≫0) assigned from task structure.
    Table 1 assigns each benchmark to a δ regime based on the authors' reading of task information structure; the paper itself flags that δ is not quantitatively estimated (§7).
  • domain assumption The SAS context trajectory has finite entropy at every step (used in Proposition 3.1).
    Assumption explicitly made in the statement of Proposition 3.1; needed for the infinite-bandwidth equivalence to hold with entropy-bounded relays.
invented entities (1)
  • Effective β as a capability parameter no independent evidence
    purpose: Models how downstream LLM capability shifts the balance between context reduction and relay information loss in the IB objective.
    No falsifiable quantitative handle is given outside the paper: β is not measured, and its monotone relationship with model capability is assumed and only qualitatively consistent with the empirical directions.

pith-pipeline@v1.3.0-alltime-deepseek · 38005 in / 9200 out tokens · 74605 ms · 2026-08-01T21:13:57.861642+00:00 · methodology

0 comments
read the original abstract

LLM powered multi-agent systems (MAS) have emerged as a promising paradigm for complex tasks. However, their advantages over single-agent systems (SAS) remain unclear, with performance varying inconsistently across settings. Here, we provide an information bottleneck perspective on elucidating the differences between MAS and SAS. Specifically, our key observation is that a SAS accumulates its full reasoning trace in one shared context, while a MAS uses isolated local contexts connected by bounded relay messages. We show that, under infinite relay bandwidth, any SAS can be simulated by a MAS that transmits the full upstream context. Thus, the nontrivial advantage of MAS arises under bounded relays, where compression introduces a fundamental trade-off: reducing redundant context can improve efficiency, but may also incur loss of task-relevant information. We formalize this trade-off as an information bottleneck controlled by an effective parameter $\beta$, which captures how the balance shifts with model capability, and shows that MAS gains arise when context reduction outweighs relay information loss. We conduct 18 controlled experiments across five benchmarks and three model scales to validate our theoretical studies. We observe that MAS consistently helps when relays are near-sufficient, especially for weaker models. In contrast, MAS gains shrink or reverse when relays incur information loss, especially for stronger models that can already extract useful information from redundant context and thus gain little from compression. Our study shows that multi-agent design is fundamentally an information-bottleneck optimization problem. This perspective explains when bounded inter-agent communication helps or hurts.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 12 linked inside Pith

  1. [1]

    Alemi, Ian Fischer, Joshua V

    Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. InInternational Conference on Learning Representations, 2017. URL https://openreview. net/forum?id=HyxQzBceg

    2017

  2. [2]

    On the reliability limits of llm-based multi-agent planning.arXiv preprint arXiv:2603.26993, 2026

    Ruicheng Ao, Siyang Gao, and David Simchi-Levi. On the reliability limits of llm-based multi-agent planning.arXiv preprint arXiv:2603.26993, 2026

    arXiv 2026

  3. [3]

    Gamegpt: Multi-agent collaborative framework for game development.arXiv preprint arXiv:2310.08067, 2023

    Dake Chen, Hanbin Wang, Yunhao Huo, Yuzhao Li, and Haoyang Zhang. Gamegpt: Multi-agent collaborative framework for game development.arXiv preprint arXiv:2310.08067, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Are more LLM calls all you need? towards the scaling properties of compound AI systems

    Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. Are more LLM calls all you need? towards the scaling properties of compound AI systems. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: //openreview.net/forum?id=m5106RRLgx

    2024

  5. [5]

    A survey on llm-based multi- agent system: Recent advances and new frontiers in application.arXiv preprint arXiv:2412.17481, 2024

    Shuaihang Chen, Yuanxing Liu, Wei Han, Weinan Zhang, and Ting Liu. A survey on llm-based multi- agent system: Recent advances and new frontiers in application.arXiv preprint arXiv:2412.17481, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. InThe Twelfth International Conference on Learning Representations, 2024. UR...

    2024

  7. [7]

    Atlas: Constraints-aware multi-agent collaboration for real-world travel planning.arXiv preprint arXiv:2509.25586, 2025

    Jihye Choi, Jinsung Yoon, Jiefeng Chen, Somesh Jha, and Tomas Pfister. Atlas: Constraints-aware multi-agent collaboration for real-world travel planning.arXiv preprint arXiv:2509.25586, 2025

    arXiv 2025

  8. [8]

    Tenenbaum, and Igor Mordatch

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. InForty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=zj7YuTE4t8

    2024

  9. [9]

    Single- agent or multi-agent systems? Why not both?arXiv preprint arXiv:2505.18286, 2025

    Mingyan Gao, Yanzi Li, Banruo Liu, Yifan Yu, Phillip Wang, Ching-Yu Lin, and Fan Lai. Single- agent or multi-agent systems? Why not both?arXiv preprint arXiv:2505.18286, 2025. URL https: //arxiv.org/abs/2505.18286

    Pith/arXiv arXiv 2025

  10. [10]

    Large language model based multi-agents: a survey of progress and challenges

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: a survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 8048–8057, 2024. 14 When Do Multi-Agent Systems Help? An Inform...

    2024

  11. [11]

    MetaGPT: Meta programming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, 20...

    2024

  12. [12]

    More agents is all you need.Transactions on Machine Learning Research, 2024

    junyou li, Qin Zhang, Yangbin Yu, QIANG FU, and Deheng Ye. More agents is all you need.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id= bgzUSZ8aeg

    2024

  13. [13]

    AI agents that matter.Transactions on Machine Learning Research, 2025

    Sayash Kapoor, Benedikt Stroebl, Zachary S Siegel, Nitya Nadgir, and Arvind Narayanan. AI agents that matter.Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview. net/forum?id=Zy4uFzMviZ

    2025

  14. [14]

    Towards a science of scaling agent systems.arXiv preprint arXiv:2512.08296, 2025

    Yubin Kim, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A Ali Heydari, Yao Yan, Zhihan Zhang, Yuchen Zhuang, Yun Liu, et al. Towards a science of scaling agent systems.arXiv preprint arXiv:2512.08296, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Efficient memory management for large language model serving with pagedattention

    WoosukKwon, ZhuohanLi, SiyuanZhuang, YingSheng, LianminZheng, CodyHaoYu, JosephGonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. InProceedings of the 29th symposium on operating systems principles, pages 611–626, 2023

    2023

  16. [16]

    Camel: Commu- nicative agents for "mind" exploration of large language model society.Advances in neural information processing systems, 36:51991–52008, 2023

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Commu- nicative agents for "mind" exploration of large language model society.Advances in neural information processing systems, 36:51991–52008, 2023

    2023

  17. [17]

    When single-agent with skills replace multi-agent systems and when they fail.arXiv preprint arXiv:2601.04748, 2026

    Xiaoxiao Li. When single-agent with skills replace multi-agent systems and when they fail.arXiv preprint arXiv:2601.04748, 2026

    arXiv 2026

  18. [18]

    Encouraging divergent thinking in large language models through multi-agent debate

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17889–...

    doi:10.18653/v1/2024.emnlp-main.992 2024

  19. [19]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of machine learning and systems, 6:87–100, 2024

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of machine learning and systems, 6:87–100, 2024

    2024

  20. [20]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023

    2023

  21. [21]

    ChatDev: Communicative agents for software development

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. ChatDev: Communicative agents for software development. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceed- ings of the 62nd Annual Meeting of the Association for Computational ...

    2024

  22. [22]

    Scaling large language model-based multi-agent collaboration

    Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Scaling large language model-based multi-agent collaboration. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=K3n5jPkrU6

    2025

  23. [23]

    Emergentcoordinationinmulti-agentlanguagemodels

    ChristophRiedl. Emergentcoordinationinmulti-agentlanguagemodels. InTheFourteenthInternational Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=SRn1MtMPRq

    2026

  24. [24]

    Understanding the information propagation effects of communication topologies in llm-based multi-agent systems

    Xu Shen, Yixin Liu, Yiwei Dai, Yili Wang, Rui Miao, Yue Tan, Shirui Pan, and Xin Wang. Understanding the information propagation effects of communication topologies in llm-based multi-agent systems. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 12358–12372, 2025

    2025

  25. [25]

    Alfred: A benchmark for interpreting grounded instructions for everyday tasks

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740–10749, 2020

    2020

  26. [26]

    Alfworld: Aligning text and embodied environments for interactive learning.arXiv preprint arXiv:2010.03768, 2020

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning.arXiv preprint arXiv:2010.03768, 2020

    Pith/arXiv arXiv 2010

  27. [27]

    Broadcast approach for the information bottleneck channel

    Avi Steiner and Shlomo Shamai Shitz. Broadcast approach for the information bottleneck channel. In 2019 IEEE International Conference on Microwaves, Antennas, Communications and Electronic Systems (COMCAS), pages 1–5, 2019. doi: 10.1109/COMCAS44984.2019.8958109

    arXiv 2019

  28. [28]

    Work- bench: a benchmark dataset for agents in a realistic workplace setting.arXiv preprint arXiv:2405.00823, 2024

    Olly Styles, Sam Miller, Patricio Cerda-Mardini, Tanaya Guha, Victor Sanchez, and Bertie Vidgen. Work- bench: a benchmark dataset for agents in a realistic workplace setting.arXiv preprint arXiv:2405.00823, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Deep learning and the information bottleneck principle

    Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In2015 IEEE Information Theory Workshop (ITW), pages 1–5, 2015. doi: 10.1109/ITW.2015.7133169

    arXiv 2015

  30. [30]

    Pereira, and William Bialek

    Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, pages 368–377, 1999. URL https://arxiv.org/abs/physics/0004057

    Pith/arXiv arXiv 1999

  31. [31]

    Single-agent LLMs outperform multi-agent systems on multi-hop reasoning under equal thinking token budgets.arXiv preprint arXiv:2604.02460, 2026

    Dat Tran and Douwe Kiela. Single-agent LLMs outperform multi-agent systems on multi-hop reasoning under equal thinking token budgets.arXiv preprint arXiv:2604.02460, 2026. URL https://arxiv.org/ abs/2604.02460

    Pith/arXiv arXiv 2026

  32. [32]

    Multi-agent collaboration mechanisms: A survey of llms.arXiv preprint arXiv:2501.06322, 2025

    Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D Nguyen. Multi-agent collaboration mechanisms: A survey of llms.arXiv preprint arXiv:2501.06322, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Tdag: A multi-agent framework based on dynamic task decomposition and agent generation.Neural Networks, 185:107200, 2025

    Yaoxiang Wang, Zhiyong Wu, Junfeng Yao, and Jinsong Su. Tdag: A multi-agent framework based on dynamic task decomposition and agent generation.Neural Networks, 185:107200, 2025

    2025

  34. [34]

    Widesearch: Benchmarking agentic broad info-seeking.arXiv preprint arXiv:2508.07999, 2025

    Ryan Wong, Jiawei Wang, Junjie Zhao, Li Chen, Yan Gao, Long Zhang, Xuan Zhou, Zuo Wang, Kai Xiang, Ge Zhang, et al. Widesearch: Benchmarking agentic broad info-seeking.arXiv preprint arXiv:2508.07999, 2025. 16 When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

    Pith/arXiv arXiv 2025

  35. [35]

    Autogen: Enabling next-gen LLM applications via multi-agent conversations

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen LLM applications via multi-agent conversations. InFirst Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=BAakY1hNKS

    2024

  36. [36]

    Travelplanner: A benchmark for real-world planning with language agents.arXiv preprint arXiv:2402.01622, 2024

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: A benchmark for real-world planning with language agents.arXiv preprint arXiv:2402.01622, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    Information bottleneck for a Rayleigh fading MIMO channel with an oblivious relay.Information, 12(4):155, 2021

    Hao Xu, Tianyu Yang, Giuseppe Caire, and Shlomo Shamai. Information bottleneck for a Rayleigh fading MIMO channel with an oblivious relay.Information, 12(4):155, 2021. URL https://doi.org/10. 3390/info12040155

    2021

  38. [38]

    Webshop: Towards scalable real-world web interaction with grounded language agents.Advances in Neural Information Processing Systems, 35: 20744–20757, 2022

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents.Advances in Neural Information Processing Systems, 35: 20744–20757, 2022

    2022

  39. [39]

    A survey of multi-ai agent collaboration: Theories, technologies and applications

    Xueqiang Zhang, Xiaofei Dong, Yiru Wang, Dan Zhang, and Feng Cao. A survey of multi-ai agent collaboration: Theories, technologies and applications. DEAI ’25, page 1875–1881, New York, NY, USA,

  40. [40]

    Autonomous agents for scientific discovery: Orchestrating scientists, language, code, and physics.arXiv preprint arXiv:2510.09901, 2025

    Lianhao Zhou, Hongyi Ling, Cong Fu, Yepeng Huang, Michael Sun, Wendi Yu, Xiaoxuan Wang, Xiner Li, Xingyu Su, Junkai Zhang, Xiusi Chen, Chenxing Liang, Xiaofeng Qian, Heng Ji, Wei Wang, Marinka Zitnik, and Shuiwang Ji. Autonomous agents for scientific discovery: Orchestrating scientists, language, code, and physics.arXiv preprint arXiv:2510.09901, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    Toward greater autonomy in materials discovery agents: Unifying planning, physics, and scientists.Transactions on Machine Learning Research, 2026

    Lianhao Zhou, Hongyi Ling, Keqiang Yan, Kaiji Zhao, Xiaoning Qian, Raymundo Arróyave, Xiaofeng Qian, and Shuiwang Ji. Toward greater autonomy in materials discovery agents: Unifying planning, physics, and scientists.Transactions on Machine Learning Research, 2026

    2026

  42. [42]

    put a heated egg in the fridge

    Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Daisy Zhe Wang, Zhenhailong Wang, Cheng Qian, Robert Tang, Heng Ji, et al. Multiagentbench: Evaluating the collaboration and competition of llm agents. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8580–8622, 2025. 17...

    2025

  43. [44]

    [schema unchanged]

    Research ONLY the Core / Permanent Range standard products for these spirits brands: Johnnie Walker, Smirnoff, as of June 2025. [schema unchanged]

    2025

  44. [45]

    Research ONLY the Core / Permanent Range standard products for these spirits brands: Chivas Regal, Absolut Vodka, as of June 2025

    2025

  45. [46]

    Workbench gpt-5 decompostion prompt You are an orchestrator for parallel web research

    Research ONLY the Core / Permanent Range standard products for these spirits brands: Grey Goose, Bacardi, as of June 2025. Workbench gpt-5 decompostion prompt You are an orchestrator for parallel web research. Two phases: decompose, synthesize. You DO NOT know the answers. Sub-agents find them by searching the web. Your job is to write RESEARCH INSTRUCTIO...

    2025

  46. [47]

    including X, Y, Z

    Enumerate units. List every atomic item the task names - each brand/person/country/event/year. Expand "including X, Y, Z" and "from A to B" into explicit lists. Call the count K

  47. [48]

    An axis is valid only if each bucket hits a DIFFERENT primary source

    Pick axis (priority: entity > region > period > attribute). An axis is valid only if each bucket hits a DIFFERENT primary source. If two buckets would open the same main page, pick another axis

  48. [49]

    Drop nothing

    Partition into 3 buckets: 28 When Do Multi-Agent Systems Help? An Information Bottleneck Perspective - K > 3: ceil(K/3) or floor(K/3) units per bucket; group units that share a source (same parent company, same country). Drop nothing. - K = 2 or 3: one unit per bucket; if K = 2, split the source-densest unit along a sub-axis for the 3rd bucket. - K = 1: s...

  49. [50]

    prompt" field MUST be an imperative research instruction beginning with

    Each subtask "prompt" field MUST be an imperative research instruction beginning with "Research...", "Find...", or "Compile...". It must (a) explicitly NAME the assigned units, (b) list the required columns/fields, (c) copy all global constraints (dates, geography, formatting, exclusions, "do not ask questions"). NEVER write answer values (dates, specs, n...

    2023

  50. [55]

    Invalid action format

    Never loop on "think:" - always attempt the next concrete action. Webshop You are a shopping agent navigating WebShop to find and purchase a product that exactly matches the given instruction. The ONLY valid actions are: search[query] - search for products; only valid on the home page click[button] - click a button, product ASIN, option value, or sub-page...

    2023

  51. [56]

    You MUST output a report before the budget runs out

    Your total step budget is 25. You MUST output a report before the budget runs out

  52. [57]

    a saltshaker 1

    Never loop on "think:" - always attempt the next concrete action. You are now in a household environment. Your task is to find objects. CRITICAL: Every object and receptacle has a NUMBER in its name. You MUST always include this number. CORRECT: go to drawer 2 WRONG: go to drawer The number comes from what you see in observations (e.g. "a saltshaker 1", "...

  53. [60]

    look at X under the Y

    For "look at X under the Y" tasks, you need to find where both X and Y are located. executor You are an execution agent in a household environment. In this contextflow run there is no separate inbox: read the explorer's`<report>`from the conversation history above, then act. Think step-by-step before each action. Go directly to the reported location - do ...

  54. [65]

    Never loop on "think:" - always attempt the next concrete action

  55. [66]

    delete all X's leads

    You MUST take the object before you can move it. "move" will fail if you are not carrying the object. Webshop. worker: searcher You are a shopping search agent for WebShop. Your job is to collect structured candidate information from WebShop. Do NOT purchase anything. The ONLY valid actions are: search[query] - search for products; only valid on the home ...

    2023

  56. [67]

    go to <receptacle N>

    NAVIGATION: Always "go to <receptacle N>" before interacting with it. "Nothing happens." means you are not there yet

  57. [68]

    CARRY LIMIT: You can hold only ONE object at a time

  58. [69]

    move <object N> to <receptacle N>

    PLACING: Use "move <object N> to <receptacle N>" to place objects

  59. [70]

    look_at_obj tasks: take the object, go to the desklamp, then use the desklamp

  60. [71]

    from A to B

    Never loop on "think:" - always attempt the next concrete action. Webshop You navigate WebShop in ONE trajectory with two behavioural phases aligned to hetero verifier_met_failures10 (single agent combining searcher + verifier roles). === PHASE 1 - SEARCHER === You are a shopping search agent for WebShop. Your job is to collect structured candidate inform...

    2022

  61. [2025]

    ISBN 9798400712791

    Association for Computing Machinery. ISBN 9798400712791. doi: 10.1145/3745238.3745531. URL https://doi.org/10.1145/3745238.3745531

    arXiv