Pith. sign in

REVIEW 3 major objections 5 minor 39 references

EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revision

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a deployed multi-agent system can keep improving after launch by converting recurring execution failures into validated, persistent edits to its own prompts and topology, while the underlying language-model weights…

desk verdict A useful mechanism for MAS evolution with an evaluation protocol that overstates deployability—the headline gains are Test-selected checkpoints, not a fixed policy. read the letter →

arxiv 2608.07196 v1 pith:KF65C3IU submitted 2026-08-07 cs.AI

Chao Fei , Qingyi Si , Kaihua Liang , Yanghua Xiao , Panos Kalnis , Hongcheng Guo This is my paper · ORCID
classification cs.AI
keywords multi-agentsystemssystem-levelevolutionpromptoptimizationtopologyrevisionrecursiveself-improvementexecutiontracesvalidationgatingtokencostreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes EMAS, a procedure that lets a deployed multi-agent system revise its own prompts and topology using the traces of its own executions, while the underlying LLM stays frozen. The claim is that recurring, sample-specific failures can be turned into persistent, validated system edits: a mistake must repeat across several tasks before a candidate change is generated, and the change is committed only if it beats the current system on a paired validation set. Across four benchmarks and two frozen LLMs, EMAS reports task-weighted accuracy gains of 6.30% and 20.10% within two evolution epochs, and a 62.2% token reduction on MBPP with Qwen3.6-27B. The paper's point is that system-level evolution, editing structure and instructions rather than weights, is a viable route to continued improvement after deployment.

What carries the argument

The central object is the step-level graph of the MAS, where each node is one atomic LLM call and each edge transfers an artifact, together with the diagnosis record $h=(o,d,a,\ell)$ that maps a trace outcome to an objective, a defect class, a permitted operation, and a canonical location. Revision candidates are gated by an operation-specific support threshold $\kappa_a$, with additions requiring fewer supporting samples than removals because removals risk destroying needed computation, and every candidate must clear a paired validation rule on a fixed, balanced validation set before becoming version $v+1$. This converts evolution into a revision-authorization problem: the system decides which edits are allowed to shape future behavior.

What would settle it

Deploy EMAS with a hard rule that always promotes the most recently accepted revision, with no retrospective checkpoint lookup, and measure accuracy on the same held-out test tasks; if that accuracy falls far below the reported best-checkpoint numbers, as the 39 regressing revisions among 93 suggest it would, then the headline gains are not attainable without a selection rule the paper does not supply.

Watch

Extended reading notes

Core claim

EMAS claims that the executable layer of a multi-agent system, the graph of LLM-call steps, the edges that carry artifacts between them, and the prompts that govern each step, is a body of reusable state that can be improved from experience just as weights are improved by training. Execution traces are converted into structured diagnoses that name an objective (accuracy or cost), a defect class, a revision operation, and a canonical location; a candidate revision is proposed only when the same diagnosis recurs across enough distinct samples, and is committed only when paired validation against the current system satisfies the objective-specific acceptance rule: strictly more correct answers for accuracy revisions, or no fewer correct answers at strictly lower token cost for cost revisions. Under a two-epoch budget, the evolved systems attain the highest task-weighted accuracy among the compared methods with both backbones and are best or tied in six of eight model, benchmark settings, with the largest gain on Qwen, MBPP, where accuracy rises from 55.09% to 89.12% while tokens per task fall by 62.2%. The paper further claims that evidence accumulation and validation gating, rather than the generation of edits, are what make evolution stable.

Load-bearing premise

The reported gains depend on selecting the best version in hindsight on the held-out test set, and the paper provides no deployment-time rule that would let a running system identify that version.

Editorial extensions

If this is right

  • A deployed agent team can keep improving without weight updates, as long as it can observe its own failures recur and test its own proposed fixes on a validation set.
  • Accuracy repair and cost reduction are expressed by the same mechanism: swap the diagnosis objective and the acceptance criterion, and the same set of graph operations applies.
  • Near-saturation categories shift naturally from accuracy-directed to cost-directed revisions, so one evolution procedure covers both regimes.
  • The stability of evolution depends mainly on the two gates, recurrence and validation, rather than on the cleverness of the generated edits; removing the validation gate on Game24 with Qwen drops the best accuracy from 94.73% to 65.56%.
  • The reported two-epoch gains are not a ceiling: extending Game24 to 15 epochs raises Qwen accuracy from 79.90% to 94.73% while lowering tokens per task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The checkpoint mechanism is retrospective: the paper itself reports that 39 of 93 committed revisions regress on Test and that headline numbers use the best Test checkpoint, so a deployable EMAS still needs an online rule for when to promote a new version, and rolling validation or retention-aware acceptance would be the natural thing to try.
  • The structured diagnosis key is itself an audit artifact: recording which defect, operation, and location produced each committed revision gives an inspectable lineage for system changes, which could support safety review of self-modifying agent systems.
  • Because evolution runs per task category, EMAS effectively builds a Pareto family of versions per category, so a user could select an operating point by token budget rather than only by peak accuracy.
  • A direct stress test of the causal story would vary the recurrence threshold continuously and measure the Test regression rate; if the regression rate does not fall as the threshold rises, the recurrence gate is not doing the stabilizing work the ablations attribute to it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces EMAS, a method for evolving multi-agent system (MAS) prompts and topology while keeping LLM weights frozen. EMAS represents an MAS as a graph of fine-grained Steps, converts execution traces into structured revision hypotheses, triggers candidate construction only when a hypothesis recurs across samples, and commits the candidate only if paired Validation shows it improves aggregate accuracy or reduces tokens without lowering accuracy. The method is evaluated on Math, MBPP, PlanBench, and Game24 with Qwen3.6-27B and Kimi-K2-6, reporting that EMAS attains the highest task-weighted accuracy for both backbones, with relative gains of 6.30% and 20.10% within two epochs, and a large MBPP improvement for Qwen. The paper also includes ablations on recurrence thresholds and the Validation gate, as well as long-horizon Game24 experiments.

Significance. If the central claims held as stated, this would be a useful contribution: it demonstrates a concrete mechanism for converting post-deployment experience into persistent, reusable system-level updates without weight updates, and it ships unusually thorough empirical apparatus, including confidence intervals, full version trajectories, reproducibility statements, provenance bundles, and explicit disclosures of non-monotonic evolution. The ablations provide genuine evidence that recurrence and paired Validation stabilize the revision process. However, the headline results are not yet established as properties of a deployable algorithm because the reported checkpoints are selected retrospectively on Test; the paper itself concedes this in the Conclusion. The significance of the contribution hinges on whether a fixed, Test-free checkpoint rule preserves a meaningful fraction of the reported gains.

major comments (3)
  1. [§4.1 / Table 1 / §5 / Table 5] The headline claim that EMAS "attains the highest task-weighted overall accuracy" is not currently a property of a deployable policy. Table 5 shows that the reported category-level results are obtained at Versions V1–V18 chosen from each trajectory, and §5 states explicitly that "the headline results use the best checkpoints observed retrospectively on Test" and that no deployment-time rule for choosing among Versions is provided. A user running EMAS therefore cannot realize the Table 1 gains without peeking at Test. This issue is load-bearing because the Abstract and §4.1 present the gains as achieved by EMAS. I ask the authors to define and evaluate a fixed, Test-free checkpoint rule (e.g., the last committed Version, the best-on-Validation Version, or a rule based on a separate holdout) and to re-report all headline numbers under that rule; if no such rule preserves the gains, the claims should be reframed as reachability results rather than as deployed performance.
  2. [§3.6 / Eq. (3) / Table 9] Equation (3) accept/reject on Validation only, and the paper correctly notes that Validation acceptance "does not guarantee improvement on held-out Test tasks." Table 9 quantifies the gap: 39 of 93 E2 revisions regress on Test, with an average loss of 3.03 percentage points. This is not by itself a flaw, but it becomes one when Table 5's reported best Versions are selected using Test outcomes. The evaluation protocol therefore conflates the algorithm's online output with an oracle-selected trajectory. Please provide the distribution of end-of-epoch or Validation-selected performance across runs, not only the best checkpoint, so the reader can assess what a deployed EMAS run would actually deliver.
  3. [Table 1 / Table 4] The comparison with AFlow and ADAS is asymmetric: those baselines are reported as means over repeats 1–3, while the EMAS entries are the best observed Versions selected retrospectively on Test (per Table 5), averaged over three runs. This selection gives EMAS an advantage that is not available to the baselines and can inflate the reported margins. The comparison should be re-run with EMAS's chosen Version determined by a Test-free rule, or the baselines should be allowed the same retrospective best-Version selection and the associated selection bias should be reported and discussed.
minor comments (5)
  1. [§2.1] The first sentence, "MAS is the orchestration of LLM, this allows the system to learn with fixed language model parameters," is grammatically incomplete and should be rewritten.
  2. [Table 7] The support-threshold table is not legible: entries such as "PROMPT-ONLY6" and "REMOVE-EDGE 106" run together, obscuring the operation-specific κ values. Please reformat the table.
  3. [Introduction / Table 3] The Introduction claims the validation set is "balanced, containing equal numbers of correct and incorrect samples," but Table 3 shows, for example, a 47-correct/3-incorrect split for Kimi on PlanBench blocks. This inconsistency should be resolved either by qualifying the claim or by explaining why such imbalanced splits are acceptable for the acceptance rule in Eq. (3).
  4. [§4.2 / Figure 3] The caption says red stars mark where each trajectory reaches its highest observed accuracy, but Table 5 shows that the reported Version is not always the final Version. Please clarify whether the red stars correspond to the Table 5 reported Versions or to the best Version over the full trajectory.
  5. [§4.2] The text mentions a "checkpoint mechanism" but Section 3 describes only accept/reject transitions and does not define checkpoints as an algorithmic component. Please clarify whether checkpoints are part of the EMAS procedure or solely an evaluation artifact used for reporting.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline 'attains' numbers are Test-selected checkpoint maxima (Section 5; Table 5), so the central performance claim reduces to a post hoc Test fit rather than a deployable algorithm output; the validation-gated method otherwise has independent empirical content.

  1. fitted input called prediction [Section 5 Conclusion and Limitations; Abstract; Table 5 checkpoint provenance]
    "Moreover, the headline results use the best checkpoints observed retrospectively on Test. These checkpoints show that better states are reachable, but they do not provide a deployment-time rule for identifying them."

    The reported EMAS accuracies in Table 1 and the Abstract are not the accuracy of a fixed system selected by the algorithm's own acceptance rule (Eq. 3, which uses only Validation). They are per-category maxima over committed Versions evaluated on Test, chosen retrospectively by Test labels; Table 5 identifies the exact 'Reported Version'. Therefore 'EMAS attains 95.79%' is by construction max_v Accuracy_v(Test), not a prediction from a deployable version. Any multi-version trajectory has a Test-selected max at least as high as its final committed Version, so the headline gain is statistically forced upward by using the evaluation target as the selection criterion, and the paper concedes no deployment-time rule exists to obtain it without Test peeking.

full rationale

EMAS is an empirical agent-design method evaluated on external benchmarks, and most of its machinery is not circular: Validation-gated acceptance (Eq. 3), recurrence thresholds, and the ablations against single-trace/no-gate variants are defined independently of the headline results and are testable against held-out data. There is no load-bearing self-citation chain or imported uniqueness theorem. However, the central claim—that EMAS 'attains' the highest accuracy—is reported from the best checkpoint selected retrospectively on Test, as the paper explicitly states in Section 5. Because the reported number is a Test-chosen maximum over the version trajectory rather than the outcome of a fixed, Test-free selection rule, the main accuracy claim reduces to a post hoc fit to the evaluation target. This is a genuine partial circularity of the fitted-input-called-prediction kind, though the paper's own disclosure and the independent value of the Validation-gate analysis keep it from being a complete derivation collapse.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on assumptions about the Diagnoser's reliability, the meaning of recurrence, and the transfer of Validation to Test. The paper itself flags the last two as imperfect. No independent evidence is provided for any of these assumptions beyond the reported experiments.

free parameters (2)
  • Operation-specific recurrence thresholds kappa_a = ADD-NODE=4, PROMPT-ONLY=6, REMOVE-EDGE=10, others in Appendix Table 7
    Hand-set thresholds decide when a candidate may be built; only a kappa=1 sensitivity test is reported, on Game24 with one backbone.
  • Validation set construction = Minimum 50 tasks, balanced V0-correct/incorrect when possible
    Design choice affecting acceptance decisions; can be imbalanced, as in PlanBench blocks with 47 correct and 3 incorrect tasks, and no sensitivity analysis is given.
assumptions (4)
  • domain assumption A single LLM Diagnoser can map each trace into the correct structured hypothesis h=(objective, defect, operation, location).
    Section 3.4: the whole revision pipeline consumes these diagnoses; if they are wrong, recurrence and Validation screen bad candidates but cannot recover missing ones.
  • domain assumption Recurrence of the same canonical key across distinct samples indicates a shared system-level defect.
    Section 3.5 gates candidate construction on recurrence; Section 5 admits 'recurrence does not imply causal equivalence' because the key can merge distinct causes.
  • domain assumption Paired Validation on the fixed split predicts Test behavior well enough to authorize persistent changes.
    Equation 3 uses Validation counts; Section 3.6 says there is no guarantee on Test, and Section 5 reports that 39 of 93 committed revisions regress on Test, so this assumption is load-bearing and partly false.
  • domain assumption The Initial Evolve designer produces a valid but improvable MAS from representative inputs.
    Section 3.3; degenerate initial designs would make evolution results unrepresentative, while perfect designs would leave no headroom.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revision." pith.science (2026). https://pith.science/paper/KF65C3IU

@misc{pith2026260807196,
  author       = {Pith},
  title        = {Pith review of: EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KF65C3IU}},
  note         = {Machine review of arXiv:2608.07196}
}
read the original abstract

Many methods for automated multi-agent system design optimize prompts and topologies during an initial design stage and then deploy the resulting system unchanged on subsequent samples. Experience from these samples is rarely consolidated into reusable system updates, while accuracy-oriented designs may incur high token costs. We introduce EMAS (Evolving Multi-Agent System), which uses this experience to revise MAS topology and prompts without updating LLM parameters, either to improve accuracy or to reduce cost. EMAS converts traces into structured diagnoses that specify a revision operation and target. It generates a candidate revision only when the same diagnosis recurs across samples and applies it only if paired validation against the current MAS meets the corresponding acceptance criterion. Across four benchmarks and two LLMs, EMAS attains the highest task-weighted overall accuracy for both backbones and is best or tied in six of eight model--benchmark settings. Within two evolution epochs, EMAS achieves relative gains of 6.30% and 20.10% in task-weighted accuracy on Kimi-K2-6 and Qwen3.6-27B, respectively. On MBPP with Qwen3.6-27B, EMAS raises accuracy from 55.09% to 89.12% while reducing token use per task by 62.2%. These results show that EMAS can turn experience from new samples into reusable updates to MAS topology and prompts.

Figures

Figures reproduced from arXiv: 2608.07196 by the authors.

Figure 1
Figure 1. Model training updates numerical weights, while EMAS evolves the executable sys￾tem around a frozen language model. (a) A mini-batch loss drives θ (t) → θ (t+1) through back￾propagation. (b) The current category-specific MAS M(v) c produces traces and accuracy–cost feed￾back on task-stream executions through its topology and prompts. Evidence accumulated across tasks supports a discrete candidate Mfc. The candidate … view at source ↗
Figure 2
Figure 2. EMAS Evolution Pipeline Overview. 3 METHODOLOGY 3.1 PROBLEM FORMULATION AND EMAS OVERVIEW EMAS evolves each MAS either to improve accuracy or to reduce cost while preserving accuracy. To support more effective diagnosis, EMAS represents each MAS as a graph in which nodes correspond to LLM calls and edges transfer artifacts. EMAS further increases the graph’s granularity by defining each node as an atomic operation, … view at source ↗
Figure 3
Figure 3. EMAS effectively evolves MASs toward higher accuracy and lower token consump￾tion. Individual committed MAS revisions can introduce local fluctuations, but the complete tra￾jectories exhibit an overall trend toward improved accuracy and fewer tokens per task. (a) Com￾plete trajectories of committed MAS revisions over two evolution epochs for both models on Math, MBPP, and PlanBench, shown by task category. (b) Compl… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accuracy–token trade-offs across evaluated MAS Versions. Each panel shows one benchmark category for both models. Math, MBPP, and PlanBench include Versions from two evolution epochs; Game24 includes Versions through Epoch 15. Filled markers and connecting lines show e…
Figure 5
Figure 5. Figure 5: Accuracy and token trajectories for the EMAS ablations through V20. Qwen3.6-27B is evolved on Game24 from V0 to V20 using Full EMAS, κ = 1, and no Validation gate. Panel (a) shows held-out accuracy; Panel (b) shows tokens per task. Lines report three-run means, and sha…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 27 canonical work pages

  1. [1]

    Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J

    Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA : Reflective prompt evolution can outperform reinforcement learning. In The Fourteenth ...

    work page 2026

  2. [2]

    Trace is the next AutoDiff : Generative optimization with rich feedback, execution traces, and LLM s

    Ching-An Cheng, Allen Nie, and Adith Swaminathan. Trace is the next AutoDiff : Generative optimization with rich feedback, execution traces, and LLM s. In Advances in Neural Information Processing Systems, volume 37, pp.\ 71596--71642, 2024. doi:10.52202/079017-2287. URL https://doi.org/10.52202/079017-2287

    work page doi:10.52202/079017-2287 2024

  3. [3]

    Promptbreeder: Self-referential self-improvement via prompt evolution

    Chrisantha Fernando, Dylan Sunil Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel. Promptbreeder: Self-referential self-improvement via prompt evolution. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp.\ 13481--13544. PMLR, 2024. URL https://proceedings...

    work page 2024

  4. [4]

    Automated design of agentic systems

    Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=t9U3LW7JVX

    work page 2025

  5. [5]

    Symbolic learning enables self-evolving agents

    Yixin Ou, Wangchunshu Zhou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, and Yuchen Eleanor Jiang. Symbolic learning enables self-evolving agents. AI Open, 6: 0 314--322, 2025. doi:10.1016/j.aiopen.2025.11.004. URL https://doi.org/10.1016/j.aiopen.2025.11.004

    doi:10.1016/j.aiopen.2025.11.004 2025

  6. [6]

    Reflexion: Language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, 2023. doi:10.52202/075280-0377. URL https://doi.org/10.52202/075280-0377

    doi:10.52202/075280-0377 2023

  7. [8]

    MASPO : Joint prompt optimization for LLM -based multi-agent systems

    Zhexuan Wang, Xuebo Liu, Li Wang, Zifei Shan, Yutong Wang, Zhenxi Song, and Min Zhang. MASPO : Joint prompt optimization for LLM -based multi-agent systems. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research. PMLR, 2026. URL https://arxiv.org/abs/2605.06623

    arXiv 2026

  8. [9]

    Le, Denny Zhou, and Xinyun Chen

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Bb4VGOWELI

    work page 2024

Show all 39 references
  1. [10]

    Self-taught optimizer ( STOP ): Recursively self-improving code generation

    Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer ( STOP ): Recursively self-improving code generation. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=46Zgqo4QIU

    2024

  2. [11]

    EvoFlow : Evolving diverse agentic workflows on the fly

    Guibin Zhang, Kaijie Chen, Guancheng Wan, Heng Chang, Hong Cheng, Kun Wang, Shuyue Hu, and Lei Bai. EvoFlow : Evolving diverse agentic workflows on the fly. arXiv preprint arXiv:2502.07373, 2025 a . URL https://arxiv.org/abs/2502.07373

    2025 arXiv

  3. [12]

    Darwin G \"o del machine: Open-ended evolution of self-improving agents

    Jenny Zhang, Shengran Hu, Cong Lu, Robert Tjarko Lange, and Jeff Clune. Darwin G \"o del machine: Open-ended evolution of self-improving agents. In The Fourteenth International Conference on Learning Representations, 2026. URL https://arxiv.org/abs/2505.22954

    2026 arXiv

  4. [13]

    AFlow : Automating agentic workflow generation

    Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow : Automating agentic workflow generation. In The Thirteenth International Conference on...

    2025

  5. [14]

    Offline training of language model agents with functions as learnable weights

    Shaokun Zhang, Jieyu Zhang, Jiale Liu, Linxin Song, Chi Wang, Ranjay Krishna, and Qingyun Wu. Offline training of language model agents with functions as learnable weights. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Ma...

    2024

  6. [15]

    Which agent causes task failures and when? on automated failure attribution of LLM multi-agent systems

    Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, and Qingyun Wu. Which agent causes task failures and when? on automated failure attribution of LLM multi-agent systems. In Proceedings of the 42nd Int...

    2025

  7. [16]

    ExpeL : LLM agents are experiential learners

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL : LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 19632--19642, 2024. doi:10.1609/aaai.v38i17.29936. URL https://ojs....

    2024 doi

  8. [17]

    Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vuli \'c , Anna Korhonen, and Sercan O. Arik. Multi-agent design: Optimizing agents with better prompts and topologies. In The Fourteenth International Conference on Learning Representations, 2026. URL https:...

    2026

  9. [18]

    GPTS warm: Language agents as optimizable graphs

    Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and J \"u rgen Schmidhuber. GPTS warm: Language agents as optimizable graphs. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning ...

    2024

  10. [19]

    and Burger, Doug and Wang, Chi , booktitle =

    Wu, Qingyun and Bansal, Gagan and Zhang, Jieyu and Wu, Yiran and Li, Beibin and Zhu, Erkang and Jiang, Li and Zhang, Xiaoyun and Zhang, Shaokun and Liu, Jiale and Awadallah, Ahmed Hassan and White, Ryen W. and Burger, Doug and Wang, Chi , booktitle =. 2024 , url =

    2024

  11. [20]

    The Twelfth International Conference on Learning Representations , year =

    Hong, Sirui and Zhuge, Mingchen and Chen, Jonathan and Zheng, Xiawu and Cheng, Yuheng and Wang, Jinlin and Zhang, Ceyao and Wang, Zili and Yau, Steven Ka Shing and Lin, Zijuan and Zhou, Liyang and Ran, Chenyu and Xiao, Lingfeng and Wu, Chenglin and Schmidhuber, J. The Twelfth ...

  12. [21]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Zhuge, Mingchen and Wang, Wenyi and Kirsch, Louis and Faccio, Francesco and Khizbullin, Dmitrii and Schmidhuber, J. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , volume =

    2024

  13. [22]

    AI Open , volume =

    Symbolic Learning Enables Self-Evolving Agents , author =. AI Open , volume =. 2025 , doi =

    2025

  14. [23]

    arXiv preprint arXiv:2412.03624 , year =

    How to Correctly Do Semantic Backpropagation on Language-Based Agentic Systems , author =. arXiv preprint arXiv:2412.03624 , year =. 2412.03624 , archiveprefix =

    arXiv

  15. [24]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Offline Training of Language Model Agents with Functions as Learnable Weights , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , volume =

    2024

  16. [25]

    Trace Is the Next

    Cheng, Ching-An and Nie, Allen and Swaminathan, Adith , booktitle =. Trace Is the Next. 2024 , doi =

    2024

  17. [26]

    The Thirteenth International Conference on Learning Representations , year =

    Automated Design of Agentic Systems , author =. The Thirteenth International Conference on Learning Representations , year =

  18. [27]

    2025 , url =

    Zhang, Jiayi and Xiang, Jinyu and Yu, Zhaoyang and Teng, Fengwei and Chen, Xionghui and Chen, Jiaqi and Zhuge, Mingchen and Cheng, Xin and Hong, Sirui and Wang, Jinlin and Zheng, Bingnan and Liu, Bang and Luo, Yuyu and Wu, Chenglin , booktitle =. 2025 , url =

    2025

  19. [28]

    The Fourteenth International Conference on Learning Representations , year =

    Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies , author =. The Fourteenth International Conference on Learning Representations , year =

  20. [29]

    2025 , eprint =

    Zhang, Guibin and Chen, Kaijie and Wan, Guancheng and Chang, Heng and Cheng, Hong and Wang, Kun and Hu, Shuyue and Bai, Lei , journal =. 2025 , eprint =

    2025

  21. [30]

    Cut the Crap: An Economical Communication Pipeline for

    Zhang, Guibin and Yue, Yanwei and Li, Zhixun and Yun, Sukwon and Wan, Guancheng and Wang, Kun and Cheng, Dawei and Yu, Jeffrey Xu and Chen, Tianlong , booktitle =. Cut the Crap: An Economical Communication Pipeline for. 2025 , url =

    2025

  22. [31]

    Proceedings of the 42nd International Conference on Machine Learning , pages =

    Multi-Agent Architecture Search via Agentic Supernet , author =. Proceedings of the 42nd International Conference on Machine Learning , pages =. 2025 , volume =

    2025

  23. [32]

    and Tan, Shangyin and Soylu, Dilara and Ziems, Noah and Khare, Rishi and Opsahl-Ong, Krista and Singhvi, Arnav and Shandilya, Herumb and Ryan, Michael J

    Agrawal, Lakshya A. and Tan, Shangyin and Soylu, Dilara and Ziems, Noah and Khare, Rishi and Opsahl-Ong, Krista and Singhvi, Arnav and Shandilya, Herumb and Ryan, Michael J. and Jiang, Meng and Potts, Christopher and Sen, Koushik and Dimakis, Alexandros G. and Stoica, Ion and ...

    2026

  24. [33]

    2026 , volume =

    Wang, Zhexuan and Liu, Xuebo and Wang, Li and Shan, Zifei and Wang, Yutong and Song, Zhenxi and Zhang, Min , booktitle =. 2026 , volume =

    2026

  25. [34]

    Which Agent Causes Task Failures and When? On Automated Failure Attribution of

    Zhang, Shaokun and Yin, Ming and Zhang, Jieyu and Liu, Jiale and Han, Zhiguang and Zhang, Jingyang and Li, Beibin and Wang, Chi and Wang, Huazheng and Chen, Yiran and Wu, Qingyun , booktitle =. Which Agent Causes Task Failures and When? On Automated Failure Attribution of. 202...

    2025

  26. [35]

    Self-Taught Optimizer (

    Zelikman, Eric and Lorch, Eliana and Mackey, Lester and Kalai, Adam Tauman , booktitle =. Self-Taught Optimizer (. 2024 , url =

    2024

  27. [36]

    Zhang, Jenny and Hu, Shengran and Lu, Cong and Lange, Robert Tjarko and Clune, Jeff , booktitle =. Darwin. 2026 , url =

    2026

  28. [37]

    Advances in Neural Information Processing Systems , volume =

    Reflexion: Language Agents with Verbal Reinforcement Learning , author =. Advances in Neural Information Processing Systems , volume =. 2023 , doi =

    2023

  29. [38]

    2024 , doi =

    Zhao, Andrew and Huang, Daniel and Xu, Quentin and Lin, Matthieu and Liu, Yong-Jin and Huang, Gao , booktitle =. 2024 , doi =

    2024

  30. [39]

    The Twelfth International Conference on Learning Representations , year =

    Large Language Models as Optimizers , author =. The Twelfth International Conference on Learning Representations , year =

  31. [40]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , volume =

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.