MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines

Chaowei Xiao; Xiaogeng Liu; Yaolun Zhang

REVIEW 4 major objections 5 minor 3 cited by

MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MetaAgent claims that one task description is enough to auto-generate a tool-using, traceback-capable multi-agent system, and that the resulting systems beat other auto-designed methods and nearly match task-specialized human designs.

desk verdict A practically useful auto-design framework for FSM-based multi-agent systems that deserves review, but the appendix suggests the implemented transition mechanism differs from the described condition verifier. read the letter →

arxiv 2507.22606 v1 pith:EZZF4NCB submitted 2025-07-30 cs.AI

Yaolun Zhang , Xiaogeng Liu , Chaowei Xiao This is my paper · ORCID

classification cs.AI

keywords multi-agentsystemsfinitestatemachineautomaticagentdesignLLMagentstoolusetracebackconditionverifiertask-levelgeneralization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MetaAgent claims that a single natural-language description of a task type is enough to automatically build a working multi-agent system. The generated system is organized as a finite state machine in which each state pairs a task-solving agent with a condition verifier, listener agents, and transition conditions written in plain language, and an optimization pass merges redundant states before deployment. Across text-based benchmarks, machine-learning tasks, and software-development tasks, the generated systems outperform other auto-designed approaches and come close to, or exceed, human-designed systems tuned for those tasks. If true, this would turn multi-agent system construction from hand-crafted engineering into a prompting step plus automated polishing, at lower token cost than per-case automatic design. The paper also argues that the FSM subsumes existing linear, debate, and orchestrator-based multi-agent structures.

What carries the argument

The load-bearing object is the tuple $M = (\Sigma, S, s_0, F, \delta)$, the standard finite state machine, re-read so that each state $s$ is a work situation rather than an abstract node. A state carries an agent $s.\text{Agent}$, an instruction $s.\text{Ins}$, a condition verifier $s.\text{Ver}$, and listeners $s.\text{Lis}$; the verifier maps agent output to natural-language transition conditions, choosing the next state, a traceback, or a null-transition with feedback. The construction stage uses a designer LLM to produce agents first and then the state/transition graph, and an adaptor LLM to merge pairwise-equivalent states until the structure stabilizes. This object carries the argument because it is simultaneously the design output, the optimization target, and the runtime controller: the same JSON states define which agent acts, when control moves, and when the system loops back.

What would settle it

Instrument the deployment stage so every state transition is logged with its cause. If the majority of transitions are triggered by a '<STATE_TRANS>' string that the task-solving agent itself was prompted to output, rather than by an independent condition-verifier call, then the separate verifier described in Algorithm 2 is not the actual controller, and the system's claimed behavior reduces to self-reporting. A second check would measure verifier accuracy on held-out transitions across new task domains.

Watch

Extended reading notes

Core claim

MetaAgent's central claim is that a finite state machine is the right universal skeleton for an automatically designed multi-agent system, and that an LLM can write that skeleton from a task description alone. In the generated FSM, each state bundles an assigned agent, a natural-language state instruction, a condition verifier that chooses the next transition, and listener agents that receive the state's output; a null-transition keeps the same agent working when no condition fires, and transitions back to earlier states provide traceback. The paper reports that the generated systems reach 0.86 on Trivial Creative Writing and 0.60 on GPQA, outperform all compared auto-design systems on the machine-learning benchmark with an average normalized score of 0.83, and pass 85% of software-development checkpoints on average versus 35% for the human-designed baseline. It also argues that linear, debate, and orchestrator-structured multi-agent systems are all restricted cases of an FSM, which is why the structure fits auto-design so well.

Load-bearing premise

The load-bearing premise is that the designer LLM produces a correct, complete finite state machine from the task description and that, during deployment, the condition verifier reliably maps agent outputs to the natural-language transition conditions.

Editorial extensions

If this is right

A task-type description, not a per-case prompt, is enough to build a reusable multi-agent system that generalizes across cases within the same task domain.
Tool use, traceback, and state merging are the active ingredients: the ablations show performance drops on every benchmark when any of the three is removed.
Because linear, debate, and orchestrator structures are special cases of an FSM, any improvement to FSM-based auto-design carries over to those existing structures rather than competing with them.
An automatically generated system can come close to, and in the software tasks exceed, systems that were hand-built and hand-tuned for those specific benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: the same generation loop should transfer to workflow domains the paper did not test, such as document processing or customer support, because nothing in the designer prompt is tied to machine learning, writing, or software.
Inference: deployment traces could be used as a free training signal: whenever a null-transition or traceback fires, the condition text that triggered it could be logged and later used to tighten the verifier, turning the FSM into a self-improving structure.
Inference: the design-stage cost numbers suggest that task-level FSM design, rather than per-case design, is what makes auto-design practical; a direct comparison of design cost per unit of generalization would make that economy explicit.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a legitimate engineering contribution with a real gap between the description and the appendix. The idea of auto-designing a finite state machine to organize LLM agents, with tools and a state-merging optimizer, is new as a combination, and the paper shows it works reasonably on ML, software, and text tasks.

The good parts are concrete. The optimization step is cheap and data-free, the ablations for tool-using, traceback, and optimization all show meaningful performance drops, and the cost analysis is a useful addition. If the system works as described, it would give practitioners a practical way to go from a task paragraph to a tool-enabled multi-agent system.

Now the soft spots, in order of importance.

The biggest one is the verifier. Section 3.2.2 and Algorithm 2 describe a separate condition verifier that checks the agent's output against transition conditions. But the running example in Appendix H.1 shows each task-solving agent's system prompt instructing it to output ': 2' when its own condition is met, and the deployment transcript shows the agent emitting that token itself. There is no separate verifier call shown. If that token is parsed directly, then the transition decision is made by the same agent that performed the work, and the claimed independent control mechanism is not what is implemented. That is close to the hard-coded string detection the paper criticizes in prior FSM work. The paper needs to clarify whether the verifier is actually called in deployment, and if so, why the agent prompts contain those tokens.

Other issues are more standard. No repeated runs or variance info, despite temperature 0 being set. AutoAgents scores 0.00 on every ML bench task, which smells like a broken adaptation; it should be fixed or dropped. EvoAgent and ADAS are cited but not evaluated, which weakens the 'surpasses other auto-designed methods' claim. The FSM generalization argument in Section 3.5 is definitional but harmless.

Overall, the central empirical claim—that the generated system beats the auto-designed baselines tested and is competitive with human-designed ones—is plausible with the tested baselines, but it is not fully established until the verifier question is resolved. This is not a desk-reject paper; it is worth a careful review, and the authors can likely answer the questions. I would send it out.

Referee Report

4 major / 5 minor

Summary. MetaAgent proposes to automatically construct a task-level multi-agent system from a general task description. An LLM designer first generates a set of agents and then an FSM whose states bind a task-solving agent, a state instruction, listener agents, and a condition verifier; transitions are expressed as natural-language conditions. An optimization stage uses an LLM 'adaptor' to merge states that are deemed redundant. At deployment, Algorithm 2 runs the current state's agent and uses the condition verifier to select the next state, including null-transitions and tracebacks. The paper reports experiments on Trivial Creative Writing, GPQA, ML Bench, and five software development tasks, comparing against prompt-based, auto-designed, and human-designed baselines, plus ablations on tools, traceback, optimization, and foundation-model quality.

Significance. If the described mechanism is realized as implemented, the paper would make a useful empirical contribution: it shows that a task-level description can cheaply produce a tool-enabled multi-agent system that is competitive with human-designed frameworks (e.g., 0.83 vs 0.86 average on ML Bench, 0.85 vs 0.35 on software checkpoints), and the design-stage token costs in Table 7 are low. The ablations are well-chosen and the distinction between per-case and per-task design is practically important. The provision of a code link is also a strength for reproducibility. However, the significance currently hinges on two unresolved points: whether the condition verifier is actually used in deployment, and whether the auto-designed baselines were evaluated fairly. The manuscript's own appendix evidence raises the first concern directly, so the empirical results do not yet establish the mechanism claimed in the abstract and Section 3.

major comments (4)

[§3.2.2, Algorithm 2, Appendix H.1] The described mechanism relies on an independent Condition Verifier (s.Ver) that evaluates each state's output against natural-language transition conditions and returns the destination state. The only end-to-end artifact in the paper contradicts this: in Appendix H.1, each generated agent's system prompt instructs the task-solving agent itself to emit literal tokens such as '<STATE_TRANS>: 2' or '<STATE_TRANS>: None', and the deployment transcript shows DataAndModelAgent emitting '<STATE_TRANS>: 2' directly, with no separate verifier invocation. If transitions are decided by the task-solving agent or by parsing that emitted token, the independent verifier is bypassed and the method degenerates to the hard-coded string-detection transitions that Section 2.1 criticizes in prior work. Please provide implementation-level confirmation of a distinct verifier (e.g., code or logs showing s.Ver called on the agent output), or revise the method description and the novelty claims accordingly.
[§4.1–4.2.1, Tables 2–4] The comparison set for 'auto-designed methods' is incomplete. EvoAgent, ADAS, and Symbolic-Learning are cited in Section 2.1 as the closest prior work on automatic multi-agent design, and Table 1 marks EvoAgent as auto-designed and tool-enabled, but none of them is evaluated in the experiments. The abstract's claim of surpassing 'other auto-designed methods' therefore cannot be substantiated beyond SPP and AutoAgents. At minimum, EvoAgent, which is directly comparable in not requiring external training data, should be run on the same benchmarks.
[Table 3, §4.2.1] AutoAgents obtains 0.00 on all five ML Bench datasets while obtaining nonzero results on software development (Table 4). The paper says the adaptation 'extract[s] the generated code and get[s] the execution result' (§4.2.1). A uniform zero across all datasets is more consistent with a harness failure (e.g., no code generated, wrong interface, or execution error) than with genuinely poor model performance. Without sample outputs or a working adaptation script, the claim that MetaAgent surpasses AutoAgents on ML Bench is unsupported.
[§4.2–4.2.1, Tables 2–4] All results are single numbers with no error bars, repeated runs, or statistical tests, despite language-model nondeterminism even at temperature 0 in practice. The key comparisons include small margins (e.g., 0.86 vs 0.79 on Trivial Creative Writing; 0.83 vs 0.86 average on ML Bench versus DataInterpreter), so without variance information the reported advantages cannot be evaluated. Please report at least 3–5 runs per condition, or a clearly documented deterministic execution protocol.

minor comments (5)

[§4.2.1, Tables 4 and 5] The text says 'Table 5 presents the results for five different software development tasks', but Table 5 in the manuscript is the foundation-model transfer table; the software development results are in Table 4.
[§4.3, Table 7, Appendix B] The text says '6 tasks in software development', but Appendix B lists five software tasks (2048, Snake, Brick Breaker, Excel, Weather) and the cost table also says '6 Software Development Tasks'; the count should be corrected.
[§3.2.2, Appendix H.1] There are typos in the method and in the generated prompts, including 'null-trastion' for 'null-transition' and 'Thses enironment' in the prompt text; these should be cleaned up if the prompts are part of the public artifact.
[Table 5] The header uses 'GPT3.5-Turbo' inconsistently; standardize as GPT-3.5-Turbo.
[§3.5] The claim that linear, debate, and orchestrator structures are special cases of FSM is conceptually useful but largely definitional; consider moving the broader philosophical claim to the discussion and keeping the concrete comparison in Figure 3.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirically evaluated against external benchmarks, and no prediction reduces by construction to its inputs.

full rationale

The paper's central claim is that a designer LLM can generate an FSM-based multi-agent system that performs well on Trivial Creative Writing, GPQA, ML Bench, and software development tasks. This is an empirical claim measured against external benchmarks (SPP, AutoAgents, MetaGPT, DataInterpreter, etc.), not a quantity derived from the framework's own assumptions. The FSM state-merging optimization is judged by an LLM, but its value is tested by an ablation (Table 6) that removes it and shows a performance drop; the optimization is not presented as a prediction that reduces to its own criterion. Section 3.5's statement that linear, debate, and orchestrator structures are special cases of FSMs is a definitional observation about the FSM formalism, not a fitted result or a circular derivation. No load-bearing argument relies on a self-citation chain: the cited prior work (Hopcroft, MetaGPT, SPP, etc.) is standard background and external comparison, not the source of the claimed superiority. The appendix H.1 example shows task-solving agents emitting '<STATE_TRANS>: 2' themselves, which may indicate that the separate Condition Verifier described in Sections 3.2.2 and Algorithm 2 is bypassed in deployment; however, this is an implementation-consistency or correctness concern, not a circularity, because the empirical comparison still tests the deployed system as a whole against external benchmarks. Accordingly, no step in the paper's derivation chain is equivalent by construction to its inputs, and the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The framework introduces no fitted numeric parameters; the free parameters are operational settings. Its load-bearing assumptions concern LLM reliability in design, transition verification, and optimization, plus the generalization of one FSM to a task domain. These are empirical premises, not proven facts.

free parameters (2)

maximum interactions M = not reported
Deployment loop terminates after M transitions (Algorithm 2). No value is given, yet it bounds all runs and can affect success rates.
sampling temperature = 0
Set to 0 in Section 4.1 for reproducibility; a design choice rather than a fitted value, but it shapes all results.

assumptions (5)

domain assumption An LLM designer can generate a valid and effective FSM from a task description.
Section 3.2.1-3.2.2. No formal verification; the entire pipeline depends on GPT-4o following the design prompt correctly.
domain assumption LLM condition verifiers reliably map agent outputs to natural-language transition conditions.
Section 3.2.2 and Algorithm 2. If the verifier misreads conditions, the FSM can loop, terminate prematurely, or take wrong branches.
ad hoc to paper Mergeability of states can be decided by an LLM using the stated criteria.
Section 3.2.3 and Appendix D.2. The criteria (role distinctness, information necessity, tool overlap) are subjective and not checked against ground truth.
domain assumption A single FSM designed for a task description generalizes to all cases in that task domain.
Section 3.5 and experiments. Only a few tasks per domain are tested (5 ML datasets, 5 software apps), and the FSM is not re-designed per case.
standard math Standard finite state machine tuple definition is used.
Section 3.1 relies on Hopcroft et al. (2001) for the FSM formalization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines." pith.science (2026). https://pith.science/paper/EZZF4NCB

@misc{pith2026250722606,
  author       = {Pith},
  title        = {Pith review of: MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZZF4NCB}},
  note         = {Machine review of arXiv:2507.22606}
}

read the original abstract

Large Language Models (LLMs) have demonstrated the ability to solve a wide range of practical tasks within multi-agent systems. However, existing human-designed multi-agent frameworks are typically limited to a small set of pre-defined scenarios, while current automated design methods suffer from several limitations, such as the lack of tool integration, dependence on external training data, and rigid communication structures. In this paper, we propose MetaAgent, a finite state machine based framework that can automatically generate a multi-agent system. Given a task description, MetaAgent will design a multi-agent system and polish it through an optimization algorithm. When the multi-agent system is deployed, the finite state machine will control the agent's actions and the state transitions. To evaluate our framework, we conduct experiments on both text-based tasks and practical tasks. The results indicate that the generated multi-agent system surpasses other auto-designed methods and can achieve a comparable performance with the human-designed multi-agent system, which is optimized for those specific tasks.

Figures

Figures reproduced from arXiv: 2507.22606 by the authors.

**Figure 1.** An example of what a state is, and how our finite state machine structure works. decreases in performance on the aforementioned tasks, highlighting the importance of these features. 2. Related Works 2.1. Multi-Agent System Previous works have discussed multi-agent systems in various scenarios. One category of Multi-Agent Systems are designed to simulate real-world scenarios (Park et al., 2023; Xu et al., 2023; Hua… view at source ↗

**Figure 2.** The construction stage of MetaAgent 3.2.3. OPTIMIZING THE FSM The initial version of the FSM frequently failed due to an excessively large number of states, many of which were redundant or could be consolidated. To address this issue, we optimized the FSM by systematically merging states that are considered equivalent based on specific criteria. Given the state set ( S ), we perform pairwise comparisons of all poss… view at source ↗

**Figure 3.** Compare FSM with Other Kinds of Multi-Agent System Structures. The figure shows the Linear, Decentralized Debate, and Coordinates with Orchestrator structures’ difference between the finite state machine. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systems
cs.CR 2026-07 conditional novelty 6.0 of 10

Activation-space divergence detects and corrects compromised LLM agents in multi-agent systems without interaction graphs or synchronized rounds, outperforming graph baselines especially under async stealthy attacks.
ProcAgent: An Agentic Framework for Procedural Task Guidance on Edge with Human-in-the-Loop
cs.AI 2026-06 conditional novelty 6.0 of 10

ProcAgent demonstrates a complete step-by-step assembly assistant that runs on a single edge device by pairing cheap continuous perception with on-demand vision-language verification, a task graph, and human confirmation.
Trajectory-Aware Retrieval Agents for Temporal Decision- Making
cs.AI 2026-07 reject novelty 5.0 of 10

TLM reports large accuracy gains on medical and financial temporal-decision tasks by fitting linear trends to retrieved embeddings, but its monotonicity theorem is circular and its baselines omit plain fine-tuned RAG.

Reference graph

Works this paper leans on

22 extracted references · 16 canonical work pages · cited by 3 Pith papers

[1]

agents": [ 3 { 4

**Role Distinguishability**: Evaluate if the roles associated with the states are sufficiently distinct. If the roles are not distinct, the states should be merged. 2. **Information Necessity**: Assess if the information transfer between the states is necessary. If the information transfer is unnecessary, the states should be merged. 3. **Tool Assignment*...

work page
[2]

URL https://openreview.net/forum ?id=zj7YuTE4t8. Fourney, A., Bansal, G., Mozannar, H., Tan, C., Salinas, E., Erkang, Zhu, Niedtner, F., Proebsting, G., Bassman, G., Gerrits, J., Alber, J., Chang, P., Loynd, R., West, R., Dibia, V ., Awadallah, A., Kamar, E., Hosn, R., and Amershi, S. Magentic-one: A generalist multi-agent system for solving complex tasks...

arXiv 2024
[3]

URL https://www.ijcai.org/procee dings/2024/890. Hong, S., Lin, Y ., Liu, B., Liu, B., Wu, B., Li, D., Chen, J., Zhang, J., Wang, J., Zhang, L., Zhang, L., Yang, M., Zhuge, M., Guo, T., Zhou, T., Tao, W., Wang, W., Tang, X., Lu, X., Zheng, X., Liang, X., Fei, Y ., Cheng, Y ., Xu, Z., and Wu, C. Data interpreter: An llm agent for data science, 2024a. URL h...

arXiv 2024
[4]

Can open an interface

Can score correctly Snake Game 1. Can open an interface

work page
[5]

Rein, D., Hou, B

URL https://openreview.net/forum ?id=dHng2O0Jjr. Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y ., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL https://arxiv.org/abs/2311.12022. 11 MetaAgent: Automatically Building Multi-Agent System based on Finite State Machine Shinn, N., Cassano,...

arXiv 2023
[7]

Yuan, S., Song, K., Chen, J., Tan, X., Li, D., and Yang, D

URL https://openreview.net/pdf?i d=WE_vluYUL-X. Yuan, S., Song, K., Chen, J., Tan, X., Li, D., and Yang, D. Evoagent: Towards automatic multi-agent generation via evolutionary algorithms, 2024. URL https://arxi v.org/abs/2406.14228. Zhang, Y ., Pan, Y ., Wang, Y ., Cai, J., Zheng, Z., Zeng, G., and Liu, Z. Pybench: Evaluating llm agent on various 12 MetaA...

arXiv 2024
[8]

Can operate normally

work page
[10]

Can operate the snake normally

work page

Show all 22 references

[11]

Can eat beans correctly
[12]

Can open an interface

The snake can grow normally Brick Breaker Game 1. Can open an interface
[13]

Can operate the paddle normally
[14]

Can eliminate bricks correctly
[15]

Can open an interface

Can score correctly excel app 1. Can open an interface
[16]

Can transfer files correctly
[17]

Can display correctly
[18]

Can open an interface

Can close correctly weather 1. Can open an interface
[19]

Has weather query function
[20]

Can fetch weather data correctly
[21]

Evaluation Criteria for Software Development Tasks C

Can display weather data aesthetically Table 8. Evaluation Criteria for Software Development Tasks C. Statistics of MLE Bench Finite State Machine In this section, we show the statistics of an example finite state machine. Table C shows the static statistics including the stat...
[2023]

Wang, X., Chen, Y ., Yuan, L., Zhang, Y ., Li, Y ., Peng, H., and Ji, H

URL https://openreview.net/pdf?i d=1PL1NIMMrw. Wang, X., Chen, Y ., Yuan, L., Zhang, Y ., Li, Y ., Peng, H., and Ji, H. Executable code actions elicit better LLM agents. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. ...

2024 arXiv
[2024]

Carroll, J

URL https://openreview.net/forum ?id=CmOmaxkt8p. Carroll, J. and Long, D. Theory of Finite Automata: With an Introduction to Formal Languages. 1989. Chen, G., Dong, S., Shu, Y ., Zhang, G., Sesay, J., Karls- son, B., Fu, J., and Shi, Y . Autoagents: A framework for automatic a...

1989 arXiv
[5700]

URL https: //doi.org/10.1145/568438.568455

doi: 10.1145/568438.568455. URL https: //doi.org/10.1145/568438.568455. Hu, S., Lu, C., and Clune, J. Automated design of agentic systems, 2024. URL https://arxiv.org/abs/ 2408.08435. Hua, W., Fan, L., Li, L., Mei, K., Ji, J., Ge, Y ., Hemphill, L., and Zhang, Y . War and peac...

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

agents": [ 3 { 4

**Role Distinguishability**: Evaluate if the roles associated with the states are sufficiently distinct. If the roles are not distinct, the states should be merged. 2. **Information Necessity**: Assess if the information transfer between the states is necessary. If the information transfer is unnecessary, the states should be merged. 3. **Tool Assignment*...

work page

[2] [2]

URL https://openreview.net/forum ?id=zj7YuTE4t8. Fourney, A., Bansal, G., Mozannar, H., Tan, C., Salinas, E., Erkang, Zhu, Niedtner, F., Proebsting, G., Bassman, G., Gerrits, J., Alber, J., Chang, P., Loynd, R., West, R., Dibia, V ., Awadallah, A., Kamar, E., Hosn, R., and Amershi, S. Magentic-one: A generalist multi-agent system for solving complex tasks...

arXiv 2024

[3] [3]

URL https://www.ijcai.org/procee dings/2024/890. Hong, S., Lin, Y ., Liu, B., Liu, B., Wu, B., Li, D., Chen, J., Zhang, J., Wang, J., Zhang, L., Zhang, L., Yang, M., Zhuge, M., Guo, T., Zhou, T., Tao, W., Wang, W., Tang, X., Lu, X., Zheng, X., Liang, X., Fei, Y ., Cheng, Y ., Xu, Z., and Wu, C. Data interpreter: An llm agent for data science, 2024a. URL h...

arXiv 2024

[4] [4]

Can open an interface

Can score correctly Snake Game 1. Can open an interface

work page

[5] [5]

Rein, D., Hou, B

URL https://openreview.net/forum ?id=dHng2O0Jjr. Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y ., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL https://arxiv.org/abs/2311.12022. 11 MetaAgent: Automatically Building Multi-Agent System based on Finite State Machine Shinn, N., Cassano,...

arXiv 2023

[6] [7]

Yuan, S., Song, K., Chen, J., Tan, X., Li, D., and Yang, D

URL https://openreview.net/pdf?i d=WE_vluYUL-X. Yuan, S., Song, K., Chen, J., Tan, X., Li, D., and Yang, D. Evoagent: Towards automatic multi-agent generation via evolutionary algorithms, 2024. URL https://arxi v.org/abs/2406.14228. Zhang, Y ., Pan, Y ., Wang, Y ., Cai, J., Zheng, Z., Zeng, G., and Liu, Z. Pybench: Evaluating llm agent on various 12 MetaA...

arXiv 2024

[7] [8]

Can operate normally

work page

[8] [10]

Can operate the snake normally

work page