Pith. sign in

REVIEW 4 major objections 4 minor 40 references

Parallel execution on separate desktops lifts GUI-agent success from 33.5% to 46.4% in a new benchmark, at half the steps and tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:55 UTC pith:5SBAPLRW

load-bearing objection A genuinely new benchmark for parallel GUI agents, but the abstract's success-rate claim is undercut by the paper's own N-ablation; the real contribution is efficiency, not accuracy. the 4 major comments →

arxiv 2607.22689 v1 pith:5SBAPLRW submitted 2026-07-17 cs.AI

Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents

Zedong Yu , Qianxing Li , Zhi Gao , Liuyu Xiang , Chenrui Shi , Yang Liu , Huiming Wu , Yujie Wei
show 3 more authors
Yuhao Fei Yubo Fu Zhaofeng He
This is my paper
classification cs.AI
keywords GUI agentsparallel executionmulti-agent coordinationbenchmarkplanner-worker architecturelarge multimodal modelstask decompositionefficiency metrics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces ParaGUIBench, the first benchmark for running multiple GUI agents concurrently on separate desktop instances, and ParaGUI, a planner-worker agent that exploits parallel tool-calling. ParaGUI decomposes a long-horizon task, dispatches self-contained sub-tasks to up to five workers in isolated containers sharing a file system, then aggregates their summaries. On the 233-task benchmark, ParaGUI reaches 46.4% success versus 33.5% for the strongest serial baseline, while using roughly half the critical-path steps and less than half the tokens. The authors argue that parallel execution can improve both accuracy and efficiency on decomposable tasks, and that the architecture transfers to serial settings as well.

Core claim

The central claim is that a planner-worker architecture can coordinate multiple GUI agents on separate desktops and beat the best serial agent on long-horizon tasks. ParaGUI's planner emits rounds of parallel sub-task instructions, each containing every earlier result the sub-task depends on (self-contained dispatch), so a worker needs no other context. Using a GPT-5.4 planner with Seed-1.8 workers, ParaGUI scores 46.4% success on ParaGUIBench, outpacing the strongest serial baseline at 33.5% with roughly half the steps and less than half the tokens. The same planner-worker design with a single worker also improves a standard 369-task desktop benchmark by 7.6 points, though it falls behind o

What carries the argument

The machinery is the ParaGUI planner-worker loop combined with ParaGUIBench's parallel-native evaluation. The planner alternates reasoning rounds with synchronous dispatch; each round emits up to N parallel sub-tasks with dependency references resolved into the instruction text (self-contained dispatch, SCD). Workers execute in separate Docker containers that share a file system, returning textual summaries that the planner aggregates. ParaGUIBench provides the multi-container infrastructure, 233 tasks annotated by dependency structure (parallel independent, parallel dependent, serial), and efficiency metrics: step reduction ratio, parallelism degree, and token cost, with rule-based verifica

Load-bearing premise

The benchmark's rule-based evaluators reliably decide whether a task is truly solved, and the 233 tasks fairly represent long-horizon GUI work rather than being curated toward workflows that happen to favor parallel dispatch.

What would settle it

Take a random sample of, say, 50 ParaGUIBench tasks, have human annotators independently judge whether the final environment state satisfies the instruction, and compare their verdicts to the rule-based evaluator. If human-system agreement is substantially below the reported success rates—or if a serial agent with an unrestricted step budget solves a large share of the 'parallel' tasks—the headline gains would reflect benchmark construction rather than inherent benefits of parallel execution.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Long-horizon GUI tasks that decompose into independent sub-tasks can be completed faster and more accurately by parallel workers, with the critical path set by the slowest worker per round.
  • Self-contained dispatch is a major capability lever: ablating it drops success from 39.2% to 27.5% on the diagnostic subset.
  • The worker backbone sets the capability ceiling, so improving single-worker GUI execution—especially reducing over-confident wrong completions—is the largest lever for further gains.
  • Scaling the worker count N mainly shortens the critical path without monotonically changing aggregate success, suggesting parallelism should be tuned per task.
  • The planner-worker architecture transfers to serial settings, yielding a 7.6-point gain over a strong GUI-only baseline on a 369-task standard suite, with losses confined to applications needing continuous visual context.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors leave implicit is adaptive gating: switch between serial and parallel execution per task or per round, which could recover the few tasks where N=1 uniquely succeeds while keeping parallel efficiency elsewhere.
  • The paper's trick of reformulating transient-state tasks into checkpoint-verifiable ones (e.g., 'bookmark the page' instead of 'reach the page') is a general benchmark-design pattern for reliable rule-based scoring.
  • The failure analysis implies a testable extension: add a verification pass where the planner or a second worker inspects the final environment state before committing an answer; the current system does not do this, and most failures are silent miscompletions.
  • The headline token savings depend on the reference baseline: against a cheap but over-confident worker, ParaGUI trades tokens for accuracy, while against a slower, better-calibrated model it wins on both axes—so fair comparisons should normalize visual-history budgets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces ParaGUIBench, claimed to be the first benchmark for parallel execution and coordination of multiple GUI agents on separate desktop instances, with 233 tasks across six categories, a multi-container Docker infrastructure with shared file system, and evaluation metrics covering success rate, critical-path steps, parallelism degree, and token cost. It also presents ParaGUI, a planner–worker agent with adaptive round-based planning, self-contained dispatch (SCD), and disjoint-region assignment. The headline empirical claim is that ParaGUI reaches 46.4% success on ParaGUIBench, outperforming the strongest serial GUI-only baseline (Claude Sonnet 4.6, 33.5%) by 12.9 points while using roughly half the steps and less than half the tokens, and that this demonstrates that parallel execution can improve both success rate and efficiency on decomposable long-horizon tasks. A transfer experiment on OSWorld with N=1 reports 59.1% vs. 51.5% for a Seed-1.8 GUI-only baseline.

Significance. The benchmark infrastructure is a genuine and useful contribution: it adapts OSWorld-style environments to genuinely concurrent GUI workers, uses rule-based evaluators that inspect environment state rather than agent self-reports, and reports efficiency metrics beyond success rate. The paper also performs careful ablations (planner vs. worker, SCD, N sweep, visual-history budget) and a paired agree-subset analysis, plus an OSWorld transfer with N=1. These are real strengths. However, the central causal claim in the abstract—that parallel execution improves success rate—is not supported by the paper's own ablations. The evidence instead supports a more modest claim: a planner–worker architecture with self-contained dispatch improves success rate, while increasing the number of parallel workers mainly reduces critical-path steps at roughly constant or slightly lower success. The difference matters because the headline comparison confounds the addition of a planner with the addition of parallel dispatch. The contribution is salvageable and likely valuable, but the paper's framing needs substantive revision.

major comments (4)
  1. [Abstract / §V-E, Table IX] RQ4's N-ablation directly contradicts the abstract's success-rate claim. On the 51-task subset, ParaGUI achieves SR 41.2% at N=1, 39.2% at N=3, and 39.2% at N=5, while critical-path steps fall monotonically (60.5, 39.0, 33.3). Adding concurrent workers shortens the critical path but does not improve—and slightly lowers—success rate. Table VII also shows SR nearly flat across parallel-pattern classes (45.7–48.0%) despite parallelism degree spanning 1.09–2.28. The +12.9-point gain over Claude Sonnet 4.6 therefore comes from planner decomposition and SCD, not parallel execution. Please reword the headline claim or add a full-benchmark N=1 control.
  2. [§V-A/b, Table III] The headline comparison conflates adding a planner with parallel dispatch. Serial baselines are GUI-only agents with no planner; ParaGUI has a GPT-5.4 planner plus N=5 workers. No planner–worker serial (N=1) configuration is run on the full 233-task benchmark, so the +18.5-point gain vs. Seed-1.8 and +12.9-point gain vs. Claude cannot be attributed to parallelization. The OSWorld transfer result (N=1, §V-F) is consistent with the alternative reading that decomposition itself drives most of the gain. A full-benchmark N=1 planner–worker run is needed to support the statement that 'parallel execution can improve success rate.'
  3. [§III-C-b, Eq. (1); Table IV] The step reduction metric is cross-system by construction. S = L_serial/L_parallel compares a single-agent GUI-only trajectory with a planner–worker trajectory; the paper itself notes that S contrasts two different agent systems. The reported efficiency gain therefore does not isolate parallelization. The paired agree-subset results confirm this: vs. Seed-1.8 (the worker backbone), ParaGUI has S=0.82 on both-correct web search and 0.82–0.96 on both-fail rows, with token costs 2.0–4.35× higher (Table IV). Large step/token reductions appear only against Claude Sonnet 4.6, which differs in planner, worker, and visual-history budget (Table V, §V-A-c). Report S against a planner–worker N=1 baseline and distinguish 'parallel execution' from 'planner–worker architecture' in the efficiency claims.
  4. [§VI Limitations; §V-E Table IX] The paper's own limitations section concedes that the worker backbone is the dominant bottleneck, that larger N mainly improves efficiency at near-constant accuracy, and that SCD is the main capability lever (Table IX: removing SCD drops SR from 39.2% to 27.5%). These statements are in tension with the abstract's unqualified causal sentence. The abstract and RQ1 conclusion should be revised to match the evidence, e.g., state that the planner–worker architecture with SCD improves success, while parallel workers improve efficiency.
minor comments (4)
  1. [§III-C-b Eq. (1)] The corpus-level S is defined as the ratio of mean L_serial to mean L_parallel. Reporting the distribution or mean of per-task ratios with confidence intervals would be more informative, since the ratio of means can be dominated by a few long tasks.
  2. [§V-B / Table VIII] Table VIII reports average steps on failed tasks computed over each system's own failed set. The paper notes the failed sets differ, but the reader should be reminded that the comparison is descriptive, not matched.
  3. [References] Minor typo in reference [4]: 'potentioal' should be 'potential'. Also, some references are to preprint/technical-report sources with no archive DOI; please ensure they will be accessible to readers.
  4. [§V-A-c / Table XII] The image=3 visual-history ablation is useful and rules out one confound. However, it is run only on the 51-task subset and does not address the planner-model and worker-model confounds; the main-text wording should not imply that the token comparison is fully controlled.

Circularity Check

0 steps flagged

No significant circularity: core results are measured outcomes, not fitted inputs; the abstract causal wording is contradicted by the paper's own N-ablation, but that is a validity issue, not a circular derivation.

full rationale

The paper's central quantities - success rate, critical-path steps, token cost, and parallelism degree - are measured outcomes obtained by running defined systems on a defined benchmark, not parameters fitted to those outcomes. The step reduction ratio S is explicitly defined as L_serial/L_parallel between a single-agent GUI-only baseline and the planner-worker ParaGUI system (Section III-C-b); while this ratio does not isolate the parallel-dispatch intervention from the planner intervention, it is a comparison of measured trajectories, not a construction that assumes the conclusion. Likewise, the parallelism degree P = L_total/L_parallel is a measured ratio of worker steps to critical-path steps, and the paper reports N=1/3/5 ablations showing that larger N mainly shortens the critical path without improving success rate (RQ4, Table IX). That internal evidence undermines the abstract's causal claim that parallel execution improves success rate, but an internally inconsistent comparison is an experimental-validity and attribution concern, not circularity: the arrow of reasoning runs from observations to claims, not from definitions to conclusions. The benchmark tasks are partly adapted from VeriWeb, OSWorld, and WebMall, and the rule-based evaluators are described operationally rather than being the quantities the paper claims to predict. There is no load-bearing self-citation chain, no imported uniqueness theorem, and no fitted parameter renamed as a prediction. The OSWorld transfer experiment (RQ5) provides an external grounding for the planner-worker architecture. Therefore, no step in the paper's derivation reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No physical or conceptual entities are invented. The benchmark and agent are systems, not postulated entities; there are no new forces, particles, dimensions, or conserved quantities. The hand-set configurations in the ledger are choices, not fitted scientific constants.

free parameters (3)
  • Default worker count N = 5
    Hand-chosen default in main experiments; ablation N=1/3/5 shows it shortens critical path without improving success rate.
  • Per-invocation step cap = 25
    Hand-set at Section IV-C-a; balances sub-task completion vs token cost and shapes failure rates.
  • Global GUI-step budget = 200
    Hand-set at Section IV-C-a; stops infinite dispatch and affects success/efficiency on long tasks.
axioms (5)
  • domain assumption The 233 tasks and their rule-based evaluators constitute a valid, unambiguous measure of GUI-task success
    Section III-B4 (uniqueness/shortcut filters) and III-C-e (rule-based verification). If evaluators are unreliable or tasks biased, all reported success rates are affected.
  • domain assumption Running each worker in a separate Docker desktop with a shared file system is sufficient for coordination without cross-worker interference
    Section III-A. Search & write results show this assumption fails when workers edit the same document.
  • domain assumption Text-only self-contained sub-tasks preserve the visual state a worker needs
    Section IV-B self-contained dispatch; the paper's own OSWorld analysis identifies visual continuity as a missing factor on Chrome/GIMP, so the assumption is load-bearing.
  • domain assumption The 51-task subset approximates the full benchmark's domain mix
    Section V-A-d; ablation conclusions are based on this hand-selected subset, not the full 233 tasks.
  • ad hoc to paper Comparing systems under each model's official visual-history retention policy is fair
    Section V-A-c and Appendix C defend not normalizing visual history; if this is wrong, the token-efficiency headline would change.

pith-pipeline@v1.3.0-alltime-deepseek · 23165 in / 14888 out tokens · 158227 ms · 2026-08-01T21:55:31.110996+00:00 · methodology

0 comments
read the original abstract

Graphical user interface (GUI) agents are systems powered by large multimodal models (LMMs). They perceive screen state and execute user instructions through GUI actions such as clicking, typing, and scrolling on desktops and mobile devices. However, current agents scale poorly to long-horizon tasks: actions incur costly LMM inferences, and performance degrades as context grows. Humans divide such workloads among collaborators who complete sub-tasks in parallel. Yet parallel coordination among GUI agents has received little attention. To close this gap, we introduce ParaGUIBench, to our knowledge, the first benchmark dedicated to parallel execution and coordination of multiple GUI agents on separate desktop instances. It consists of three components: a multi-device Docker infrastructure with a shared file system; a dataset of 233 tasks spanning six task categories; and an evaluation system with efficiency metrics, including step reduction ratio and token cost. We further introduce ParaGUI, a planner-worker agent that decomposes GUI tasks and dispatches sub-tasks to concurrent workers on separate desktop instances. On ParaGUIBench, ParaGUI reaches a 46.4% success rate, outperforming the strongest serial baseline (Claude Sonnet 4.6) by 12.9 points while using roughly half the steps and less than half the tokens. These results show that parallel execution can improve both success rate and efficiency on decomposable, long-horizon GUI tasks, pointing to a direction worth further study.

Figures

Figures reproduced from arXiv: 2607.22689 by Chenrui Shi, Huiming Wu, Liuyu Xiang, Qianxing Li, Yang Liu, Yubo Fu, Yuhao Fei, Yujie Wei, Zedong Yu, Zhaofeng He, Zhi Gao.

Figure 1
Figure 1. Figure 1: Sequential vs. parallel GUI execution. (A) A traditional agent runs a long-horizon task as one serial chain of perception–decision–action steps. (B) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The ParaGUI planner–worker architecture. Each round, the planner dispatches a batch of sub-tasks to up to [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A successful ParaGUI trace on ParaGUIBench. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: A silent miscompletion on ParaGUIBench. The planner decomposes correctly, and the [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation on the 51-task subset in the capability–efficiency plane. Vertical: Success Rate ( [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 6 linked inside Pith

  1. [1]

    OpenAI GPT-5 system card,

    A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthramet al., “OpenAI GPT-5 system card,”arXiv preprint arXiv:2601.03267, 2026

    Pith/arXiv arXiv 2026

  2. [2]

    Anthropic economic index report: Uneven geographic and enterprise ai adoption,

    R. Appel, P. McCrory, A. Tamkin, M. McCain, T. Neylon, and M. Stern, “Anthropic economic index report: Uneven geographic and enterprise ai adoption,”arXiv preprint arXiv:2511.15080, 2025

    arXiv 2025

  3. [3]

    Speculative futures of education: utopian and dystopian scenarios envisioned by ChatGPT, Gemini, and DeepSeek,

    J. M. S. Wong, “Speculative futures of education: utopian and dystopian scenarios envisioned by ChatGPT, Gemini, and DeepSeek,”Discover Education, vol. 4, no. 1, p. 261, 2025

    2025

  4. [4]

    Qwen 2.5: A comprehensive review of the leading resource- efficient LLM with potentioal to surpass all competitors,

    I. Ahmed, S. Islam, P. P. Datta, I. Kabir, N. U. R. Chowdhury, and A. Haque, “Qwen 2.5: A comprehensive review of the leading resource- efficient LLM with potentioal to surpass all competitors,”Authorea Preprints, 2025

    2025

  5. [5]

    Kimi k2.5: Visual agentic intelligence,

    K. Team, T. Bai, Y . Bai, Y . Baoet al., “Kimi k2.5: Visual agentic intelligence,” 2026. [Online]. Available: https://arxiv.org/abs/2602.02276

    Pith/arXiv arXiv 2026

  6. [6]

    Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods,

    Y . Cao, H. Zhao, Y . Cheng, T. Shu, Y . Chen, G. Liu, G. Liang, J. Zhao, J. Yan, and Y . Li, “Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods,”IEEE Trans. Neural Netw. Learn. Syst., vol. 36, no. 6, pp. 9737–9757, Jun. 2025

    2025

  7. [7]

    A survey on vision- language-action models for embodied AI,

    Y . Ma, Z. Song, Y . Zhuang, J. Hao, and I. King, “A survey on vision- language-action models for embodied AI,”IEEE Trans. Neural Netw. Learn. Syst., 2026, early access. PREPRINT, JULY 2026 13

    2026

  8. [8]

    OSWorld-Human: Benchmarking the efficiency of computer-use agents,

    R. Abhyankar, Q. Qi, and Y . Zhang, “OSWorld-Human: Benchmarking the efficiency of computer-use agents,”arXiv preprint arXiv:2506.16042, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Survey on efficient large language models: Principles, algorithms, applications, and open issues,

    J. Cheng, H. Kang, Y . Shao, N. Li, P. Chen, R. Wang, S. Long, X. Yang, and L. Ma, “Survey on efficient large language models: Principles, algorithms, applications, and open issues,”IEEE Trans. Neural Netw. Learn. Syst., vol. 37, no. 5, pp. 2025–2045, May 2026

    2025

  10. [10]

    OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments,

    T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Leiet al., “OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments,”Advances in Neural Information Processing Systems, vol. 37, pp. 52 040–52 094, 2024

    2024

  11. [11]

    WebArena: A realistic web environment for building autonomous agents,

    S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y . Bisk, D. Fried, U. Alon, and G. Neubig, “WebArena: A realistic web environment for building autonomous agents,” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  12. [12]

    VisualWebArena: Evaluating multimodal agents on realistic visual web tasks,

    J. Y . Koh, R. Lo, L. Jang, V . Duvvur, M. Lim, P.-Y . Huang, G. Neubig, S. Zhou, R. Salakhutdinov, and D. Fried, “VisualWebArena: Evaluating multimodal agents on realistic visual web tasks,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 881–905

    2024

  13. [13]

    Mind2Web: Towards a generalist agent for the web,

    X. Deng, Y . Gu, B. Zheng, S. Chen, S. Stevens, B. Wang, H. Sun, and Y . Su, “Mind2Web: Towards a generalist agent for the web,”Advances in Neural Information Processing Systems, vol. 36, pp. 28 091–28 114, 2023

    2023

  14. [14]

    Windows Agent Arena: Evaluating multi-modal OS agents at scale,

    R. Bonatti, D. Zhao, F. Bonacci, D. Dupont, S. Abdali, Y . Li, Y . Lu, J. Wagle, K. Koishida, A. Bucker, L. Jang, and Z. Hui, “Windows Agent Arena: Evaluating multi-modal OS agents at scale,” inProceedings of the 42nd International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 267, 2025, pp. 4874– 4910

    2025

  15. [15]

    AndroidWorld: A dynamic benchmarking environment for autonomous agents,

    C. Rawles, S. Clinckemaillie, Y . Chang, J. Waltz, G. Lau, M. Fair, A. Li, W. Bishop, W. Li, F. Campbell-Ajalaet al., “AndroidWorld: A dynamic benchmarking environment for autonomous agents,” inInternational Conference on Learning Representations (ICLR), 2025

    2025

  16. [16]

    Holistic agent leaderboard: The missing infrastructure for AI agent evaluation,

    S. Kapoor, B. Stroebl, P. Kirgis, N. Nadgir, Z. S. Siegel, B. Wei, T. Xue, Z. Chen, F. Chen, S. Utpalaet al., “Holistic agent leaderboard: The missing infrastructure for AI agent evaluation,” inInternational Conference on Learning Representations (ICLR), 2026

    2026

  17. [17]

    WABER: Evaluating reliability and efficiency of web agents with existing benchmarks,

    S. Kara, F. Faisal, and S. Nath, “WABER: Evaluating reliability and efficiency of web agents with existing benchmarks,” inICLR 2025 Workshop on Foundation Models in the Wild, 2025

    2025

  18. [18]

    An LLM compiler for parallel function calling,

    S. Kim, S. Moon, R. Tabrizi, N. Lee, M. W. Mahoney, K. Keutzer, and A. Gholami, “An LLM compiler for parallel function calling,” in Forty-first International Conference on Machine Learning, 2024

    2024

  19. [19]

    DynTaskMAS: A dynamic task graph- driven framework for asynchronous and parallel LLM-based multi-agent systems,

    J. Yu, Y . Ding, and H. Sato, “DynTaskMAS: A dynamic task graph- driven framework for asynchronous and parallel LLM-based multi-agent systems,” inProceedings of the International Conference on Automated Planning and Scheduling, vol. 35, no. 1, 2025, pp. 288–296

    2025

  20. [20]

    Plan-over- Graph: Towards parallelable LLM agent schedule,

    S. Zhang, X. Ma, Z. Cao, Z. Zhang, and H. Zhao, “Plan-over- Graph: Towards parallelable LLM agent schedule,”arXiv preprint arXiv:2502.14563, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Flow: Modularized agentic workflow automation,

    B. Niu, Y . Song, K. Lian, Y . Shen, Y . Yao, K. Zhang, and T. Liu, “Flow: Modularized agentic workflow automation,” inThe Thirteenth International Conference on Learning Representations (ICLR), 2025

    2025

  22. [22]

    MetaGPT: Meta programming for a multi-agent collaborative framework,

    S. Hong, M. Zhuge, J. Chen, X. Zheng, Y . Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Linet al., “MetaGPT: Meta programming for a multi-agent collaborative framework,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  23. [23]

    MegaA- gent: A large-scale autonomous LLM-based multi-agent system without predefined SOPs,

    Q. Wang, T. Wang, Z. Tang, Q. Li, N. Chen, J. Liang, and B. He, “MegaA- gent: A large-scale autonomous LLM-based multi-agent system without predefined SOPs,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025

    2025

  24. [24]

    GCM: Interpretable multiagent reinforcement learning via graph cooperation modeling,

    X. Wu, Y . Zhu, C. Chen, and C. Chen, “GCM: Interpretable multiagent reinforcement learning via graph cooperation modeling,”IEEE Trans. Neural Netw. Learn. Syst., vol. 37, no. 4, pp. 1823–1837, Apr. 2026

    2026

  25. [25]

    Solving scalable multiagent routing problems with reinforcement learning,

    Y . Hu, Y . Yao, J. Chen, Z. Wang, Q. Jia, and Y . Pan, “Solving scalable multiagent routing problems with reinforcement learning,”IEEE Trans. Neural Netw. Learn. Syst., vol. 36, no. 11, pp. 19 604–19 618, Nov. 2025

    2025

  26. [26]

    UFO 3: Weaving the digital agent galaxy,

    C. Zhang, L. Li, H. Huang, C. Ni, B. Qiao, S. Qin, Y . Kang, M. Ma, Q. Lin, S. Rajmohanet al., “UFO 3: Weaving the digital agent galaxy,” arXiv preprint arXiv:2511.11332, 2025

    arXiv 2025

  27. [27]

    Flash-Searcher: Fast and effective web agents via DAG-based parallel execution,

    T. Qin, Q. Chen, S. Wang, H. Xing, K. Zhu, H. Zhu, D. Shi, X. Liu, G. Zhang, J. Liuet al., “Flash-Searcher: Fast and effective web agents via DAG-based parallel execution,” inInternational Conference on Learning Representations (ICLR), 2026

    2026

  28. [28]

    Graph-enhanced large language models in asynchronous plan reasoning,

    F. Lin, E. La Malfa, V . Hofmann, E. M. Yang, A. Cohn, and J. B. Pierrehumbert, “Graph-enhanced large language models in asynchronous plan reasoning,” inProceedings of the 41st International Conference on Machine Learning (ICML), 2024

    2024

  29. [29]

    Robotouille: An asynchronous planning benchmark for LLM agents,

    G. Gonzalez-Pumariega, L. S. Yean, N. Sunkara, and S. Choudhury, “Robotouille: An asynchronous planning benchmark for LLM agents,” in International Conference on Learning Representations (ICLR), 2025

    2025

  30. [30]

    AsyncTool: Evaluating the asynchronous function calling capability under multi-task scenarios,

    K. Shi, Z. Zhang, S. Huang, A. Nie, Z. Fang, Q. Wang, L. Chen, H. Chen, Z. Chen, and F. Zhao, “AsyncTool: Evaluating the asynchronous function calling capability under multi-task scenarios,”arXiv preprint arXiv:2605.27995, 2026

    Pith/arXiv arXiv 2026

  31. [31]

    MultiAgentBench: Evaluating the collaboration and competition of LLM agents,

    K. Zhu, H. Du, Z. Hong, X. Yang, S. Guo, D. Z. Wang, Z. Wang, C. Qian, R. Tang, H. Jiet al., “MultiAgentBench: Evaluating the collaboration and competition of LLM agents,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 8580–8622

    2025

  32. [32]

    Collab-Overcooked: Benchmarking and evaluating large language models as collaborative agents,

    H. Sun, S. Zhang, L. Niu, L. Ren, H. Xu, H. Fu, F. Zhao, C. Yuan, and X. Wang, “Collab-Overcooked: Benchmarking and evaluating large language models as collaborative agents,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 4922–4951

    2025

  33. [33]

    Veriweb: Verifiable long-chain web benchmark for agentic information-seeking,

    S. Liu, M. Liu, H. Zhou, Z. Cui, Y . Zhou, Y . Zhou, J. Gao, H. Zhou, Y . Yang, W. Fan, P. Zhang, G. Zhang, J. Shi, W. Xuan, J. Huang, S. Luo, F. Wu, H. Qi, Q. Zeng, J. Wang, A. Feng, J. Lv, S. Jiang, Z. Ren, W. Zhou, Z. Yin, W. Zhang, G. Li, W. Yu, L. Ma, L. Bai, Q. Lin, M. Song, and D. Tao, “Veriweb: Verifiable long-chain web benchmark for agentic infor...

    arXiv 2025

  34. [34]

    WebMall – a multi-shop benchmark for evaluating web agents,

    R. Peeters, A. Steiner, L. Schwarz, J. Y . Caspary, and C. Bizer, “WebMall – a multi-shop benchmark for evaluating web agents,” inProceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2026

    2026

  35. [35]

    Introducing GPT-5.4,

    OpenAI, “Introducing GPT-5.4,” https://openai.com/index/ introducing-gpt-5-4/, 2026, accessed: 2026-03-05

    2026

  36. [36]

    Seed-1.8 Model Card: Technical Specifications and Capabilities,

    ByteDance, “Seed-1.8 Model Card: Technical Specifications and Capabilities,” ByteDance, Tech. Rep., 2026, technical Report. [Online]. Available: https://lf3-static.bytednsdoc.com/obj/eden-cn/ lapzild-tss/ljhwZthlaukjlkulzlp/research/Seed-1.8-Modelcard.pdf

    2026

  37. [37]

    Qwen3-vl technical report,

    Qwen Team, “Qwen3-vl technical report,”arXiv preprint arXiv:2511.21631, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Holo3 – open foundation models for navigation and computer use agents,

    H Company, “Holo3 – open foundation models for navigation and computer use agents,” https://huggingface.co/Hcompany/Holo3-35B-A3B, 2026, accessed: 2026-06-04

    2026

  39. [39]

    Claude Sonnet 4.6,

    Anthropic, “Claude Sonnet 4.6,” https://www.anthropic.com/claude/ sonnet, 2026, accessed: 2026-06-04

    2026

  40. [40]

    Surfer: A world model-based framework for vision-language robot manipulation,

    P. Ren, K. Zhang, H. Zheng, Z. Li, Y . Wen, F. Zhu, S. Ma, and X. Liang, “Surfer: A world model-based framework for vision-language robot manipulation,”IEEE Trans. Neural Netw. Learn. Syst., vol. 36, no. 12, pp. 20 092–20 104, Dec. 2025. APPENDIXA EXTENDEDEXPERIMENTALANALYSIS A. Efficiency by Parallel-Execution Pattern A breakdown by parallel class × cate...

    2025