REVIEW 12 cited by
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The massive successes of large language models (LLMs) encourage the emerging exploration of LLM-augmented Autonomous Agents (LAAs). An LAA is able to generate actions with its core LLM and interact with environments, which facilitates the ability to resolve complex tasks by conditioning on past interactions such as observations and actions. Since the investigation of LAA is still very recent, limited explorations are available. Therefore, we provide a comprehensive comparison of LAA in terms of both agent architectures and LLM backbones. Additionally, we propose a new strategy to orchestrate multiple LAAs such that each labor LAA focuses on one type of action, \textit{i.e.} BOLAA, where a controller manages the communication among multiple agents. We conduct simulations on both decision-making and multi-step reasoning environments, which comprehensively justify the capacity of LAAs. Our performance results provide quantitative suggestions for designing LAA architectures and the optimal choice of LLMs, as well as the compatibility of both. We release our implementation code of LAAs to the public at \url{https://github.com/salesforce/BOLAA}.
Forward citations
Cited by 12 Pith papers
-
Reproducing human biases in route choice using large language models: Toward scalable behavioral modeling
LLM agents with demographic profiles reproduce CPT-style risk attitudes in route choice and yield fitted parameters (α=0.4, β=0.64, λ=1.43) that predict human data competitively.
-
LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability
A benchmark for LLM agents in partially observable joint decision-making reveals that deliberation challenges current models but can enable reflection and error correction.
-
CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent with Decoupled Reinforcement Learning
A decoupled planner-executor GUI agent, trained by per-app reinforcement learning followed by specialist-to-generalist distillation, lifts ScienceBoard success from about 7.6% to 21.0% average and 40% pass@8.
-
SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
A self-evolving computer-use agent trained with full-trajectory state judging and curriculum task generation goes from 11.3% to 34.5% average success on five OSWorld apps, and a specialist-to-generalist variant beats ...
-
MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models
MCPEval is an automated MCP-based framework that generates, verifies, and scores LLM agent tool-use tasks; its experiments reveal a consistent gap between how well agents execute tool calls and how well they synthesiz...
-
PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization
Pseudocode-structured plans paired with preference optimization improve LLM agent success rates and generalization across interactive benchmarks.
-
NADER: Neural Architecture Design via Multi-Agent Collaboration
NADER uses a multi-agent LLM team with a graph-based block representation and a reflection memory to iteratively propose and test modified neural architectures, claiming gains beyond NAS-Bench-201's optimum on CIFAR a...
-
Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.
-
DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent
A synthetic benchmark of 9,000 multi-hop web-research questions with difficulty tiers and teacher-generated search trajectories, plus an open-source RL training framework that reportedly lets 3B-parameter agents beat ...
-
A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications
A survey of 80+ Deep Research systems that proposes a four-layer taxonomy (foundation models, tool use, planning, synthesis) and compares commercial and open-source implementations.
-
Flow: Modularized Agentic Workflow Automation
Flow represents a task as a dependency graph of subtasks and lets LLM agents redraw that graph during execution, reporting better success rates than three baselines on three coding tasks.
-
A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application
This survey organizes recent LLM-based multi-agent research into task-solving, simulation, and agent-evaluation applications, and identifies efficiency and evaluation gaps as key open problems.
Discussion (0). Continue with ORCID to comment.