REVIEW 9 cited by
OAgents: An Empirical Study of Building Effective Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
OAgents: An Empirical Study of Building Effective Agents
read the original abstract
Recently, Agentic AI has become an increasingly popular research field. However, we argue that current agent research practices lack standardization and scientific rigor, making it hard to conduct fair comparisons among methods. As a result, it is still unclear how different design choices in agent frameworks affect effectiveness, and measuring their progress remains challenging. In this work, we conduct a systematic empirical study on GAIA benchmark and BrowseComp to examine the impact of popular design choices in key agent components in a fair and rigorous manner. We find that the lack of a standard evaluation protocol makes previous works, even open-sourced ones, non-reproducible, with significant variance between random runs. Therefore, we introduce a more robust evaluation protocol to stabilize comparisons. Our study reveals which components and designs are crucial for effective agents, while others are redundant, despite seeming logical. Based on our findings, we build and open-source OAgents, a new foundation agent framework that achieves state-of-the-art performance among open-source projects. OAgents offers a modular design for various agent components, promoting future research in Agentic AI.
Forward citations
Cited by 9 Pith papers
-
IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agents
IdleSpec improves LLM agent accuracy by generating and aggregating speculative plans during idle time between tool calls and observations using complementary drafting strategies.
-
Inference-Time Budget Control for LLM Search Agents
A VOI-based controller for dual inference budgets improves multi-hop QA performance by prioritizing search actions and selectively finalizing answers.
-
AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP
A tool that injects faults into an LLM agent's MCP tool calls, replays them under mitigation, and verifies the fix—revealing that agents mostly fail silently on stale or bad data.
-
AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP
AgentCheck turns an MCP server into a fault-injection surface so developers can reproduce tool failures, test mitigations, and verify fixes before deployment.
-
AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP
AgentCheck turns MCP tool servers into a fault-injection surface so developers can reproduce agent failures, test mitigations, and confirm fixes under identical tool faults.
-
EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies
EcoGym is a new open benchmark with three economic environments that reveals no leading LLM dominates at sustained plan-and-execute decision making across scenarios.
-
ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Emergent Adaptation
An LLM agent that can call a reconfiguration tool to update its sub-goals, toolbox, strategy, and context outperforms static-config agents across FRAMES, xbench, GAIA, and SWE-bench Lite.
-
WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
WebWatcher introduces a vision-language deep research agent trained on synthetic multimodal trajectories and RL that outperforms baselines on VQA benchmarks, along with a new BrowseComp-VL evaluation.
-
Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training
Cognitive Kernel-Pro provides an open-source agent framework with curated training data across web, file, code, and reasoning domains plus test-time reflection and voting, achieving SOTA results on GAIA among free agents.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.