MCPEval is an automated MCP-based framework that generates, verifies, and scores LLM agent tool-use tasks; its experiments reveal a consistent gap between how well agents execute tool calls and how well they synthesize final answers.
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Large Action models are essential for enabling autonomous agents to perform complex tasks. However, training such models remains challenging due to the diversity of agent environments and the complexity of noisy agentic data. Existing infrastructure offers limited support for scalable, agent-specific fine-tuning and standardized agent data processing. We introduce ActionStudio, a lightweight and extensible data and training framework designed for large action models. ActionStudio unifies diverse agent trajectories using our proposed Unified Format 2.0, supports a range of training workflows with optimized multi-node distributed setup, and integrates robust preprocessing and real-time verification tools. ActionStudio demonstrates up to 9x higher throughput compared to existing agentic training frameworks, and our trained models yield top performances across public and realistic agent benchmarks. To support the broader research community, we open-source the ActionStudio framework and release actionstudio-98k, a curated dataset of 98k high-quality trajectories. Code: https://github.com/SalesforceAIResearch/xLAM.
fields
cs.AI 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models
MCPEval is an automated MCP-based framework that generates, verifies, and scores LLM agent tool-use tasks; its experiments reveal a consistent gap between how well agents execute tool calls and how well they synthesize final answers.