Pith. sign in

REVIEW 27 cited by

ToolACE: Winning the Points of LLM Function Calling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.00920 v2 pith:T5P7AR3L submitted 2024-09-02 cs.LG cs.AIcs.CL

Weiwen Liu , Xu Huang , Xingshan Zeng , Xinlong Hao , Shuai Yu , Dexun Li , Shuai Wang , Weinan Gan
show 19 more authors
Zhengying Liu Yuanqing Yu Zezhong Wang Yuxian Wang Wu Ning Yutai Hou Bin Wang Chuhan Wu Xinzhi Wang Yong Liu Yasheng Wang Duyu Tang Dandan Tu Lifeng Shang Xin Jiang Ruiming Tang Defu Lian Qun Liu Enhong Chen
This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords datadiversemodelstoolaceaccuracycallingfunctionfunction-calling
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Function calling significantly extends the application boundary of large language models, where high-quality and diverse training data is critical for unlocking this capability. However, real function-calling data is quite challenging to collect and annotate, while synthetic data generated by existing pipelines tends to lack coverage and accuracy. In this paper, we present ToolACE, an automatic agentic pipeline designed to generate accurate, complex, and diverse tool-learning data. ToolACE leverages a novel self-evolution synthesis process to curate a comprehensive API pool of 26,507 diverse APIs. Dialogs are further generated through the interplay among multiple agents, guided by a formalized thinking process. To ensure data accuracy, we implement a dual-layer verification system combining rule-based and model-based checks. We demonstrate that models trained on our synthesized data, even with only 8B parameters, achieve state-of-the-art performance on the Berkeley Function-Calling Leaderboard, rivaling the latest GPT-4 models. Our model and a subset of the data are publicly available at https://huggingface.co/Team-ACE.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 27 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. rStar2-Agent: Agentic Reasoning Technical Report

    cs.CL 2025-08 conditional novelty 7.0 of 10

    A 14B model trained with agentic RL and a resample-on-correct rollout strategy scores 80.6% on AIME24 and 69.8% on AIME25, nearly matching DeepSeek-R1 (671B) in one week on 64 GPUs.

  2. DICE-BENCH: Evaluating the Tool-Use Capabilities of Large Language Models in Multi-Round, Multi-Party Dialogues

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A new benchmark and metric show that large language models still struggle to call tools when the needed details are scattered across multi-party, multi-round group dialogues.

  3. SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Information

    cs.CL 2026-08 conditional novelty 6.0 of 10

    A new human-curated benchmark for LLM mobile assistants shows models struggle to locate scattered personal information, with the best model at 57.3% accuracy.

  4. Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

    cs.CL 2026-08 conditional novelty 6.0 of 10

    Structuring spoken language understanding as function calling improves semantic extraction accuracy over traditional intent-and-slot formats for both text and audio models.

  5. Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

    cs.AI 2026-07 accept novelty 6.0 of 10

    Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.

  6. Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Calls

    cs.SE 2026-02 conditional novelty 6.0 of 10

    A simulated tool environment with argument validation, response synthesis, and task-state feedback improves LLM tool-call accuracy at test time.

  7. SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A two-phase interactive RL framework (DIRL) lets a 3B VLM learn to coordinate multiple vision and robot tools, reaching top benchmark scores and 86% real-robot pick-and-place success.

  8. MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models

    cs.IR 2025-07 conditional novelty 6.0 of 10

    A multi-task retriever that combines tool-usage detection with query-centered graph and search-based modules improves tool retrieval accuracy over prior baselines.

  9. FLOW-BENCH: Towards Conversational Generation of Enterprise Workflows

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A new 101-case benchmark and LLM pipeline for converting natural-language edits into business workflow definitions, with up to 0.83 exact match on the authors' own test set.

  10. ToolACE-DEV: Self-Improving Tool Learning via Decomposition and EVolution

    cs.CL 2025-05 conditional novelty 6.0 of 10

    ToolACE-DEV decomposes tool learning into documentation adaption, tool generation, and invocation, then lets the model self-train on its own majority-voted invocations, improving function-calling accuracy across three...

  11. CallNavi, A Challenge and Empirical Study on LLM Function Calling and Routing

    cs.SE 2025-01 conditional novelty 6.0 of 10

    CallNavi is a new benchmark for LLM function calling with unfiltered, nested, multi-step API tasks; GPT-4o leads the leaderboard and a two-step routing pipeline improves fine-tuned models.

  12. Evaluating Multi-Turn Bargain Skills in LLM-Based Seller Agent

    cs.AI 2025-09 conditional novelty 5.0 of 10

    BargainBench tests LLM seller agents on turn-level buyer intent recognition in synthetic e-commerce bargaining dialogues, where the best models score roughly 55 percent F1.

  13. MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations

    cs.CL 2025-07 conditional novelty 5.0 of 10

    MemTool is a short-term memory framework with three modes (autonomous, workflow, hybrid) that lets LLM agents add and remove tools across multi-turn conversations, evaluated over 100 turns on 13+ models.

  14. We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems

    cs.LG 2025-06 conditional novelty 5.0 of 10

    MCP-powered LLM agents are vulnerable to prompt injection from third-party services, and simple detection or filtering defenses do not reliably stop these attacks.

  15. CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios

    cs.SE 2025-06 conditional novelty 5.0 of 10

    CRITICTOOL, a benchmark of 2,740 tool-calling error scenarios built from BFCL and T-Eval with GPT-4o-based error injection, finds most LLMs rarely recover from tool-use errors, with GPT-4o best at 69.01 overall and to...

  16. CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learning

    cs.LG 2025-06 conditional novelty 5.0 of 10

    CheMatAgent uses hierarchical Monte Carlo tree search with separate policy and execution models, plus trained reward models, to improve tool selection and parameter filling on a new chemistry benchmark, ChemToolBench.

  17. Tools in the Loop: Quantifying Uncertainty of LLM Question Answering Systems That Use Tools

    cs.LG 2025-05 conditional novelty 5.0 of 10

    The paper proposes adding the entropy of an LLM's final answer to the entropy of an external tool's output as an uncertainty score for tool-calling QA systems, and shows it predicts answer correctness on synthetic and...

  18. A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

    cs.CR 2025-04 conditional novelty 5.0 of 10

    A large collaborative survey organizes LLM and LLM-agent safety issues into a full-stack lifecycle framework from data preparation to deployment.

  19. A Framework for Testing and Adapting REST APIs as LLM Tools

    cs.SE 2025-04 conditional novelty 5.0 of 10

    A framework that converts REST APIs into LLM-callable tools, generates data-aware natural-language test cases, and reports an error taxonomy from over 2,400 executions.

  20. Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Typhoon 2 improves Thai LLM performance through continual pre-training on curated Thai data and post-training, releasing text, vision, audio, and safety models.

  21. Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translation

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Fine-tuning experiments show instruction-following data improves LLM function-calling accuracy and relevance detection, a Decision Token plus synthetic negative examples helps non-relevant cases, and a tailored transl...

  22. AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applications

    cs.AI 2025-08 unverdicted novelty 4.0 of 10

    AgentScope 1.0 packages the components needed to build, evaluate, and deploy LLM agent applications into one developer framework.

  23. Routine: A Structural Planning Framework for LLM Agent System in Enterprise

    cs.AI 2025-07 conditional novelty 4.0 of 10

    A structured planning format called Routine sharply improves LLM agents' multi-step tool-calling accuracy in an enterprise scenario.

  24. Enhancing Accuracy and Maintainability in Nuclear Plant Data Retrieval: A Function-Calling LLM Approach Over NL-to-SQL

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A function-calling LLM that selects pre-approved SQL functions outperformed direct NL-to-SQL in human-evaluated correctness for nuclear plant data retrieval.

  25. MedOrch: Medical Diagnosis with Tool-Augmented Reasoning Agents for Flexible Extensibility

    cs.CL 2025-05 reject novelty 4.0 of 10

    MedOrch is a modular framework in which LLMs call medical tools to answer clinical questions; its headline results on Alzheimer's, chest X-ray, and VQA benchmarks are weakened by best-of-five scoring.

  26. ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agents

    cs.CL 2025-05 reject novelty 3.0 of 10

    ScaleMCP describes an agentic MCP tool-retrieval system, but its experiments show the proposed TDWA embedding loses to concatenation and the agentic retrieval loop is not evaluated.

  27. Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline

    cs.AI 2024-12 reject novelty 3.0 of 10

    A LoRA fine-tuned 7B model trained on AI-synthesized enterprise HR API data beat GPT-4 and GPT-4o on the authors' private benchmark.

Pith tools