Pith. sign in

REVIEW 19 cited by

APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.18518 v1 pith:DJQ47AEO submitted 2024-06-26 cs.CL cs.AIcs.LGcs.SE

Zuxin Liu , Thai Hoang , Jianguo Zhang , Ming Zhu , Tian Lan , Shirley Kokane , Juntao Tan , Weiran Yao
show 9 more authors
Zhiwei Liu Yihao Feng Rithesh Murthy Liangwei Yang Silvio Savarese Juan Carlos Niebles Huan Wang Shelby Heinecke Caiming Xiong
This is my paper · ORCID
classification cs.CLcs.AIcs.LGcs.SE
keywords datasetsfunction-callingapigendatasetdiversehigh-qualitymodelsagent
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ISE: An Execution-Grounded Recipe for Multi-Turn OS-Agent Trajectories

    cs.CL 2026-06 conditional novelty 7.0 of 10

    ISE creates 23,132 execution-grounded multi-turn OS agent trajectories via intent simulation and live execution, improving agent performance on ClawEval from 19.3 to 37.7 pass@1 with Qwen3-8B.

  2. When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use

    cs.CL 2026-08 conditional novelty 6.0 of 10

    On a new multilingual function-calling benchmark, supervised fine-tuning largely resolves argument language mismatch, and reinforcement learning adds targeted but incremental gains.

  3. Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Calls

    cs.SE 2026-02 conditional novelty 6.0 of 10

    A simulated tool environment with argument validation, response synthesis, and task-state feedback improves LLM tool-call accuracy at test time.

  4. SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents

    cs.AI 2025-09 conditional novelty 6.0 of 10

    A 20B autonomously reasoning deep-research agent trained with synthetic-data RL reaches 28.7% on Humanity's Last Exam, exceeding several larger and proprietary baselines.

  5. MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models

    cs.IR 2025-07 conditional novelty 6.0 of 10

    A multi-task retriever that combines tool-usage detection with query-centered graph and search-based modules improves tool retrieval accuracy over prior baselines.

  6. ToolACE-DEV: Self-Improving Tool Learning via Decomposition and EVolution

    cs.CL 2025-05 conditional novelty 6.0 of 10

    ToolACE-DEV decomposes tool learning into documentation adaption, tool generation, and invocation, then lets the model self-train on its own majority-voted invocations, improving function-calling accuracy across three...

  7. Advancing and Benchmarking Personalized Tool Invocation for LLMs

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Fine-tuning Qwen2.5-7B on PTool-synthesized data raises its PTBench overall score from 0.074 to 0.268, including on users not seen in training.

  8. Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling

    cs.AI 2025-04 conditional novelty 6.0 of 10

    Fine-tuned small language models can reach high accuracy on function calling, but zero-shot and few-shot performance is poor, and the study's few-shot results are compromised by using test-set examples in the prompt.

  9. When2Call: When (not) to Call Tools

    cs.CL 2025-04 conditional novelty 6.0 of 10

    When2Call measures when language models should call tools versus ask questions or refuse, and shows that RPO training substantially improves this decision-making.

  10. Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A shuffle-based token classifier plus group-level loss reweighting improves supervised fine-tuning of LLM agents on tool-use benchmarks.

  11. The "I Don't Know" Filter: Enhancing Agentic Reliability in Function Calling

    cs.SE 2026-07 conditional novelty 5.5 of 10

    A multi-sample uncertainty classifier filter improves agent reliability (IDKS) by abstaining on uncertain function calls across open SLMs and BFCL-style benchmarks.

  12. RAG Strategies for Natural Language-Based SQL Query and REST API Call Generation

    cs.SE 2026-02 conditional novelty 5.0 of 10

    In a 631-case enterprise benchmark, iterative retrieval (CoRAG) achieved 10.29% exact match vs 7.45% for standard RAG on combined SQL/API generation with hybrid documentation, while no-RAG gives 0% exact match.

  13. ADC: Enhancing Function Calling Via Adversarial Datasets and Code Line-Level Feedback

    cs.SE 2024-12 reject novelty 5.0 of 10

    ADC fine-tunes Llama-3.1-8B with code execution traces and adversarial function-call data, improving BFCL v2 overall from 63.19 to 79.01, but with no released artifacts or error bars.

  14. Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Typhoon 2 improves Thai LLM performance through continual pre-training on curated Thai data and post-training, releasing text, vision, audio, and safety models.

  15. Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translation

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Fine-tuning experiments show instruction-following data improves LLM function-calling accuracy and relevance detection, a Decision Token plus synthetic negative examples helps non-relevant cases, and a tailored transl...

  16. The Aloe Family Recipe for Open and Specialized Healthcare LLMs

    cs.CL 2025-05 conditional novelty 4.0 of 10

    Aloe Beta, a family of open-weights health LLMs built from Llama 3.1 and Qwen 2.5, matches or exceeds closed medical models on MCQA benchmarks while improving safety via DPO.

  17. A Survey of AI Agent Protocols

    cs.AI 2025-04 conditional novelty 4.0 of 10

    The paper proposes a two-dimensional classification of AI agent protocols and qualitatively compares them without carrying out measurements.

  18. The Breeze 2 Herd of Models: Traditional Chinese LLMs Based on Llama with Vision-Aware and Function-Calling Capabilities

    cs.CL 2025-01 conditional novelty 3.0 of 10

    Breeze2 continues pretraining Llama on a 900GB Traditional Chinese corpus, then adds LLaVA-style vision and function-calling fine-tuning, reporting top size-class scores on a mix of public and self-built benchmarks.

  19. Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline

    cs.AI 2024-12 reject novelty 3.0 of 10

    A LoRA fine-tuned 7B model trained on AI-synthesized enterprise HR API data beat GPT-4 and GPT-4o on the authors' private benchmark.

Pith tools