Pith. sign in

REVIEW 3 major objections 6 minor 14 cited by

Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training a language model to call tools with a binary reward that checks only format and exact match produces 7B and 14B models that outperform GPT-4o on function-calling benchmarks.

desk verdict A useful empirical study of rule-based RL for tool calling, but the headline claims rest on single-run gaps with no error bars, so treat them as provisional. read the letter →

arxiv 2505.00024 v2 pith:L6GEWZBZ submitted 2025-04-25 cs.CL cs.AI

Shaokun Zhang , Yi Dong , Jieyu Zhang , Jan Kautz , Bryan Catanzaro , Andrew Tao , Qingyun Wu , Zhiding Yu
show 1 more author
Guilin Liu
This is my paper · ORCID
classification cs.CLcs.AI
keywords toolcallingfunctionrule-basedreinforcementlearningGRPObinaryrewardLLMreasoningSFTvsRLtool-usegeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a language model can learn to call external tools from a reward that only checks that its output has the right format and that the tool call exactly matches a single annotated answer, with no supervision on intermediate reasoning. The authors report that it can: models trained this way, Tool-N1-7B and Tool-N1-14B, outperform GPT-4o on the BFCL, API-Bank, and ACEBench benchmarks. They also compare supervised fine-tuning (SFT), reinforcement learning (RL), and their combination on 5,518 distilled reasoning trajectories, and find that pure RL matches or outperforms the widely used SFT-then-RL pipeline. If these results hold, lightweight verifiable rewards could replace expensive distilled reasoning annotations when training tool-using language models.

What carries the argument

The load-bearing mechanism is the binary rule-based reward of Eq. (5) combined with the GRPO policy update. The reward is computed after parsing the model's output into two tagged components: a <think> block and a <tool_call> block. Format correctness requires both tags to be present, and tool-call correctness requires the parsed call to match the ground-truth dictionary in tool name and in every argument key-value pair, with argument order free to vary. This single sparse signal, which gives no reward to the reasoning text itself, is what pushes the policy to discover its own reasoning strategies, and the paper's ablation shows that a fine-grained reward giving partial credit for format or function-name matches hurts performance, which the authors attribute to reward hacking.

What would settle it

Construct a test set from queries that admit two or more equally valid tool calls for the same user intent; if a model trained with Eq. (5) fails on the alternative valid calls while a model trained with an execution-based verifier succeeds, the exact-match uniqueness assumption is the bottleneck.

Watch

Extended reading notes

Core claim

This paper establishes that rule-based reinforcement learning with a binary reward is a strong training signal for tool-calling language models. The reward, defined in Eq. (5), returns 1 only if the model output wraps its reasoning in <think> tags, wraps the tool call in <tool_call> tags, and the parsed tool call exactly matches the ground-truth tool name and all argument key-value pairs; argument order is free to vary because matching is done on parsed dictionaries. The reasoning text itself receives no reward. Trained with the GRPO algorithm on data from xLAM and a ToolACE subset, the resulting models reach average accuracies of 84.82 (7B) and 85.97 (14B) on the Berkeley Function Calling Leaderboard (BFCL), against 83.97 for GPT-4o, and score 82.19 on API-Bank and 87.00 on ACEBench, again above GPT-4o. The paper further reports that in a controlled comparison on 5,518 distilled reasoning trajectories, pure RL (83.24 average BFCL) slightly edges out Reason-SFT+RL (83.17) and clearly beats both SFT variants, indicating that the commonly adopted SFT-then-RL recipe is not necessary for tool-calling performance.

Load-bearing premise

The method assumes that every training query has exactly one correct tool call and that the dataset's ground-truth annotations are clean and complete, because the reward gives full credit only to an exact match with that single annotation.

Editorial extensions

If this is right

  • Tool-calling capability can be trained with a sparse binary reward, so curated reasoning traces are not a prerequisite for strong function-calling performance.
  • The widely used SFT-then-RL recipe is not automatically superior to pure RL for tool calling; practitioners can skip the SFT warm-up without losing accuracy.
  • Open 7B and 14B models trained with this recipe outperform a closed frontier model (GPT-4o) on standard function-calling benchmarks.
  • The method scales with model size and transfers across backbones, with Qwen-based models benefiting more than LLaMA-based ones at the same scale.
  • Longer reasoning traces are not necessary for tool-use quality; response length stayed flat during training, contrasting with math-reasoning RL runs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is applying the same binary exact-match reward to other agentic actions with a unique verifiable target, such as SQL generation or GUI automation.
  • Because the reward depends on clean, unique ground-truth annotations, deliberately corrupting a fraction of the training tool calls would likely erode the RL advantage; the paper's ablations do not vary annotation quality.
  • The SFT-vs-RL comparison uses only 5,518 trajectories, so the 'pure RL is enough' conclusion is conditional on data budget; with substantially more SFT data the ordering could reverse.
  • The flat response-length curve across training suggests tool calling rewards short, focused reasoning rather than long chains, which would make the optimal rollout budget for tool agents different from math-reasoning RL.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Tool-N1, a series of tool-calling LLMs trained with rule-based reinforcement learning (GRPO) using a binary reward that checks output format and exact tool-call matching, without requiring distilled reasoning traces. The authors evaluate on BFCL, API-Bank, and ACEBench, and compare SFT, RL, and SFT-then-RL training recipes on a curated 5,518-instance subset of ToolACE. They claim that Tool-N1-7B/14B clearly outperform GPT-4o and that pure RL matches or exceeds the SFT-then-RL paradigm for tool calling.

Significance. If the empirical claims hold, this is a useful result for the tool-learning community: it suggests that a simple verification reward can substitute for expensive SFT on distilled reasoning trajectories, and it provides a controlled comparison of training recipes. The paper uses external benchmarks with official scripts, evaluates across model scales and backbones, and includes ablations on reward granularity and data composition. The main limitation is that the headline comparisons rest on single-run point estimates with gaps as small as 0.07 points, so the significance of the central findings depends on the stability of those numbers.

major comments (3)
  1. [§5.1, Tables 2–3] No uncertainty quantification is provided for any experiment, and the central comparisons rest on single-run point estimates. The key recipe comparison in Table 3 shows pure RL (83.24) ahead of Reason-SFT+RL (83.17) by 0.07 points, and the main BFCL gap between Tool-N1-7B and GPT-4o is 0.85 points (Table 2). These margins are within typical run-to-run variability for such benchmarks, so the abstract's 'clearly outperform' and Finding 2's 'stronger performance' are not supported by the evidence as presented. Please report mean±std over at least three independent training seeds, or otherwise demonstrate that the gaps are stable.
  2. [§5.3.2, Finding 2] The phrasing 'does not necessarily outperform' is a tautology, while the stronger claim that pure RL 'demonstrates stronger performance than any other combinations' is contradicted by the 0.07-point gap in Table 3. Moreover, the recipe study uses a single 5,518-instance dataset and a single evaluation benchmark, so the general conclusion about the 'widely adopted SFT-then-RL paradigm' is over-generalized. Either provide additional evidence across datasets and benchmarks or temper the conclusion to 'pure RL is competitive with SFT-then-RL under equal data budgets.'
  3. [§4.3, Eq. (5)] The reward is described as assessing 'functional correctness' of tool invocations, but it actually grants reward 1 only on an exact match of the tool name and all argument key-value pairs against a single ground-truth call. If the annotation is noisy or more than one tool call would satisfy the query, this binary reward is a biased proxy for functional correctness. This matters because the claimed advantage over SFT is attributed to the reward's flexibility. Please either rename the criterion as 'exact match against the reference tool call' or add a robustness analysis (e.g., a multi-annotation study or evaluation of alternative correct calls).
minor comments (6)
  1. [§4.2, Appendix E] The template example shows `["name": <function-name>, ...]` which is not valid JSON; use the array-of-objects syntax `[{"name": ..., "arguments": ...}]` as in Appendix D.
  2. [§5.1] The sentence 'We utilize a subsets of ToolACE' is ungrammatical, and Table 3 does not state which benchmark the 'Avg' column refers to.
  3. [Eq. (1)] The notation `𝒵 = {𝑧𝑖}𝐼 𝑖=𝑖` contains a typo; it should be `𝒵 = {𝑧𝑖}_{𝑖=1}^{𝐼}`.
  4. [§5.4] The phrase 'an vertain response length' should be 'a certain response length', and Section 1 contains 'simple-tructured' which should be 'simple-structured'.
  5. [Figures 5 and 6] Figures 5 and 6 would benefit from error bars or an explicit statement that each curve is a single run; without this, the reader cannot assess the stability of the scaling and backbone comparisons.
  6. [Table 4] The header layout of Table 4 is difficult to parse; please clearly separate the fine-grained and binary reward columns with subheadings.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central RL-vs-SFT and benchmark claims are empirical comparisons, not reductions to the training reward.

full rationale

The paper's central claims—that Tool-N1-7B/14B outperform GPT-4o on BFCL/API-Bank/ACEBench and that pure RL does not underperform SFT-then-RL—are empirical benchmark results, not derivations from the reward definition. Equation (5) defines a binary reward using ground-truth tool calls from xLAM/ToolACE; this is a training objective, and the conclusions concern held-out benchmark performance, not predictions reconstructed from those labels. The 5,518-trajectory study in Section 5.3.2 compares SFT, RL, and SFT+RL under equal data budgets on the same instances; that comparison is an experimental outcome, and the RL reward does not encode the SFT result. The benchmarks BFCL, API-Bank, and ACEBench are external to the training data, and the paper does not use benchmark test labels in training. Self-citations in Related Work are contextual and do not carry the main argument; there is no imported uniqueness theorem or ansatz justified solely by the authors' prior papers. The absence of error bars and the small margins in Table 3 are legitimate correctness and robustness concerns, but they are not circularity: a small empirical gap is not an input-output equivalence. No circular reduction was found.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No new physical entities or formal objects are introduced. The load-bearing ingredients are the training labels, data filtering, reward design, and benchmark assumptions, none of which are independently verified beyond the paper's own experiments.

free parameters (5)
  • learning_rate = 1e-6
    Selected by grid search; affects training stability and final performance but is not the object of study.
  • KL divergence coefficient = 1e-3
    Set for all runs; controls deviation from the reference policy and could change the SFT vs RL comparison.
  • entropy coefficient = 0
    Chosen because the authors observe that entropy hurts exploration; this choice directly shapes the RL training dynamics.
  • rollout number = 5
    Number of candidate responses per prompt in GRPO; affects advantage estimation and was tuned by grid search.
  • temperature = 0.7
    Rollout sampling temperature; fixed configuration, not analyzed.
assumptions (3)
  • domain assumption Ground-truth tool calls in the training data are correct and uniquely appropriate; reward Eq. (5) awards 1 only for exact dictionary match against a single ground-truth call.
    If multiple tool calls are functionally correct, exact match against one annotation penalizes valid alternatives. This is load-bearing for the RL signal and for the claim that binary reward teaches functional correctness.
  • domain assumption The filtered xLAM and ToolACE subsets are representative of general tool calling after discarding invalid JSON and out-of-list tools.
    The method is evaluated only on this data composition; filtering choices in Section 4.1 and Table 1 are not independently validated.
  • domain assumption BFCL v3 Non-Live and Live categories, API-Bank Call mode, and ACEBench Atom/Single-turn sub-evaluations measure general tool-calling ability, not just in-distribution behavior.
    The paper draws general conclusions from these benchmarks while excluding multi-turn and retrieval modes, so the evidence is narrower than the claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning." pith.science (2026). https://pith.science/paper/L6GEWZBZ

@misc{pith2026250500024,
  author       = {Pith},
  title        = {Pith review of: Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6GEWZBZ}},
  note         = {Machine review of arXiv:2505.00024}
}
read the original abstract

Enabling large language models with external tools has become a pivotal strategy for extending their functionality beyond text space. To enhance LLMs' tool-calling abilities, previous approaches primarily rely on supervised fine-tuning (SFT) with trajectories distilled from stronger models, often resulting in imitative reasoning that limits generalization. In this work, we explore rule-based reinforcement learning to enhance tool-calling in LLMs, resulting in Nemotron-Research-Tool-N1, a series of tool-calling reasoning models. Rather than enforcing supervision over intermediate distilled reasoning traces, Tool-N1 is trained with a binary RL reward that assesses only the format validity and functional correctness of tool invocations. This lightweight supervision allows the model to develop reasoning strategies independently, without relying on annotated trajectories. Experiments on several major benchmarks show that Tool-N1-7B/14B clearly outperform GPT-4o. We conduct a systematic study on the design of rule-based reinforcement learning strategies for training tool-calling models. Using 5,518 distilled reasoning trajectories, we compare SFT, RL, and the SFT-then-RL pipeline, finding that the widely adopted SFT-then-RL paradigm does not necessarily outperform pure RL.

Figures

Figures reproduced from arXiv: 2505.00024 by the authors.

Figure 1
Figure 1. Overview of the training pipeline for Nemotron-Research-Tool-N1 (Tool-N1). Starting from standard SFT tool-calling data comprising user queries and candidate tools, we train LLMs to produce structured reasoning and tool calls using a binary reward function within the GRPO algorithm. As supervision is only applied to the format and tool-call correctness, the training process does not require curated reasoning traject… view at source ↗
Figure 2
Figure 2. The resoning prompt template used during training and inference. The prompt guides the LLM to [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison on two additional benchmarks, ACEBench and API-Bank. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Scaling performance across model sizes us [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Learning curves across training steps for Tool-N1-7B and Tool-N1-14B. We report KL divergence, [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: More Learning curves across training steps for Tool-N1-7B and Tool-N1-14B. We report reward, [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Full system prompts used during model training and inference. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 14 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents

    cs.AI 2025-05 conditional novelty 7.0 of 10

    A new CAPTCHA benchmark with a reasoning-depth metric shows multimodal LLM agents solve at most 40% of interactive puzzles, far short of the 93% human success rate.

  2. Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems

    cs.MA 2025-04 conditional novelty 7.0 of 10

    The Who&When benchmark and three baseline methods show that LLMs can attribute multi-agent failures to the right agent at chance-plus levels but almost never to the exact failing step.

  3. Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    CREST, a hierarchical credit assignment method, improves multi-turn tool-use agent training by combining turn-level advantages with entropy-gated self-teacher token modulation, outperforming RL and distillation baselines.

  4. TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning

    cs.CL 2026-08 conditional novelty 6.0 of 10

    TurnSight uses a frozen reference model plus future tool outputs to score each reasoning turn, then uses those scores to fine-tune an RL agent's credit assignment, beating existing methods on three benchmarks.

  5. TCPO: Turn-Level Credit Policy Optimization

    cs.AI 2026-08 conditional novelty 6.0 of 10

    TCPO improves multi-turn LLM RL by converting verifier scores into turn-level credit via three reference-based comparisons, beating baseline methods on math, code, and agent tasks.

  6. Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    CurioSFT preserves exploration during supervised fine-tuning by distilling toward the model's own temperature-scaled distribution and adaptively increasing entropy at high-entropy tokens, improving SFT accuracy by ~2....

  7. LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model

    cs.CV 2025-08 conditional novelty 6.0 of 10

    RL training on preference-labeled critic data transforms a 7B vision-language model into both a stronger critic and a stronger generative policy, improving average benchmark accuracy by 5.7% and enabling self-critique...

  8. MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use

    cs.AI 2025-08 conditional novelty 6.0 of 10

    MUA-RL adds an LLM-simulated user to the RL rollout loop for multi-turn tool use, improving small Qwen3 models on TAU2, BFCL-V3 Multi Turn, and ACEBench Agent.

  9. StepFun-Prover Preview: Let's Think and Verify Step by Step

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A reinforcement learning pipeline with Lean verifier feedback trains a 32B model that scores 70.0% pass@1 on miniF2F-test, a new state of the art.

  10. Visual Agentic Reinforcement Fine-Tuning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Reinforcement fine-tuning with verifiable rewards enables open-source vision-language models to use web search and image-processing code, improving visual QA and multi-hop reasoning.

  11. The Hallucination Tax of Reinforcement Finetuning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Standard RFT sharply reduces LLM refusal on unanswerable questions, and adding 10% synthetic unanswerable math during RFT restores refusal with small accuracy losses.

  12. Reinforced Visual Perception with Tools

    cs.CV 2025-09 conditional novelty 5.0 of 10

    ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.

  13. How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on $\tau$-bench

    cs.CL 2025-08 conditional novelty 5.0 of 10

    IRMA reformulates tool-agent inputs with memory, domain constraints, and tool suggestions, and reports improved tau-bench pass^5 reliability over ReAct, function calling, and self-reflection.

  14. WebDancer: Towards Autonomous Information Seeking Agency

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A four-stage data-and-training pipeline (synthetic QA, trajectory rejection sampling, SFT, DAPO reinforcement learning) produces WebDancer, a web agent that outperforms vanilla ReAct and several open-source agentic fr...

Reference graph

Works this paper leans on

53 extracted references · 20 canonical work pages · cited by 14 Pith papers

  1. [1]

    Chemcrow: Augmenting large-language models with chemistry tools

    Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Chemcrow: Augmenting large-language models with chemistry tools. NeurIPS 2023 AI for Science Workshop, 2023. 1, 3

    work page 2023

  2. [2]

    Acebench: Who wins the match point in tool learning?arXiv preprint arXiv:2501.12851, 2025

    Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, et al. Acebench: Who wins the match point in tool learning?arXiv preprint arXiv:2501.12851, 2025. 2, 6, 8

    arXiv 2025

  3. [3]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025. 1, 4, 9

    arXiv 2025

  4. [4]

    Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536, 2025

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536, 2025. 3

    arXiv 2025

  5. [5]

    Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars.arXiv preprint arXiv:2503.01307, 2025

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars.arXiv preprint arXiv:2503.01307, 2025. 9

    arXiv 2025

  6. [6]

    On designing effective rl reward at training time for llm reasoning.arXiv preprint arXiv:2410.15115,

    JiaxuanGao, ShushengXu, WenjieYe, WeilinLiu, ChuyiHe, WeiFu, ZhiyuMei, GuangjuWang, andYiWu. On designing effective rl reward at training time for llm reasoning.arXiv preprint arXiv:2410.15115,

    arXiv

  7. [7]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 1, 3, 4, 5, 9, 10

    arXiv 2025

  8. [8]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. Advances in Neural Information Processing Systems, 2024. 3

    work page 2024

Show all 53 references
  1. [9]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516,

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516,

    arXiv

  2. [10]

    Language models can solve computer tasks.Advances in Neural Information Processing Systems, pages 39648–39677, 2023

    Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language models can solve computer tasks.Advances in Neural Information Processing Systems, pages 39648–39677, 2023. 3

    2023

  3. [11]

    Internet-augmented dialogue generation.arXiv preprint arXiv:2107.07566, 2021

    Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-augmented dialogue generation.arXiv preprint arXiv:2107.07566, 2021. 1, 3

    2021 arXiv

  4. [12]

    Internet-augmented language models through few-shot prompting for open-domain question answering.arXiv preprint arXiv:2203.05115, 2022

    Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet-augmented language models through few-shot prompting for open-domain question answering.arXiv preprint arXiv:2203.05115, 2022. 3

    2022 arXiv

  5. [13]

    Api-bank: A comprehensive benchmark for tool-augmented llms

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A comprehensive benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244, 2023. 2, 6, 8

    2023 arXiv

  6. [14]

    Process reward model with q-value rankings.arXiv preprint arXiv:2410.11287,

    Wendi Li and Yixuan Li. Process reward model with q-value rankings.arXiv preprint arXiv:2410.11287,

    arXiv

  7. [15]

    Hammer: Robust function-calling for on-device language models via function masking.International Conference on Learning Representations, 2024

    Qiqiang Lin, Muning Wen, Qiuying Peng, Guanyu Nie, Junwei Liao, Jun Wang, Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, et al. Hammer: Robust function-calling for on-device language models via function masking.International Conference on Learning Representations, 2024. 6

    2024

  8. [16]

    Code-r1: Reproducing r1 for code with reliable rewards.arXiv preprint arXiv:2503.18470, 2025

    Jiawei Liu and Lingming Zhang. Code-r1: Reproducing r1 for code with reliable rewards.arXiv preprint arXiv:2503.18470, 2025. 3, 10

    2025

  9. [17]

    Toolace: Winning the points of llm function calling.International Conference on Learning Representations, 2024

    Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, et al. Toolace: Winning the points of llm function calling.International Conference on Learning Representations, 2024. 1, 3, 4, 6, 8

    2024

  10. [18]

    Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets

    Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh RN, et al. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. Advances in Neural Information Processing Systems, 2024. 6

    2024

  11. [19]

    Ui-r1: Enhancing action prediction of gui agents by reinforcement learning.arXiv preprint arXiv:2503.21620, 2025

    Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hong- sheng Li. Ui-r1: Enhancing action prediction of gui agents by reinforcement learning.arXiv preprint arXiv:2503.21620, 2025. 1

    2025 arXiv

  12. [20]

    Sql-r1: Training natural language to sql reasoning model by reinforcement learning.arXiv preprint arXiv:2504.08600, 2025

    Peixian Ma, Xialie Zhuang, Chengjin Xu, Xuhui Jiang, Ran Chen, and Jian Guo. Sql-r1: Training natural language to sql reasoning model by reinforcement learning.arXiv preprint arXiv:2504.08600, 2025. 1

    2025

  13. [21]

    m & m’s: A benchmark to evaluate tool-use for m ulti-step m ulti-modal tasks

    Zixian Ma, Weikai Huang, Jieyu Zhang, Tanmay Gupta, and Ranjay Krishna. m & m’s: A benchmark to evaluate tool-use for m ulti-step m ulti-modal tasks. InEuropean Conference on Computer Vision, 2024. 3

    2024

  14. [22]

    Taco: Learning multi-modal action models with synthetic chains-of-thought-and-action

    Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, et al. Taco: Learning multi-modal action models with synthetic chains-of-thought-and-action. arXiv preprint arXiv:2412.05479, 2024. 3

    2024

  15. [23]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.arXiv preprint arXiv:2503.07365, 2025. 5

    2025 arXiv

  16. [24]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettle- moyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025. 3

    2025 arXiv

  17. [25]

    Webgpt: Browser-assisted question-answering with human feedback.arXiv preprint arXiv:2112.09332, 2021

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback.arXiv preprint arXiv:2112.09332, 2021. 3

    2021 arXiv

  18. [26]

    Feedback loops with language models drive in-context reward hacking.International Conference on Machine Learning, 2024

    Alexander Pan, Erik Jones, Meena Jagadeesan, and Jacob Steinhardt. Feedback loops with language models drive in-context reward hacking.International Conference on Machine Learning, 2024. 11

    2024

  19. [27]

    Art: Automatic multi-step reasoning and tool-use for large language models

    Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv:2303.09014, 2023. 3

    2023 arXiv

  20. [28]

    Apigen-mt: Agentic pipeline for multi-turn data generation via simulated agent-human interplay.arXiv preprint arXiv:2504.03601, 2025

    Akshara Prabhakar, Zuxin Liu, Weiran Yao, Jianguo Zhang, Ming Zhu, Shiyu Wang, Zhiwei Liu, Tulika Awalgaonkar, Haolin Chen, Thai Hoang, et al. Apigen-mt: Agentic pipeline for multi-turn data generation via simulated agent-human interplay.arXiv preprint arXiv:2504.03601, 2025. 6

    2025 arXiv

  21. [29]

    Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ ...

    2023

  22. [30]

    Tool learning with large language models: A survey.Frontiers of Computer Science, 2025

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Tool learning with large language models: A survey.Frontiers of Computer Science, 2025. 1, 3

    2025

  23. [31]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. 3, 4, 5

    2024 arXiv

  24. [32]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025. 1, 3, 10

    2025 arXiv

  25. [33]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    arXiv

  26. [34]

    Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage.arXiv preprint arXiv:2208.03188, 2022

    Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage.arXiv preprint arXiv:2208.03188, 2022. 3

    2022 arXiv

  27. [35]

    Adaptive in-conversation team building for language model agents.arXiv preprint arXiv:2405.19425,

    Linxin Song, Jiale Liu, Jieyu Zhang, Shaokun Zhang, Ao Luo, Shijian Wang, Qingyun Wu, and Chi Wang. Adaptive in-conversation team building for language model agents.arXiv preprint arXiv:2405.19425,

    arXiv

  28. [36]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023. 9

    2023 arXiv

  29. [37]

    Executable code actions elicit better llm agents

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. InInternational Conference on Machine Learning, 2024. 3

    2024

  30. [38]

    What are tools anyway? a survey from the language model perspective.arXiv preprint arXiv:2403.15452, 2024

    Zhiruo Wang, Zhoujun Cheng, Hao Zhu, Daniel Fried, and Graham Neubig. What are tools anyway? a survey from the language model perspective.arXiv preprint arXiv:2403.15452, 2024. 3

    2024 arXiv

  31. [39]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 2022. 1, 3

    2022

  32. [40]

    Mathchat: Converse to tackle challenging math problems with llm agents

    Yiran Wu, Feiran Jia, Shaokun Zhang, Hangyu Li, Erkang Zhu, Yue Wang, Yin Tat Lee, Richard Peng, Qingyun Wu, and Chi Wang. Mathchat: Converse to tackle challenging math problems with llm agents. arXiv preprint arXiv:2306.01337, 2023. 1, 3

    2023 arXiv

  33. [41]

    Patil, Ion Stoica, and Joseph E

    Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Berkeley function calling leaderboard.https://gorilla.cs.berkeley.edu/blogs/ 8_berkeley_function_calling_leaderboard.html, 2024. 2, 6

    2024

  34. [42]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024. 6

    2024 arXiv

  35. [43]

    Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems, 2023. 1, 3, 5

    2023

  36. [44]

    Re- act: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. Re- act: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations, 2023. 1, 3, 5 14 Nemotron-Research-Tool-N1: Exploring Tool-Using La...

    2023

  37. [45]

    Magnet: Multi-turn tool-use data synthesis and distillation via graph translation

    Fan Yin, Zifeng Wang, I Hsu, Jun Yan, Ke Jiang, Yanfei Chen, Jindong Gu, Long T Le, Kai-Wei Chang, Chen-Yu Lee, et al. Magnet: Multi-turn tool-use data synthesis and distillation via graph translation. arXiv preprint arXiv:2503.07826, 2025. 1, 3, 4

    2025 arXiv

  38. [46]

    Steptool: A step-grained reinforcement learning framework for tool learning in llms

    Yuanqing Yu, Zhefan Wang, Weizhi Ma, Zhicheng Guo, Jingtao Zhan, Shuai Wang, Chuhan Wu, Zhiqiang Guo, and Min Zhang. Steptool: A step-grained reinforcement learning framework for tool learning in llms. arXiv preprint arXiv:2410.07745, 2024. 3

    arXiv

  39. [47]

    Boosting tool use of large language models via iterative reinforced fine-tuning.arXiv preprint arXiv:2501.09766, 2025

    Yirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu, Wu Ning, Yutai Hou, Xu Huang, Bing Qin, and Ting Liu. Boosting tool use of large language models via iterative reinforced fine-tuning.arXiv preprint arXiv:2501.09766, 2025. 3

    2025

  40. [48]

    Data-centric artificial intelligence: A survey.ACM Computing Surveys, 2025

    Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. Data-centric artificial intelligence: A survey.ACM Computing Surveys, 2025. 3

    2025

  41. [49]

    xlam: A family of large action models to empower ai agent systems

    Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, et al. xlam: A family of large action models to empower ai agent systems. Proceedings of the 2025 Conference of the Nations of the Americas Chapter...

    2025

  42. [50]

    Ecoact: Economic agent determines when to register what action.arXiv preprint arXiv:2411.01643, 2024

    Shaokun Zhang, Jieyu Zhang, Dujian Ding, Mirian Hipolito Garcia, Ankur Mallick, Daniel Madrigal, Menglin Xia, Victor Rühle, Qingyun Wu, and Chi Wang. Ecoact: Economic agent determines when to register what action.arXiv preprint arXiv:2411.01643, 2024. 3, 5

    2024 arXiv

  43. [51]

    Training languagemodelagentswithoutmodifyinglanguagemodels

    ShaokunZhang, JieyuZhang, JialeLiu, LinxinSong, ChiWang, RanjayKrishna, andQingyunWu. Training languagemodelagentswithoutmodifyinglanguagemodels. InternationalConferenceonMachineLearning , pages arXiv–2402, 2024. 3

    2024

  44. [52]

    Ser- vices_1_FindProvider

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System D...

    2024 arXiv

  45. [2024]

    3 12 Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.