Pith. sign in

REVIEW 5 major objections 6 minor 8 cited by

MCP-Zero: Active Tool Discovery for Autonomous LLM Agents

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MCP-Zero lets an LLM ask for the tools it needs instead of receiving every tool schema, cutting prompt tokens by roughly 98 percent while keeping selection accuracy high.

desk verdict Useful active-retrieval framework and a new MCP dataset, but the evaluation stops short of task completion; the token-efficiency result holds, the accuracy claim needs an end-to-end check. read the letter →

arxiv 2506.01056 v4 pith:6SNOTFM4 submitted 2025-06-01 cs.AI cs.SE

Xiang Fei , Xiawu Zheng , Hao Feng This is my paper
classification cs.AIcs.SE
keywords MCP-ZeroactivetooldiscoveryLLMagentsModelContextProtocolhierarchicalsemanticroutingretrievaltokenefficiencyMCP-tools
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MCP-Zero argues that an LLM agent should decide for itself which external tools it needs, rather than having every tool schema stuffed into the prompt or having a retriever pre-select tools from the user's query alone. The paper's central claim is that when the model emits a short structured request naming a server domain and an operation, that request is semantically closer to the tool's own documentation than the original user query is, so retrieval can be both tiny and accurate. On a newly built index of 308 MCP servers and 2,797 tools, the method is reported to select the correct tool from nearly three thousand candidates and to cut prompt-token consumption by about 98 percent on a public API benchmark while keeping top-1 selection accuracy near or above baselines. The reason this matters is that tool ecosystems are growing faster than context windows can hold, and the paper proposes that the agent's own request is the right input for scalable tool retrieval.

What carries the argument

The load-bearing object is the `<tool_assistant>` request block, in which the model writes a server field and a tool field at the moment it detects a capability gap. Retrieval then runs in two stages: server-level matching on the server field against server descriptions and LLM-generated capability summaries, followed by tool-level ranking on the tool field against tool descriptions, with a combined score of $\text{score} = (s_{\text{server}} \cdot s_{\text{tool}}) \cdot \max(s_{\text{server}}, s_{\text{tool}})$, where the two similarities are cosine similarities at server and tool levels. The paper's theoretical claim is that the embedding of the agent-generated request is closer to the tool description embedding than the embedding of the original user query is, and that this alignment advantage is what lets top-1 retrieval replace exhaustive schema injection. Iteration is the third piece: if the returned tool is inadequate, the model rewrites the request and retrieves again, so the toolchain is built incrementally per subtask.

What would settle it

Take the needle-in-a-haystack setup, rewrite tool descriptions to remove operation verbs so they no longer echo the wording an active request would use, and re-run the same requests: if top-1 accuracy collapses to the query-retrieval baseline, the claimed active-request advantage is mostly lexical overlap with the original descriptions. Alternatively, measure end-to-end task completion, not just top-1 retrieval, on a multi-turn benchmark; if high retrieval accuracy does not raise task success above the full-context baseline, the paper's claim that accuracy is maintained in real agent performance is not established.

Watch

Extended reading notes

Core claim

The central discovery claimed is that active tool requests by the LLM itself, in the form of a two-line block specifying a server domain and a tool operation, solve the scale problem of tool-augmented agents without giving up decision authority. In the paper's formulation, traditional passive selection must evaluate all $n$ tools with $O(n)$ cost and spreads attention thinly, while active discovery filters $m$ servers and then ranks tools inside them, reducing complexity to $O(m+k)$. The claimed empirical result is that this active-request retrieval achieves accurate selection from 2,797 tools across 248.1k tokens of documentation and reduces APIBank prompt tokens from about 6.3k to 111 in the full single-turn setting, with top-1 accuracy of about 95 to 97 percent across three LLMs. The authors take this as evidence that paying for tools only when they are needed preserves or improves selection accuracy while making multi-turn, cross-domain tool chains feasible.

Load-bearing premise

The load-bearing premise is that a one-line request written by the LLM is semantically closer to the tool's documentation than the user's original wording; if that alignment advantage shrinks, say on terse or poorly documented tools, retrieval accuracy falls and the token savings lose their accuracy guarantee.

Editorial extensions

If this is right

  • System prompts shrink to a short instruction instead of a dump of hundreds of JSON schemas, freeing context budget for reasoning and conversation.
  • Tool ecosystems can scale to thousands of servers without exponential growth in per-request tokens, since each request costs only the retrieved top-k schemas.
  • Complex tasks spanning several domains can be handled by chaining tools from different servers one request at a time, rather than by pre-loading a superset of all possible tools.
  • A lightweight index of tool names, descriptions, and LLM-written capability summaries is enough to support retrieval, so new tools can be added to an agent without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the alignment advantage is robust across documentation styles, the same active-request pattern should transfer to any registry with descriptive metadata, such as internal enterprise APIs, command-line tools, or database schemas, not just MCP servers.
  • The paper's own discussion suggests that a grammar-based decoder could replace the single in-context example; a testable extension is whether constraining the request format at decode time eliminates the remaining ICL gain and stabilizes retrieval on weaker models.
  • Because the reported savings are measured in prompt tokens, inference cost should shrink roughly linearly with them; the unstated implication is that active discovery could raise the number of agents served per dollar, though the paper does not report end-to-end latency or cost.
  • The suggested pairing with a tool-building agent that creates missing tools on the fly implies a closed loop where discovery feeds creation and creation feeds discovery; this direction is outlined by the paper but not implemented.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes MCP-Zero, an active tool discovery framework for LLM agents. Instead of injecting all tool schemas into the context or performing one-shot query-to-tool retrieval, the LLM generates structured <tool_assistant> requests specifying a server domain and a tool, and a two-stage hierarchical semantic router matches these requests against a vector index of MCP server and tool descriptions. The framework also supports iterative capability extension, where the agent can issue multiple requests during a task. The authors construct MCP-tools, a dataset of 308 MCP servers and 2,797 tools, and evaluate the approach on a needle-in-a-haystack task and on APIBank, reporting up to 98% token-consumption reduction while maintaining high top-1 tool-selection accuracy relative to standard schema injection.

Significance. If the results hold, MCP-Zero addresses a genuine scalability problem in tool-augmented agents: the context overhead of injecting thousands of tool schemas. The paper ships code, a dataset, and precomputed embeddings, which supports reproducibility. The token-reduction arithmetic is straightforward, and the reported retrieval accuracy numbers are suggestive of a real efficiency-accuracy trade-off. However, the evaluation measures tool-selection accuracy rather than end-to-end task completion, and several experimental details are missing, so the broader claims about autonomous task execution and task-completion accuracy are not yet substantiated. The work is a useful step toward retrieval-based tool discovery in the MCP ecosystem, but its central claim needs additional evidence or a more careful restatement.

major comments (5)
  1. [§5.3, Table 1; §7.4] The evaluation protocol measures top-1 tool-selection accuracy, not task completion. Table 1 explicitly states that accuracy is 'based on the top-1 result,' and the needle-in-a-haystack test in §5.2 similarly requires models to 'retrieve the target tool.' No experiment executes a task with the retrieved tool, checks parameter correctness, or measures downstream success. This conflicts with the abstract and §3.1, which describe an 'autonomous agent framework' and claim 'high task completion accuracy.' Because the 98% token reduction comes from discarding all but one schema, a retrieval error is unrecoverable in the reported APIBank numbers unless the iterative loop is actually executed, but no evidence shows iterative refinement with tool responses. Section 7.4 concedes that 'Enhanced Experimental Validation' is future work. The claims should be restated as tool-selection accuracy, or an end-to-end task-completion experiment should be added.
  2. [§5.3, Table 1] The query-retrieval baselines ('Domain Q.Retrieval' and 'Full Q.Retrieval') are not described in the experimental setup. The text does not state what embedding model is used, whether the same hierarchical router with server summaries is applied, or whether the same top-k policy is used. Without this information, the 'Necessity of Active Requests' conclusion is not a controlled comparison, and the accuracy gap could stem from retrieval implementation details rather than from the active <tool_assistant> request. In addition, the 'Domain' subset curation is unstated: how many tools it contains, which domains it covers, and how it was selected. These details are necessary to interpret both the scalability comparison and the query-vs-active-request contrast.
  3. [§5.2, Figures 5 and 6] The needle-in-a-haystack evaluation is under-specified. The text says 'injecting 1 to 2,797 tools into the environment, selecting task descriptions from various positions as queries,' but it does not report the number of trials per collection size, how tools and target positions were sampled, or any variance measure. The figures lack clear axis labels: Figure 5's caption lists 'Succeed/Failed' and 'Target Tool Position' without explaining the axes or the legend, and Figure 6 uses a logarithmic y-axis labeled 'Average Token Cost' but does not define whether token cost is per successful retrieval, per turn, or cumulative. These omissions impede reproducibility of the main scaling result.
  4. [§3.2, Eq. (5)] The 'Semantic Alignment Advantage' theorem, asserted as Alignment(r, t) = cos(e_r, e_t) > cos(e_q, e_t), is presented as an inequality without proof or direct empirical support. No experiment measures cosine similarities between requests, queries, and tool descriptions. The only indirect evidence is the APIBank comparison, which is confounded by the retrieval algorithm differences described above. Since Eq. (5) is the framework's core justification for generating active requests rather than using the raw user query, it needs a direct test, such as reporting average cosine similarities or an ablation that uses user queries with the identical hierarchical routing and top-k policy.
  5. [§2.2, §5.3] The paper positions MCP-Zero against existing retrieval-based tool selection systems, but no experiment compares against any of the cited systems (RAG-MCP, AnyTool, Re-Invoke, ToolRerank, or others). Table 1 only compares with standard schema injection and a home-made query-retrieval baseline. Without at least one representative prior retrieval system as a baseline, the claim that MCP-Zero improves on the state of the art in retrieval accuracy or token efficiency is not supported.
minor comments (6)
  1. [Throughout] There are several typos: 'conlude' in §5.3, 'Q.Retriaval' in Table 1, 'is a active' in §3.1, 'authorsemantically' in §5.3, and 'maximisestool' in §7.3.
  2. [Figures 5 and 6] The figure captions are insufficient; please add full axis labels, legend descriptions, and units. In particular, Figure 5's '0% 50% 100%' is ambiguous, and Figure 6 should specify whether the token cost is per successful trial, per turn, or cumulative over the conversation.
  3. [§5.2] The terminology is inconsistent: the text uses 'MCP-Zero + ICL' while Figure 5 labels a column 'MCP-Zero(w/ Context Example)'; please unify the naming.
  4. [References] Reference [2] (Model Context Protocol) has an access date of 'September 13, 2025,' which postdates the manuscript version (24 Jun 2025); please correct the access date.
  5. [§4.1] The filtering step that reduced 396 servers to 308 is described only qualitatively ('MCP-compliant tool definitions, comprehensive documentation, sufficient detail'). Please provide concrete filter criteria or a data-cleaning script with counts per filter so the dataset construction is reproducible.
  6. [§5.1] The labels 'APIBench1' and 'APIBench2' are confusing; citation [18] is Gorilla rather than an 'APIBench' dataset, and [19] is not clearly identified as 'APIBench2.' Please clarify which references correspond to these names.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: the empirical evaluation is self-contained; one theoretical passage (Eq. 5) restates the design assumption rather than deriving it.

  1. other [Section 3.2, Eq. (5) and following paragraph]
    "Agent-generated requests r exhibit stronger alignment with tool documentation compared to raw user queries q: Alignment(r,t)=cos(e_r,e_t)>cos(e_q,e_t) (5) ... This improvement stems from agents operating in the same semantic space as tool documentation. ... These theoretical foundations explain the empirical performance gains observed in our experiments: 98% token reduction with maintained accuracy reflects the efficiency of targeted information acquisition over exhaustive tool enumeration."

    The inequality in Eq. (5) is asserted rather than derived from the preceding equations; it is exactly the design hypothesis that <tool_assistant> server/tool requests live in tool-description space. The paper then invokes this asserted inequality as a 'theoretical foundation' explaining the empirical 98% token reduction, so the explanation reduces to restating the method's intended semantic-alignment mechanism. This is a mild self-definitional flavor, but it is not load-bearing: the APIBank and MCP-tools evaluations are external and do not depend on Eq. (5).

full rationale

The core claims are not circular. APIBank is an external benchmark, and MCP-tools is a newly constructed dataset; no parameters are fitted to test results, and no self-citation chain supplies the core argument. The hierarchical semantic routing (Section 3.1) is a concrete two-stage retrieval algorithm with a stated scoring rule, and the token-consumption comparisons in Table 1 and Figures 5-6 are empirical measurements, not identities. The only questionable passage is Section 3.2, where Eq. (5) states the semantic-alignment advantage as an assumed inequality and later calls the section's assertions 'theoretical foundations' for the empirical gains. That is a post-hoc restatement of the architecture rather than an independent derivation, giving a mild tautological flavor, but it does not make any experimental prediction equivalent to an input by construction. I therefore assign a low score of 1.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No fitted numeric parameters are central to the claims; the hand-designed scoring heuristic and top-k choice carry tunable degrees of freedom. The main unproven premises are the semantic-alignment advantage and the validity of retrieval accuracy as a proxy for task success.

free parameters (2)
  • Retrieval score function weights = Implicit unit weights in Eq. (1)
    Eq. (1) uses score = s_server * s_tool * max(s_server, s_tool), a hand-designed combination of cosine similarities with no reported ablation or fitting.
  • Retrieval top-k = k=1 for reported results; k configurable
    The paper reports top-1 retrieval; larger k improves fault tolerance but increases context, so it is a tunable performance-context trade-off.
assumptions (4)
  • domain assumption MCP servers and tools consistently provide descriptive documentation that can be embedded.
    Section 3.1 states the design relies on MCP-mandated descriptions; retrieval quality depends on this.
  • domain assumption LLM-generated tool requests are semantically closer to tool documentation than raw user queries (Eq. 5).
    Section 3.2 asserts cos(e_r, e_t) > cos(e_q, e_t); this is not proven and is the basis of the active-request advantage.
  • domain assumption Top-1 retrieval accuracy on APIBank and needle-in-a-haystack measures task-relevant success.
    Experiments report retrieval accuracy, not end-to-end task completion, yet the paper concludes task-accuracy preservation.
  • domain assumption The 308 filtered MCP servers are representative of the MCP ecosystem for evaluation.
    Section 4.1 filters 396 to 308 servers based on three qualitative criteria; selection could bias evaluation difficulty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MCP-Zero: Active Tool Discovery for Autonomous LLM Agents." pith.science (2026). https://pith.science/paper/6SNOTFM4

@misc{pith2026250601056,
  author       = {Pith},
  title        = {Pith review of: MCP-Zero: Active Tool Discovery for Autonomous LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6SNOTFM4}},
  note         = {Machine review of arXiv:2506.01056}
}
read the original abstract

True intelligence requires active capability acquisition, yet current LLM agents inject pre-defined tool schemas into prompts, reducing models to passive selectors and falling short of robust general-purpose agency. We introduce MCP-Zero, an active agent framework that restores tool discovery autonomy to LLMs themselves. Instead of overwhelming models with all available tools, MCP-Zero enables agents to actively identify capability gaps, and request specific tools on-demand, transforming them from large-scale retrievers into genuine autonomous agents. The framework operates through three core mechanisms: (1) Active Tool Request, where models autonomously generate structured requests specifying their exact tool requirements; (2) Hierarchical Semantic Routing, a two-stage algorithm that matches requests to relevant servers and tools through improved semantic alignment; (3) Iterative Capability Extension, enabling agents to progressively build cross-domain toolchains while maintaining minimal context footprint. We construct MCP-tools, a comprehensive dataset of 308 MCP servers and 2,797 tools from the official Model-Context-Protocol repository. Experiments demonstrate that MCP-Zero preserves agent autonomy while achieving substantial efficiency gains: (i) accurate tool selection from nearly 3k candidates across 248.1k tokens; (ii) 98\% reduction in token consumption on APIBank while maintaining high accuracy; and (iii) consistent multi-turn performance that scales with tool ecosystem growth. This work establishes active tool discovery as a fundamental design pattern for scalable autonomous agent systems.

Figures

Figures reproduced from arXiv: 2506.01056 by the authors.

Figure 1
Figure 1. Comparison of tool selection paradigms for LLM agents. (a) System-prompt-based methods inject all MCP tool schemas into [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example of a single MCP tool definition from the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Illustrative example of MCP-Zero’s iterative active in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The schema of the MCP-tools dataset. • Complexity Reduction: From O(n) to O(m + k) where m + k ≪ n. • Semantic Consistency: Direct embedding alignment in tool description space. • Information Efficiency: Targeted uncertainty reduction rather than exhaustive search. • A…
Figure 5
Figure 5. Figure 5: Needle-in-a-haystack test results demonstrating MCP-Zero’s performance under extreme scale conditions with varying tool [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Token efficiency comparison in needle-in-a-haystack [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Where Facts Go Missing: A Layerwise Taxonomy and Per-Layer Attribution of Information Omission in Air-Gapped LLMAgent Pipelines

    cs.MA 2026-07 conditional novelty 6.0 of 10

    In a controlled 75,476-trial stress test, about 73% of omitted-fact failures in LLM agent pipelines are traced to deterministic middleware (redaction, pagination, truncation) rather than model behavior.

  2. FitText: Evolving Agent Tool Ecologies via Memetic Retrieval

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    FitText embeds memetic evolutionary retrieval inside the agent's reasoning loop to iteratively refine pseudo-tool descriptions, raising retrieval rank from 8.81 to 2.78 on ToolRet and pass rate to 0.73 on StableToolBench.

  3. Give Them an Inch and They Will Take a Mile:Understanding and Measuring Caller Identity Confusion in MCP-Based AI Systems

    cs.CR 2026-03 conditional novelty 6.0 of 10

    Most sampled MCP servers do not bind authorization to the identity of the calling agent, so a single authorization can be silently reused by other callers.

  4. Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions

    cs.SE 2026-02 conditional novelty 6.0 of 10

    Most MCP tool descriptions (97.1%) contain quality smells, and augmenting them improves agent success by a median of 5.85 percentage points at a 67.46% increase in execution steps.

  5. Galaxy: A Cognition-Centered Framework for Proactive, Privacy-Preserving, and Self-Evolving LLM Agents

    cs.AI 2025-08 reject novelty 6.0 of 10

    Galaxy couples a cognitive tree structure with a meta-agent to make LLM assistants proactive, privacy-preserving, and self-evolving.

  6. Adapting Embedding Models for Agent Capability Retrieval

    cs.IR 2026-07 conditional novelty 5.0 of 10

    Fine-tuning three off-the-shelf retrieval models on AgentSelect improved query-to-agent ranking on two unseen marketplace catalogs, MuleRun and ClawHub, across all three model families.

  7. MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations

    cs.CL 2025-07 conditional novelty 5.0 of 10

    MemTool is a short-term memory framework with three modes (autonomous, workflow, hybrid) that lets LLM agents add and remove tools across multi-turn conversations, evaluated over 100 turns on 13+ models.

  8. AugServe: Adaptive Request Scheduling for Augmented Large Language Model Inference Serving

    cs.CL 2025-12 conditional novelty 4.0 of 10

    An adaptive two-stage scheduler plus dynamic token batching improves SLO-satisfying throughput for tool-augmented LLM inference versus vLLM and InferCept in the reported experiments.

Reference graph

Works this paper leans on

32 extracted references · 10 canonical work pages · cited by 8 Pith papers

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Model context protocol.https://docs

    Anthropic. Model context protocol.https://docs. anthropic.com/en/docs/agents- and- tools/ mcp, 2024. Accessed: September 13, 2025. 4

    work page 2024

  3. [3]

    Improving language models by retriev- ing from trillions of tokens

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retriev- ing from trillions of tokens. InInternational conference on machine learning, pages 2206–2240. PMLR, 2022. 3

    work page 2022

  4. [4]

    Langchain.https://github.com/ langchain-ai/langchain, 2022

    Harrison Chase. Langchain.https://github.com/ langchain-ai/langchain, 2022. Python framework for developing applications powered by language models. 3

    work page 2022

  5. [5]

    Re-invoke: Tool invo- cation rewriting for zero-shot tool retrieval.arXiv preprint arXiv:2408.01875, 2024

    Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang, Vincent Cohen-Addad, Mohammadhossein Bateni, Chen-Yu Lee, and Tomas Pfister. Re-invoke: Tool invo- cation rewriting for zero-shot tool retrieval.arXiv preprint arXiv:2408.01875, 2024. 3

    arXiv 2024

  6. [6]

    Anytool: Self- reflective, hierarchical agents for large-scale api calls.arXiv preprint arXiv:2402.04253, 2024

    Yu Du, Fangyun Wei, and Hongyang Zhang. Anytool: Self- reflective, hierarchical agents for large-scale api calls.arXiv preprint arXiv:2402.04253, 2024. 3

    arXiv 2024

  7. [7]

    Rag-mcp: Mitigating prompt bloat in llm tool selection via retrieval-augmented genera- tion.arXiv preprint arXiv:2505.03275, 2025

    Tiantian Gan and Qiyao Sun. Rag-mcp: Mitigating prompt bloat in llm tool selection via retrieval-augmented genera- tion.arXiv preprint arXiv:2505.03275, 2025. 2, 3, 4

    arXiv 2025

  8. [8]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Ab- hinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. 2

    arXiv 2024

Show all 32 references
  1. [9]

    Retrieval augmented language model pre- training

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre- training. InInternational conference on machine learning, pages 3929–3938. PMLR, 2020. 3

    2020

  2. [10]

    Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, et al. Mrkl systems: A mod- ular, neuro-symbolic architecture that combines large lan- guage models, external knowledge sources and discre...

    2022 arXiv

  3. [11]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich 10 K¨uttler, Mike Lewis, Wen-tau Yih, Tim Rockt ¨aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing sy...

    2020

  4. [12]

    Api-bank: A comprehensive benchmark for tool-augmented llms.arXiv preprint arXiv:2304.08244, 2023

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A comprehensive benchmark for tool-augmented llms.arXiv preprint arXiv:2304.08244, 2023. 8

    2023 arXiv

  5. [13]

    Evaluation report on mcp servers.arXiv preprint arXiv:2504.11094, 2025

    Zhiling Luo, Xiaorong Shi, Xuanrui Lin, and Jinyang Gao. Evaluation report on mcp servers.arXiv preprint arXiv:2504.11094, 2025. 3, 6, 7

    2025 arXiv

  6. [14]

    The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey.arXiv preprint arXiv:2404.11584, 2024

    Tula Masterman, Sandi Besen, Mason Sawtell, and Alex Chao. The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey.arXiv preprint arXiv:2404.11584, 2024. 2

    2024 arXiv

  7. [15]

    Effi- cient and scalable estimation of tool representations in vector space.arXiv preprint arXiv:2409.02141, 2024

    Suhong Moon, Siddharth Jha, Lutfi Eren Erdogan, Sehoon Kim, Woosang Lim, Kurt Keutzer, and Amir Gholami. Effi- cient and scalable estimation of tool representations in vector space.arXiv preprint arXiv:2409.02141, 2024. 2, 3, 8

    2024 arXiv

  8. [16]

    Webgpt: Browser-assisted question-answering with human feedback

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. 3

    2021 arXiv

  9. [17]

    Art: Automatic multi-step reasoning and tool-use for large language models.arXiv preprint arXiv:2303.09014,

    Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Han- naneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. Art: Automatic multi-step reasoning and tool-use for large language models.arXiv preprint arXiv:2303.09014,

    arXiv

  10. [18]

    Gorilla: Large language model connected with massive apis.Advances in Neural Information Processing Systems, 37:126544–126565, 2024

    Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large language model connected with massive apis.Advances in Neural Information Processing Systems, 37:126544–126565, 2024. 2, 3, 8

    2024

  11. [19]

    Revisiting, bench- marking and exploring api recommendation: How far are we?IEEE Transactions on Software Engineering, 49(4): 1876–1897, 2022

    Yun Peng, Shuqing Li, Wenwei Gu, Yichen Li, Wenxuan Wang, Cuiyun Gao, and Michael R Lyu. Revisiting, bench- marking and exploring api recommendation: How far are we?IEEE Transactions on Software Engineering, 49(4): 1876–1897, 2022. 8

    2022

  12. [20]

    Toolllm: Facilitating large language models to master 16000+ real-world apis.arXiv preprint arXiv:2307.16789,

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis.arXiv preprint arXiv:2307.16789,

    arXiv

  13. [21]

    Tool learning with foundation models

    Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, et al. Tool learning with foundation models. ACM Computing Surveys, 57(4):1–40, 2024. 2

    2024

  14. [22]

    Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution.arXiv preprint arXiv:2505.20286, 2025

    Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Ji- acheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, et al. Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution.arXiv preprint arXiv:2505.20286, 2025. 10

    2025 arXiv

  15. [23]

    Colt: Towards completeness-oriented tool retrieval for large language models.arXiv e-prints, pages arXiv–2405, 2024

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Colt: Towards completeness-oriented tool retrieval for large language models.arXiv e-prints, pages arXiv–2405, 2024. 3

    2024

  16. [24]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Jun- yang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang...

    2025

  17. [25]

    In- context retrieval-augmented language models.Transactions of the Association for Computational Linguistics, 11:1316– 1331, 2023

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Am- non Shashua, Kevin Leyton-Brown, and Yoav Shoham. In- context retrieval-augmented language models.Transactions of the Association for Computational Linguistics, 11:1316– 1331, 2023. 3

    2023

  18. [26]

    Toolformer: Lan- guage models can teach themselves to use tools.Advances in Neural Information Processing Systems, 36:68539–68551,

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess `ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Lan- guage models can teach themselves to use tools.Advances in Neural Information Processing Systems, 36:68539–68551,

  19. [27]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154–38180,

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154–38180,

  20. [28]

    Toolalpaca: Generalized tool learning for language models with 3000 simulated cases

    Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301, 2023. 8

    2023 arXiv

  21. [29]

    Autogen: Enabling next-gen llm ap- plications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm ap- plications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023. 3

    2023 arXiv

  22. [30]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 2

    2025 arXiv

  23. [31]

    React: Synergizing rea- soning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing rea- soning and acting in language models. InInternational Con- ference on Learning Representations (ICLR), 2023. 3

    2023

  24. [32]

    Toolrerank: Adaptive and hierarchy-aware reranking for tool retrieval.arXiv preprint arXiv:2403.06551, 2024

    Yuanhang Zheng, Peng Li, Wei Liu, Yang Liu, Jian Luan, and Bin Wang. Toolrerank: Adaptive and hierarchy-aware reranking for tool retrieval.arXiv preprint arXiv:2403.06551, 2024. 3 11

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.