Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MemTool gives LLM agents explicit remove and search operations so they can keep their active tool set small across long multi-turn sessions, and separates removal from search to make that behavior reliable for models without strong…

desk verdict The Remove_Tools primitive is a real, under-explored idea and the 13-model/100-turn sweep is worth having, but the headline removal-efficiency claim is not yet robust because the metric can be satisfied by blanket clearing and the workload distribution is unreported. read the letter →

arxiv 2507.21428 v1 pith:KCY7U7JS submitted 2025-07-29 cs.CL

Elias Lumer , Anmol Gulati , Vamse Kumar Subbiah , Pradeep Honaganahalli Basavaraju , James A. Burke This is my paper
classification cs.CL
keywords LLMagentstoolcallingshort-termmemorymulti-turnconversationdynamicretrievalcontextengineeringModelProtocolfunction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MemTool is a short-term memory framework for LLM agents that lets the active set of tools or MCP servers change from turn to turn within a single conversation. The central claim is that explicit removal and search operations are enough for strong reasoning models to keep their tool context small—90-94% removal on a three-turn average—and that deterministic or hybrid control makes the same behavior reliable for weaker models. The paper evaluates three modes on 13 models over 100 sequential queries from the ScaleMCP benchmark, measuring both removal efficiency and task accuracy. If the results hold, dynamic tool-using agents could run long production sessions without resetting context or tripping API tool limits.

What carries the argument

The mechanism that carries the argument is a pair of management operations that MemTool exposes to the LLM: Remove_Tools, which removes named tools from the active API context, and Search_Tools, which queries a vector knowledge base of 5,000 MCP servers and adds the top-k matches. The framework persists the active tool set across turns and appends the results of remove and search calls to the conversation history, so the agent's next decision sees both the user's new query and the tool list. The three modes differ only in who controls these operations: the agent in Autonomous Mode, a fixed prune-then-search pipeline in Workflow Mode, and a deterministic prune with agent-controlled search in Hybrid Mode. A hard tool-count limit of 128, matching the strictest API constraint, generates an error that forces removal when the set would overflow.

What would settle it

Rerun the 100-turn evaluation with controlled query orderings—all-related versus all-unrelated consecutive queries—and check whether Autonomous Mode's three-turn rolling removal ratio stays near 90% on the all-related stream; if it collapses there, the claimed memory management is an artifact of topic turnover rather than genuine pruning.

Watch

Extended reading notes

Core claim

The paper's central claim is that short-term tool memory for an LLM agent is a manageable resource, not a fixed limitation: the agent should be able to drop tools that are no longer relevant and add new ones when a user query changes topic. MemTool operationalizes this with two primitive operations—Remove_Tools and Search_Tools—and studies three modes: Autonomous Agent Mode, where the LLM chooses both operations while answering; Workflow Mode, where two fresh LLM calls deterministically prune and then search before the agent answers; and Hybrid Mode, which prunes deterministically but lets the agent search and add tools. Over 100 sequential ScaleMCP queries, reasoning-oriented models in Autonomous Mode removed 90-94% of tools within a three-turn window, while medium models removed far less; Workflow and Hybrid modes held removal near or above 90% for nearly all models while retaining competitive task completion. The paper interprets this as evidence that tool memory can be optimized at inference time without fine-tuning, and that the right amount of agent autonomy depends on model capability.

Load-bearing premise

The load-bearing premise is that the 100 sampled tool-use queries are representative of real multi-turn workloads and that the unreported mix of related versus unrelated consecutive queries does not drive the removal results, since if most successive queries are unrelated, a 90% removal ratio is easy to achieve without any real memory management.

Editorial extensions

If this is right

  • Long-running sessions can keep hundreds of MCP servers available for search without letting the active tool list grow to API limits, so agents do not need to be reset or reinitialized between user questions.
  • Workflow Mode provides a model-agnostic control path: even small models keep removal ratios above roughly 90%, so deployment teams can select models on cost and task accuracy rather than on their ability to self-prune.
  • Hybrid Mode is the strongest all-rounder in the reported tables, combining reliable deterministic pruning with agent-driven search that preserves task completion.
  • Full autonomy over tool memory should be reserved for reasoning-oriented models; weaker models given the same freedom tend to accumulate tools until they hit the limit.
  • Tool-context management is a distinct axis from chat summarization and truncation, so context-engineering systems should treat the tool list as a separately managed short-term memory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported removal ratios are workload-dependent: the paper does not break down the 100 queries by whether consecutive turns are related, so the 90-94% numbers should be read as applying to this benchmark's ordering until that distribution is reported.
  • A regret-aware metric would strengthen the evaluation: count how often a removed tool must be re-searched within a few turns, so that over-aggressive pruning is penalized rather than rewarded.
  • The three modes form an autonomy spectrum that could be adapted per turn or per model, promoting a session from Workflow to Hybrid or Autonomous once the model demonstrates reliable pruning on its own.
  • The remove and search primitives are natural hooks for long-term memory: pruned tools could be persisted and re-activated when a later query revisits the same topic, an extension the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces MemTool, a short-term memory framework for LLM agents that dynamically manage tools or MCP servers across multi-turn conversations. Three modes are proposed: Autonomous Agent Mode (full autonomy), Workflow Mode (deterministic prune-then-search), and Hybrid Mode (deterministic pruning with autonomous search). The authors evaluate the three modes on 13 LLMs over 100 sequential queries sampled from the ScaleMCP benchmark, reporting tool-removal ratios, residual tool counts, tool correctness, and task completion. The main findings are that reasoning models achieve high removal ratios in Autonomous Agent Mode (90–94% on a 3-window average), while Workflow and Hybrid modes yield consistently high removal ratios across all models, with modest task-completion trade-offs. The paper concludes with practical recommendations for mode selection based on model capability and task priorities.

Significance. If validated, MemTool addresses a real and underexplored problem: short-term memory management for dynamic tool-use agents in multi-turn sessions. The paper's strength is its clear specification of the three architectures in pseudocode (Algorithms 1–3) and its evaluation across a broad set of commercial and open models over 100 turns, which is more extensive than typical tool-retrieval studies. The work also makes a useful conceptual distinction among autonomy levels. However, the headline quantitative claims are currently not established: the removal metrics do not measure whether removal decisions are correct, the evaluation lacks baselines, and the relatedness structure of the 100-query sequence is uncontrolled. These issues limit the confidence one can place in the paper's central contribution, although the framework itself is plausible and worth further study.

major comments (5)
  1. [§4.1, Eqs. (1)–(2)] The removal metrics as defined reward indiscriminate removal. RemovalRatio sums tools the model itself chose to remove, normalized by tools added, without any check on whether removed tools would be needed in subsequent turns. A model that clears its entire tool list after every query achieves a perfect removal ratio while destroying tool reuse, yet the paper interprets a high ratio as evidence of good short-term memory management (e.g., §4.2.1). This is load-bearing because the paper's central claim is that certain models 'manage' their tool memory well, but the metric cannot distinguish selective pruning from blanket clearing. The authors should report a ground-truth reuse-aware measure, such as removal precision (fraction of removed tools that were not called again within 3 turns), or otherwise show that high removal ratios coincide with maintained task accuracy relative to a no-removal baseline.
  2. [§3.2 and §4.1] The evaluation sequence is described only as 'Each query can be related or unrelated to the immediate prior query,' with no distribution and no per-transition analysis. This is a load-bearing data assumption: if most consecutive queries are unrelated, tools from previous turns are typically dead, so any model that follows the removal instruction achieves a high removal ratio trivially, and the 90–94% claim reduces to instruction-following rather than memory management. If most transitions are related, removal must be selective and the metric becomes meaningful, but the paper never reports the relatedness distribution or a reuse-latency analysis (e.g., the probability that a tool added at turn t is called again in turns t+1 to t+3). The authors should sample the 100 queries with a controlled mix of related and unrelated transitions and report results conditioned on transition type.
  3. [§4.1, Tables 1–2] There are no baselines in the evaluation. The paper does not compare against a no-management setting (where tools simply accumulate), nor against prior dynamic tool-retrieval methods without removal, such as the ToolsHed/ScaleMCP pipelines cited in §2.2. Because the paper claims Workflow Mode extends existing RAG approaches, the reader cannot judge whether MemTool adds value beyond the state of the art. At minimum, the authors should include (i) a fixed-window/no-removal baseline and (ii) a retrieval-only baseline that does not persist tools across turns, and report task completion and tool counts for these conditions. Without such comparisons, the trade-off statements in the conclusion are unsupported.
  4. [§4.1 and Appendix A, Table 2] The reported results are internally inconsistent in several places. Table 1 lists 11 models per mode, whereas Appendix Table 2 lists 13 models (GPT-4o Mini and Claude 3.7 Sonnet are absent from Table 1). Section 4.2.1 states 'roughly 95%' and '87.8–100%' removal, which does not match Table 1's AvgRemovalRatio3T range of 0.878–0.941. Section 4.4.1 says OpenAI o3 achieved 'a perfect 100% RemovalRatio' in Hybrid Mode, but RemovalRatio is not reported in Table 2, so this number cannot be verified. These discrepancies undermine the reproducibility of the paper's central quantitative claims and must be reconciled, with a single table containing all models and all metrics.
  5. [§4.2.2 and §4.1 (Task Completion measurement)] The evaluation appears to use a single run per model per mode, with no repeated seeds, error bars, or confidence intervals, and the Task Completion metric (Eq. 4) is computed by an LLM judge (GPT-4o mini) with no reported validation against human judgments. Given the known variability of LLM outputs and of LLM-as-a-judge, reported differences such as 0.905 vs. 0.878 in Table 1 may be within noise. The authors should provide multiple runs or bootstrap intervals and report judge agreement on a held-out sample, otherwise the ranking of models within each mode cannot be considered reliable.
minor comments (6)
  1. [§4.1, Eq. (3)] The notation in Eq. (3) is undefined: K and the subscript p are not explained, and the triple sum is unclear. Please rewrite the definition of AvgResidual+3 with explicit turn indices and a clear description of the averaging window.
  2. [Algorithm 1, line 18] The pseudocode's 'Raise Error' when the tool limit is exceeded is ambiguous: after the error is raised, the loop continues with an over-limit tool set, but it is not specified how the agent recovers or whether the error message is appended to the conversation. Clarify the intended control flow.
  3. [Figure 1] Figure 1 is very dense and the step numbering differs across the three panels, making it hard to follow the workflow. Label the steps consistently and, if possible, separate the three modes into sub-figures.
  4. [§4.2.1] Typo: 'Clause Opus 4' should be 'Claude Opus 4'.
  5. [References] References [Lumer et al. 2025c] and [Lumer et al. 2025d] are the same ToolsHed ICAART paper duplicated with different formatting, and [Lumer et al. 2024] is the arXiv version of the same work. Please consolidate these citations.
  6. [Abstract and §4.2.1] The abstract states '13+ LLMs' but Table 1 shows 11 models per mode; please ensure the abstract matches the actual number of models evaluated in the main table.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline results are direct measurements on an externally defined benchmark, not derivations from fitted parameters or self-citation chains.

full rationale

The paper's central claims are empirical measurements: tool counts, removal ratios, residuals, and task-completion scores collected across 100-turn sessions on the ScaleMCP benchmark. The removal ratio and 3-window average are defined by the paper's own equations (Eqs. 1 and 2) from observed tool additions and removals; they are not fitted parameters renamed as predictions, and no result is shown to be equivalent to its own input by construction. The benchmark is cited to prior work by the same group (Lumer et al., 2025b), but it is a publicly available benchmark with ground-truth tool calls and task answers, so it functions as external, falsifiable evidence rather than a self-citation chain that forces the conclusion. The paper's use of its own prior results to justify embedding choice and the SearchTool design is likewise not circular because those prior results are independent empirical findings about retriever performance, not about the present removal-efficiency claim. The concerns raised by the reader's take—that the relatedness distribution of consecutive queries is unreported and that removal ratio counts the model's own pruning decisions rather than ground-truth tool deadness—are genuine threats to the validity and interpretation of the metric, but they are correctness/robustness concerns, not circularity. The Limitations section candidly acknowledges mode-specific weaknesses, including autonomous-mode removal failures and hybrid-mode tool-limit breaches, which further indicates the results are not constructed to guarantee the conclusion. Under the stated hard rules, the absence of any equation-level reduction or fitted-input prediction means the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The MemTool evaluation rests on the authors' own ScaleMCP benchmark, an LLM judge with no human validation, and a single 100-turn trajectory per condition. These are domain assumptions, not free parameters fitted to data; the only hand-set numerical choices are the retrieval top-k, the 128-tool API limit, and the 3-turn metric window, none of which are fitted to optimize results.

free parameters (4)
  • top_k = 5
    Vector search returns 5 tools per query; chosen by the authors, not fitted to data.
  • tool_limit_L = 128
    API tool limit set to 128 to standardize across models; Gemini and Claude can handle more, so this constrains all models.
  • window_size_3T = 3 turns
    Rolling 3-turn window for removal ratio and residual metrics; arbitrary smoothing choice.
  • judge_model = GPT-4o mini
    LLM-as-judge for task completion and tool correctness; no human validation or agreement reported.
assumptions (6)
  • domain assumption ScaleMCP benchmark provides valid expected answers for tool calls and task completion.
    The evaluation relies on the authors' own ScaleMCP benchmark (Lumer et al., 2025b) as ground truth; no external validation is provided.
  • domain assumption LLM-as-judge alignment scoring is a valid measure of task completion.
    Task completion is scored by GPT-4o mini alignment to expected answers (Equation 4); no human agreement study is reported.
  • domain assumption The 100 sampled queries are representative of multi-turn tool-use workloads.
    Stratified sample of 100 instances from ScaleMCP; relatedness between consecutive queries is unreported, which affects removal difficulty.
  • domain assumption The embedding model text-embedding-ada-002 is adequate for tool retrieval.
    Authors rely on ScaleMCP's prior embedding tests to justify this choice; no new retrieval accuracy results are reported here.
  • domain assumption LLM behavior is stable across the single run used for each condition.
    No seeds or repeated runs are reported; all metrics are point estimates from one 100-turn trajectory per model-mode.
  • standard math Basic arithmetic for ratios is correct.
    Equations 1-5 are straightforward ratios and averages.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations." pith.science (2026). https://pith.science/paper/KCY7U7JS

@misc{pith2026250721428,
  author       = {Pith},
  title        = {Pith review of: MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KCY7U7JS}},
  note         = {Machine review of arXiv:2507.21428}
}
read the original abstract

Large Language Model (LLM) agents have shown significant autonomous capabilities in dynamically searching and incorporating relevant tools or Model Context Protocol (MCP) servers for individual queries. However, fixed context windows limit effectiveness in multi-turn interactions requiring repeated, independent tool usage. We introduce MemTool, a short-term memory framework enabling LLM agents to dynamically manage tools or MCP server contexts across multi-turn conversations. MemTool offers three agentic architectures: 1) Autonomous Agent Mode, granting full tool management autonomy, 2) Workflow Mode, providing deterministic control without autonomy, and 3) Hybrid Mode, combining autonomous and deterministic control. Evaluating each MemTool mode across 13+ LLMs on the ScaleMCP benchmark, we conducted experiments over 100 consecutive user interactions, measuring tool removal ratios (short-term memory efficiency) and task completion accuracy. In Autonomous Agent Mode, reasoning LLMs achieve high tool-removal efficiency (90-94% over a 3-window average), while medium-sized models exhibit significantly lower efficiency (0-60%). Workflow and Hybrid modes consistently manage tool removal effectively, whereas Autonomous and Hybrid modes excel at task completion. We present trade-offs and recommendations for each MemTool mode based on task accuracy, agency, and model capabilities.

Figures

Figures reproduced from arXiv: 2507.21428 by the authors.

Figure 1
Figure 1. MemTool three modes architecture and end-to-end walkthrough of user-assistant interaction across [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. MemTool Autonomous Agent Mode: Tool Count across 100 multi-turn queries for various LLMs. While [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. MemTool Workflow Mode: Tool Count across [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: MemTool Hybrid Mode: Tool Count across 100 multi-turn queries for various LLMs. Most models stay stable and far below the 128-tool limit. 4.1 Nano exhibit lower downstream performance, they still maintain high removal ratios, suggesting Hybrid Mode allows efficient mem…
Figure 5
Figure 5. Figure 5: Autonomous Agent System Prompt, with the SearchTool (SYS_SEARCH_TOOL) and RemoveTool [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Search Tools prompt for Workflow Mode LLM call to add more tools by generating search keywords [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Remove Tools prompt for Workflow Mode and Hybrid Mode LLM call to remove irrelevant tools by [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: OpenAI o3 — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: GPT-4o — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: GPT-4o Mini — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: GPT-4.1 — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: GPT-4.1 Mini — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: GPT-4.1 Nano — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Claude Opus 4 — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Claude Sonnet 4 — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Claude 3.7 Sonnet — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]
Figure 17
Figure 17. Figure 17: Claude 3.5 Sonnet — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]
Figure 18
Figure 18. Figure 18: Gemini 2.5 Pro — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]
Figure 19
Figure 19. Figure 19: Gemini 2.5 Flash — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: LLaMA 3 70B — Tool Count over Time (Autonomous Agent Mode) [PITH_FULL_IMAGE:figures/full_fig_p023_20.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LMEB: Long-horizon Memory Embedding Benchmark

    cs.CL 2026-03 unverdicted novelty 7.0 of 10

    LMEB is a new benchmark that evaluates embedding models on long-horizon memory retrieval and shows this skill is largely orthogonal to traditional passage-retrieval performance.

Reference graph

Works this paper leans on

60 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Richmond Alake. 2025. Architecting agent memory: Principles, patterns, and best practices. Presented at AIEWF 2025 (AI Engineer World's Fair). San Francisco, CA. Hosted by MongoDB. Accessed: July 19, 2025

    work page 2025

  2. [2]

    Hill, and Srinivas Chappidi

    Raviteja Anantha, Bortik Bandyopadhyay, Anirudh Kashi, Sayantan Mahinder, Andrew W. Hill, and Srinivas Chappidi. 2023. https://arxiv.org/abs/2312.10332 Protip: Progressive tool retrieval improves planning . Preprint, arXiv:2312.10332

    arXiv 2023

  3. [3]

    Anthropic . 2025. Anthropic. https://www.anthropic.com/

    work page 2025

  4. [4]

    Drew Breunig. 2025. How to fix your context. https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html?ref=blog.langchain.com

    work page 2025

  5. [5]

    Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang, Vincent Cohen-Addad, Mohammadhossein Bateni, Chen-Yu Lee, and Tomas Pfister. 2024. https://arxiv.org/abs/2408.01875 Re-invoke: Tool invocation rewriting for zero-shot tool retrieval . Preprint, arXiv:2408.01875

    arXiv 2024

  6. [6]

    Zihao Cheng, Hongru Wang, Zeming Liu, Yuhang Guo, Yuanfang Guo, Yunhong Wang, and Haifeng Wang. 2025. https://arxiv.org/abs/2505.13176 Toolspectrum : Towards personalized tool utilization for large language models . Preprint, arXiv:2505.13176

    work page Pith review arXiv 2025

  7. [7]

    Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina, and Stéphane Clinchant. 2025. https://arxiv.org/abs/2501.16214 Provence: efficient and robust context pruning for retrieval-augmented generation . Preprint, arXiv:2501.16214

    arXiv 2025

  8. [8]

    Confident AI . 2025. https://www.deepeval.com/ Deepeval: The open-source llm evaluation framework

    work page 2025

Show all 60 references
  1. [9]

    Yu Du, Fangyun Wei, and Hongyang Zhang. 2024. https://arxiv.org/abs/2402.04253 Anytool: Self-reflective, hierarchical agents for large-scale api calls . Preprint, arXiv:2402.04253

    2024 arXiv

  2. [10]

    Xiang Fei, Xiawu Zheng, and Hao Feng. 2025. https://arxiv.org/abs/2506.01056 Mcp-zero: Active tool discovery for autonomous llm agents . Preprint, arXiv:2506.01056

    2025 arXiv

  3. [11]

    Google . 2025 a . Gemini. https://gemini.google.com/

    2025

  4. [12]

    Google . 2025 b . Google model provider long‑term memory. https://ai.google/. Google AI LLMs persist user memory across sessions. Accessed July 19, 2025

    2025

  5. [13]

    Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2024 a . https://arxiv.org/abs/2305.11554 Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings . Preprint, arXiv:2305.11554

    2024 arXiv

  6. [14]

    Yupu Hao, Pengfei Cao, Zhuoran Jin, Huanxuan Liao, Yubo Chen, Kang Liu, and Jun Zhao. 2024 b . https://arxiv.org/abs/2409.13202 Citi: Enhancing tool utilizing ability in large language models without sacrificing general performance . Preprint, arXiv:2409.13202

    2024 arXiv

  7. [15]

    Yupu Hao, Pengfei Cao, Zhuoran Jin, Huanxuan Liao, Yubo Chen, Kang Liu, and Jun Zhao. 2025. https://arxiv.org/abs/2503.00771 Evaluating personalized tool-augmented llms from the perspectives of personalization and proactivity . Preprint, arXiv:2503.00771

    2025 arXiv

  8. [16]

    Kelly Hong, Anton Troynikov, and Jeff Huber. 2025. Context rot: How increasing input tokens impacts llm performance. https://research.trychroma.com/context-rot. Chroma Research

    2025

  9. [17]

    Andrej Karpathy. 2025. Keynote: Software is changing (again). Presented at AI Startup School. San Francisco, CA. Slides available at https://drive.google.com/file/d/1a0h1

    2025

  10. [18]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. https://arxiv.org/abs/2505.06120 Llms get lost in multi-turn conversation . Preprint, arXiv:2505.06120

    2025 arXiv

  11. [19]

    LangChain . 2025. Context engineering. https://blog.langchain.com/context-engineering-for-agents/

    2025

  12. [20]

    Letta . 2025. Letta: Stateful agents with transparent long‑term memory. https://github.com/letta-ai/letta. Open‑source framework based on MemGPT. Accessed July 19, 2025

    2025

  13. [21]

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. https://arxiv.org/abs/2304.08244 Api-bank: A comprehensive benchmark for tool-augmented llms . Preprint, arXiv:2304.08244

    2023 arXiv

  14. [22]

    Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong Wang, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Xinzhi Wang, Yong Liu, Yasheng Wang, and 8 others. 2024. https://arxiv.org/abs/2409.009...

    2024 arXiv

  15. [23]

    Burke, and Vamse Kumar Subbiah

    Elias Lumer, Pradeep Honaganahalli Basavaraju, Myles Mason, James A. Burke, and Vamse Kumar Subbiah. 2025 a . https://arxiv.org/abs/2502.07223 Graph rag-tool fusion . Preprint, arXiv:2502.07223

    2025 arXiv

  16. [24]

    Elias Lumer, Anmol Gulati, Vamse Kumar Subbiah, Pradeep Honaganahalli Basavaraju, and James A. Burke. 2025 b . https://arxiv.org/abs/2505.06416 Scalemcp: Dynamic and auto-synchronizing model context protocol tools for llm agents . Preprint, arXiv:2505.06416

    2025 arXiv

  17. [26]

    Elias Lumer, Vamse Subbiah, James Burke, Pradeep Basavaraju, and Austin Huber. 2025 d . https://doi.org/10.5220/0013303000003890 Toolshed: Scale tool-equipped agents with advanced rag-tool fusion and tool knowledge bases . In Proceedings of the 17th International Conference on...

    2025 doi

  18. [27]

    Burke, Pradeep Honaganahalli Basavaraju, and Austin Huber

    Elias Lumer, Vamse Kumar Subbiah, James A. Burke, Pradeep Honaganahalli Basavaraju, and Austin Huber. 2024. https://arxiv.org/abs/2410.14594 Toolshed: Scale tool-equipped agents with advanced rag-tool fusion and tool knowledge bases . Preprint, arXiv:2410.14594

    2024 arXiv

  19. [28]

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. https://arxiv.org/abs/2402.17753 Evaluating very long-term conversational memory of llm agents . Preprint, arXiv:2402.17753

    2024 arXiv

  20. [29]

    Mem0 . 2025. Mem0: The memory layer for personalized ai. https://mem0.ai/. A universal memory layer for AI agents. Accessed July 19, 2025

    2025

  21. [30]

    Meta Platforms . 2025. Meta llama. https://llama.meta.com/

    2025

  22. [31]

    Model Context Protocol . 2025. Tools documentation. https://modelcontextprotocol.io/docs/concepts/tools

    2025

  23. [32]

    OpenAI. 2024. https://platform.openai.com/docs/guides/function-calling Function calling

    2024

  24. [33]

    OpenAI . 2025 a . Openai. https://openai.com/

    2025

  25. [34]

    OpenAI . 2025 b . Openai model provider long‑term memory. https://openai.com/. OpenAI LLMs persist memory across user sessions. Accessed July 19, 2025

    2025

  26. [35]

    Patil, Ion Stoica, and Joseph E

    Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2024. https://arxiv.org/abs/2310.08560 Memgpt: Towards llms as operating systems . Preprint, arXiv:2310.08560

    2024 arXiv

  27. [36]

    Vicky Zhao, Lili Qiu, and Jianfeng Gao

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Jianfeng Gao. 2025. https://arxiv.org/abs/2502.05589 On memory construction and retrieval for personalized conversational agents . Preprint, ar...

    2025 arXiv

  28. [37]

    O'Brien, Carrie J

    Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. https://arxiv.org/abs/2304.03442 Generative agents: Interactive simulacra of human behavior . Preprint, arXiv:2304.03442

    2023 arXiv

  29. [38]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. https://arxiv.org/abs/2305.15334 Gorilla: Large language model connected with massive apis . Preprint, arXiv:2305.15334

    2023 arXiv

  30. [39]

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. https://arxiv.org/abs/2408.08921 Graph retrieval-augmented generation: A survey . Preprint, arXiv:2408.08921

    2024 arXiv

  31. [40]

    Perplexity . 2025. Perplexity ai: Persistent memory in conversational ai. https://www.perplexity.ai/. Perplexity stores long‑term user interactions. Accessed July 19, 2025

    2025

  32. [41]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. https://arxiv.org/abs/2307.16789 Toolllm: ...

    2023 arXiv

  33. [42]

    Chen, and Shafiq Joty

    Mathieu Ravaut, Aixin Sun, Nancy F. Chen, and Shafiq Joty. 2024. https://arxiv.org/abs/2310.10570 On context utilization in summarization with large language models . Preprint, arXiv:2310.10570

    2024 arXiv

  34. [43]

    Philipp Schmid. 2025. The new skill in ai is not prompting, it's context engineering. https://www.philschmid.de/context-engineering

    2025

  35. [44]

    Lianlei Shan, Shixian Luo, Zezhou Zhu, Yu Yuan, and Yong Wu. 2025. https://arxiv.org/abs/2504.02441 Cognitive memory in large language models . Preprint, arXiv:2504.02441

    2025 arXiv

  36. [45]

    Ning Shang, Li Lyna Zhang, Siyuan Wang, Gaokai Zhang, Gilsinia Lopez, Fan Yang, Weizhu Chen, and Mao Yang. 2025. https://arxiv.org/abs/2502.20082 Longrope2: Near-lossless llm context window scaling . Preprint, arXiv:2502.20082

    2025 arXiv

  37. [46]

    Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. 2025. https://arxiv.org/abs/2501.09136 Agentic retrieval-augmented generation: A survey on agentic rag . Preprint, arXiv:2501.09136

    2025 arXiv

  38. [47]

    Vellum.ai . 2025. Llm leaderboard. https://www.vellum.ai/llm-leaderboard

    2025

  39. [48]

    Qingyue Wang, Yanhe Fu, Yanan Cao, Shuai Wang, Zhiliang Tian, and Liang Ding. 2025. https://arxiv.org/abs/2308.15022 Recursively summarizing enables long-term dialogue memory in large language models . Preprint, arXiv:2308.15022

    2025 arXiv

  40. [49]

    Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025 a . https://arxiv.org/abs/2410.10813 Longmemeval: Benchmarking chat assistants on long-term interactive memory . Preprint, arXiv:2410.10813

    2025 arXiv

  41. [50]

    Mengsong Wu, Tong Zhu, Han Han, Chuanyuan Tan, Xiang Zhang, and Wenliang Chen. 2024. https://arxiv.org/abs/2405.08355 Seal-tools: Self-instruct tool learning dataset for agent tuning and detailed benchmark . Preprint, arXiv:2405.08355

    2024 arXiv

  42. [51]

    Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. 2025 b . https://arxiv.org/abs/2504.15965 From human memory to ai memory: A survey on memory mechanisms in the era of llms . Preprint, arXiv:2504.15965

    2025 arXiv

  43. [52]

    Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. 2025. https://arxiv.org/abs/2502.12110 A-mem: Agentic memory for llm agents . Preprint, arXiv:2502.12110

    2025 arXiv

  44. [53]

    Fung, Hao Peng, and Heng Ji

    Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R. Fung, Hao Peng, and Heng Ji. 2024. https://arxiv.org/abs/2309.17428 Craft: Customizing llms by creating and retrieving from specialized toolsets . Preprint, arXiv:2309.17428

    2024 arXiv

  45. [54]

    Zep . 2025. Zep: A context engineering platform for ai agents. https://www.getzep.com/. A long‑term memory service using temporal knowledge graphs. Accessed July 19, 2025

    2025

  46. [55]

    Yu, and Xian Li

    Weizhi Zhang, Xinyang Zhang, Chenwei Zhang, Liangwei Yang, Jingbo Shang, Zhepei Wei, Henry Peng Zou, Zijie Huang, Zhengyang Wang, Yifan Gao, Xiaoman Pan, Lian Xiong, Jingguo Liu, Philip S. Yu, and Xian Li. 2025. https://arxiv.org/abs/2506.06254 Personaagent: When large languag...

    2025 arXiv

  47. [56]

    Yuanhang Zheng, Peng Li, Wei Liu, Yang Liu, Jian Luan, and Bin Wang. 2024. https://arxiv.org/abs/2403.06551 Toolrerank: Adaptive and hierarchy-aware reranking for tool retrieval

    2024 arXiv

  48. [57]

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2023. https://arxiv.org/abs/2305.10250 Memorybank: Enhancing large language models with long-term memory . Preprint, arXiv:2305.10250

    2023 arXiv

  49. [58]

    Dongsheng Zhu, Weixian Shi, Zhengliang Shi, Zhaochun Ren, Shuaiqiang Wang, Lingyong Yan, and Dawei Yin. 2025. https://arxiv.org/abs/2501.12432 Divide-then-aggregate: An efficient tool learning method via parallel tool invocation . Preprint, arXiv:2501.12432

    2025 arXiv

  50. [59]

    Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, and Chao Zhang. 2025. https://arxiv...

    2025 arXiv

  51. [60]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  52. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.