Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

A tool-use agent with two linked memories lifts multi-turn task accuracy by up to 50 percent.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 17:57 UTC pith:GTTGL57W

load-bearing objection A well-ablated, plausible memory method for tool-use agents, with an under-specified RL evaluation that needs fixing before the headline numbers can be trusted. the 3 major comments →

arxiv 2512.07287 v3 pith:GTTGL57W submitted 2025-12-08 cs.LG cs.AI

Experience-Evolving Multi-Turn Tool-Use Agent with Hybrid Episodic-Procedural Memory

Sijia Li , Yuchen Huang , Zifan Liu , Zijian Li , Jingjing fu , Lei Song , Jiang Bian , Jun Zhang
show 1 more author
Rui Wang
This is my paper
classification cs.LG cs.AI
keywords multi-turn tool useLLM agentsepisodic memoryprocedural memorytool graphexperience reusememory-guided reinforcement learningstate summarization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a multi-turn tool-use agent can choose its next tool much more reliably by reusing successful past interactions at a finer grain than whole trajectories. It stores two complementary memories in one graph: procedural memory, which records which tool tends to follow which tool and how efficiently, and episodic memory, which attaches compact situation summaries to each tool-to-tool transition. At each step the agent either recalls situation-matched summaries or follows frequent transition patterns, and the same memory guides exploration during reinforcement learning. On four multi-turn benchmarks and several base models, the method reports consistent gains—up to 50%+ at inference and up to 40%+ for RL policies on out-of-distribution tasks—with the largest gains on weaker reasoners. The load-bearing idea is that partially overlapping experiences, not complete episodes, are the right unit for transfer.

Core claim

The central discovery is that decomposing past successful trajectories into state-annotated tool transitions lets an agent reuse experience without suffering from over-specificity. The paper constructs a directed tool graph whose edges carry both a weight, combining transition frequency with an efficiency bonus, and, when a summarization tool was invoked, compact state summaries of the context in which the transition succeeded. During inference the agent decides whether to summarize the current state; if it does, it compares the new summary against stored ones and suggests the top-2 tools by similarity, and if not, it follows the top-2 edge weights. The same mechanism is used inside RL rollo

What carries the argument

The state-annotated tool-transition graph: nodes are tools, and each directed edge from one tool to another stores a normalized weight combining how often the transition appears in successful trajectories and how few steps those trajectories took, plus zero or more episodic state summaries observed at that transition. At decision time the agent locates its last tool in the graph, then either matches its current summarized state against the edge summaries (episodic memory) or ranks outgoing edges by weight (procedural memory), and feeds the top-k tools into the system prompt as suggestions. During RL, the same graph biases rollouts toward historically successful transitions, with a skip proba

Load-bearing premise

The episodic-memory path depends on the summarization tool producing state summaries whose embedding similarity tracks decision-relevant similarity—two similar summaries should call for the same next tool—and the paper does not provide a direct validation of that correspondence.

What would settle it

Take decision points from test trajectories, hide the true next tool, and measure whether the top-2 tools chosen by summary similarity match the next tool in matched successful trajectories. If agreement is near chance once the most common transition is controlled for, or if paraphrasing a summary while preserving its meaning changes the suggested tools, the episodic memory is not carrying the claimed signal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Weaker or smaller reasoners gain the most from the method, suggesting memory can substitute for some reasoning capacity at inference time.
  • Using both state summaries and transition weights beats either alone; ablations show removing either component costs several points across benchmarks.
  • Efficiency-aware edge weights matter: accuracy-only weights underperform the full weighting on all three main benchmarks.
  • Adaptive switching between memory types beats always using episodic retrieval or a learned graph-planning traversal, and top-2 suggestions beat top-1 or top-3.
  • Integrating the same memory into RL rollouts improves the learned policy on out-of-distribution tasks, with gains up to 40%+ over standard policy optimization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the summaries are faithful, the approach naturally extends to other sequential decision domains—code editing, database troubleshooting, web navigation—where the 'tools' are actions and the summaries are concise task states; the memory could then transfer across tasks with different surface wording.
  • The 'should I summarize now?' decision is effectively a meta-controller; a cheaper alternative the paper does not explore would be a small classifier trained on when summarization improved the next-tool match, replacing the LLM's self-judgment.
  • The RL gains suggest the memory graph acts like shaped reward without explicit reward shaping; this could be tested directly by comparing memory-guided rollouts against a reward-shaped policy optimization baseline of similar complexity.
  • A testable extension for the episodic path: use a domain-specific state encoder instead of a generic sentence embedding; if gains grow, the bottleneck is state representation rather than retrieval.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes H-EPM, a hybrid episodic–procedural memory for multi-turn tool-use agents. H-EPM builds a directed tool-transition graph from successful historical trajectories: nodes are tools, edges encode procedural statistics (transition frequency and efficiency), and edges are annotated with compact state summaries produced by an agent-invoked summarization tool. At inference, the agent decides whether to summarize the current state; if yes, it retrieves the top-k next-tool candidates by cosine similarity between the current summary and stored summaries (episodic memory); if no, it uses normalized edge weights (procedural memory). The same mechanism is integrated into GRPO rollouts, with a skip rate p_skip to modulate procedural guidance and encourage exploration. The method is evaluated on τ-Bench, τ²-Bench, ToolSandbox, and ACEBench across several base models, and for RL with Qwen3-4B. The paper reports inference-time gains up to 50%+ and RL gains up to 40%+.

Significance. If the reported results are reliable, H-EPM is a meaningful contribution to experience reuse in multi-turn tool-use agents. The paper has clear strengths: it evaluates on diverse benchmarks and multiple base models, provides informative ablations (e.g., the 'w/o memory' variant isolates the effect of the summarization tool, and 'w/o state' vs. 'w/o weight' separates the episodic and procedural contributions), includes a case study, and appends a limitation statement. The RL integration is a novel way to use memory to shape exploration. However, the manuscript currently lacks code, reports only point estimates, and leaves the RL evaluation protocol ambiguous, which prevents the headline quantitative claims from being fully verified.

major comments (3)
  1. [§5.1, Appendix E.1, Table 3] The RL evaluation protocol is ambiguous, which undermines the claim in the abstract of up to 40% policy improvement. Section 5.1 says that except for the rollout stage, all settings are identical to standard GRPO, but Appendix E.1 states that additional guidance and summarization content are 'excluded during training' using a masking strategy. Masking only prevents those tokens from contributing to the loss; the model still conditions on them during the forward pass. Table 3 never states whether H-EPM memory is used at evaluation. If the numbers are obtained with memory prompts at inference, the gains conflate policy learning with test-time memory retrieval. If the policy is evaluated without memory, that should be stated explicitly and the train/inference distribution mismatch should be discussed. Please report both 'policy-only' and 'policy+memory' evaluations for each checkpoint.
  2. [Tables 1 and 3; §5.2; Appendix C.2] All performance tables report single point estimates with no standard deviations, no number of evaluation episodes, and no multiple seeds. Several headline differences are small (e.g., Table 3 ToolSandbox: H-EPM(p_skip=0.8) 0.522 vs. GRPO 0.503; τ-Bench: H-EPM(p_skip=1.0) 0.544 vs. H-EPM(p_skip=0.9) 0.558), so the claim of 'consistently' outperforming baselines is not statistically supported. Moreover, the 'up to 40%' figure comes from selecting p_skip=1.0 on τ²-Bench; the text in §5.2 states that p_skip is adjusted per domain, and the coefficient c is tuned on τ-Bench (Table 7). This test-set tuning of hyperparameters makes the quantitative gains optimistic. Please report mean±std over at least three seeds and validate hyperparameter choices on a held-out split.
  3. [§4.3, Appendix E.1, Table 4(A)] The episodic retrieval mechanism assumes that cosine similarity between all-MiniLM-L6-v2 embeddings of state summaries is a reliable proxy for decision-relevant state similarity. The paper provides no validation of this assumption—no correlation with task outcomes, no sensitivity to the embedding model, and no error analysis of retrieved candidates. Since the 'w/o state' ablation in Table 4(A) shows that removing state information degrades performance, the episodic component is essential to the method; without validating the similarity measure, the mechanism's reliability and its generalizability to new tasks are not established. Please add evidence that similar summaries lead to correct next-tool choices, or at least analyze the sensitivity of the results to the choice of embedding model.
minor comments (6)
  1. [§1 title] The section title 'Instruction' should be 'Introduction'.
  2. [Table 4(D)] The sub-labels for K=2 and K=3 are both '(II)'; the K=3 row should be labeled '(III)'.
  3. [Table 4(A)] Typo: 'W/O WIGHT' should be 'W/O WEIGHT'.
  4. [Appendix C.5] Typo in the appendix title: 'Foward Strategy' should be 'Forward Strategy'.
  5. [Appendix E.1] The sentence 'all results for RL except those in Table 6 are obtained with the masking strategy' references Table 6, which is an ACEBench inference table, not an RL table. It likely should refer to Table 10 in Appendix C.5.
  6. [§5.1 and elsewhere] Inconsistent model naming: 'Qwen3-4B-Instruct-2507' and 'Qwen3-4B-Instruction' are used interchangeably; 'τ-bench Retail' in §5.1 likely refers to τ²-Bench. Also 'Multi-turn tool use is challenges' in §2.1 should be 'is challenging'.

Circularity Check

2 steps flagged

No construction-level circularity in the held-out inference results, but the RL 'up to 40%+ policy gain' (Table 3) is unverified because the paper never states whether test-time memory was used—and on τ2-Bench the inference-only H-EPM already exceeds the RL result—while c and p_skip are selected using evaluation-benchmark scores.

specific steps
  1. other [Table 3 (§5.2); §5.1 RL settings; Appendix C.5; Appendix E.1]
    "To ensure stable policy learning, the additional information introduced during rollout is removed during training. Specifically, we consider two strategies: (i) masking the extra information when computing the loss or (ii) directly discarding it prior to policy gradient computation. ... All additional guidance and summarization content are excluded during training using two strategies (Appendix C.1), and all results for RL except those in Table 6 are obtained with the masking strategy."

    Masking excludes guidance tokens only from the loss, not from the conditioning context, so the policy trains and acts with 'Suggested next tools' in context. The paper never states whether Table 3 was run with H-EPM guidance at test time. Quantitatively, on τ2-Bench the inference-only H-EPM (Table 1: 0.232, +46.8%) already exceeds the best RL H-EPM (Table 3: 0.223, +41.1%), so the headline 'up to 40%+ on out-of-distribution tasks' is fully reproduced by the §4.3 inference mechanism alone. If Table 3 was evaluated with memory (the natural reading of 'for RL training, we use an implicit strategy'), the claim that the policy 'internalizes effective decision strategies' reduces to the already-established inference-time memory effect, and the RL attribution has missing support—an omitted protoc

  2. fitted input called prediction [§5.2 skip-rate analysis; Appendix C.2 (Table 7)]
    "The effectiveness of H-EPM further depends on the skip rate pskip, which controls the extent to which procedural tool-transition suggestions are utilized. ... Finally, tasks in τ 2-Bench primarily involve state-dependent error checking. In this case, entirely skipping weight-based tool suggestions (pskip = 1.0) is most effective. ... Table 7 reports the performance of GPT-4.1 on τ-Bench. Parameter c should strike a balance between efficiency and accuracy."

    The two free parameters are chosen using the evaluation benchmarks' own scores: c=1 is selected from Table 7's τ-Bench test accuracies (0.660/0.710/0.695/0.678), and p_skip is set per benchmark as the best cell of the Table 3 grid ('entirely skipping weight-based tool suggestions (pskip = 1.0) is most effective' for τ2-Bench). The headline 'up to 40%+' is exactly that best cell (τ2-Bench, p_skip=1.0, +41.1%). This is mild fitting to the test distribution rather than construction-level circularity—the full grid is reported and the memory itself is fit only on training trajectories—but the strongest claimed improvement is partly selected, not predicted.

full rationale

The core inference-time claim is not circular: the memory graph (§4.2) is constructed from training-set trajectories (τ-Bench/τ2-Bench have separate train/test sets; ToolSandbox is split 800/200), edge weights encode counts and step-efficiency of those training successes, and Table 1 reports held-out test accuracy against ReasoningBank, ToolNet, and Top-3 retrieval. The ablations (Table 4) isolate summarization, weight, and state components, and the full method exceeds the 'w/o memory' variant, so the components carry independent weight. ACEBench's online-memory setting is a deliberate self-evolution protocol (memory built from test-time successes), not a reuse of test labels. There is no load-bearing self-citation: the only overlapping-author reference (Huang et al. 2025, cited in §1) is generic motivation; no uniqueness theorem or ansatz-by-citation is invoked. Two concerns keep the score above 2. First, the RL evaluation protocol for Table 3 is omitted: Appendix E.1 says guidance and summarization content are 'excluded during training' via masking, but masking removes tokens from the loss, not from the conditioning context, and the paper never states whether the trained policy was evaluated with or without H-EPM memory. On τ2-Bench the pure-inference H-EPM (0.232, +46.8%) already exceeds the best RL H-EPM (0.223, +41.1%), so the headline 'up to 40% on out-of-distribution tasks' is numerically subsumed by the §4.3 inference mechanism; unless Table 3 was evaluated policy-only, the 'policy internalizes strategies' claim reduces to the inference-time memory effect. Second, c (Appendix C.2, Table 7) and p_skip (§5.2) are selected using evaluation-benchmark scores, and the headline percentage is the best cell of that grid—mild fitting rather than construction-level circularity. Neither issue is a definitional equivalence, so the score is 3 rather than 6+.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 1 invented entities

The main free parameters are c, p_skip, and top-k, all chosen via small sweeps or ablations on the evaluation benchmarks. The core assumptions are that tool-transition statistics transfer across tasks and that the summarization/similarity pipeline faithfully represents decision context. The only invented entity is the state-summarization tool, which is a designed component rather than an independently verified mechanism.

free parameters (4)
  • c (efficiency coefficient) = 1 (selected from {0,1,5,10} on τ-Bench)
    Controls the balance between transition frequency and efficiency in the edge weight formula (Section 4.2). The value 1 is chosen by a sweep on τ-Bench and used elsewhere.
  • p_skip (procedural skip rate) = 0.8, 0.9, or 1.0 per benchmark
    In RL rollouts, the top-k procedural suggestions are skipped with probability p_skip. The optimal value is selected per benchmark (0.8 for ToolSandbox, 0.9 for τ-Bench, 1.0 for τ2-Bench) based on test-set results (Section 5.2).
  • top-k (number of suggested tools) = 2
    Selected from ablation {1,2,3}; k=2 gives the best trade-off (Table 4(D)).
  • Embedding model for state similarity = all-MiniLM-L6-v2
    Chosen for cosine similarity comparisons in episodic retrieval (Appendix E.1). Not fitted in the paper.
axioms (3)
  • domain assumption A tool-transition graph built from successful trajectories encodes transferable procedural knowledge across tasks.
    This is the premise of the procedural-memory component (Section 4.2). It assumes that tool-to-tool dependencies observed in training tasks generalize to test tasks.
  • domain assumption State summaries produced by the summarization tool capture decision-relevant information, and cosine similarity in the embedding space reflects task-relevant state similarity.
    Episodic retrieval relies on comparing the current state summary to stored summaries using all-MiniLM-L6-v2 (Section 4.3, Appendix E.1). No validation of this similarity assumption is provided.
  • domain assumption The LLM agent reliably decides when to invoke the summarization tool and adheres to the suggested top-2 tool list.
    The ablation 'w/o adapt' and the Limitation section (Appendix F) indicate that adherence varies by model; stronger models follow guidance less. This assumption is load-bearing for the adaptive retrieval mechanism.
invented entities (1)
  • State-summarization tool no independent evidence
    purpose: A new tool added to the agent's API that compresses the current dialogue and tool history into a compact state summary, used for episodic memory retrieval and decision-making.
    This tool is introduced by the paper and is not an independently validated artifact. Its effectiveness is only measured downstream through overall task accuracy.

pith-pipeline@v1.3.0-alltime-deepseek · 19474 in / 9038 out tokens · 82468 ms · 2026-08-03T17:57:28.357862+00:00 · methodology

0 comments
read the original abstract

As intents unfold and environments change, multi-turn agents face continuously shifting decision contexts. Although reusing past experience is intuitively appealing, existing approaches remain limited: full trajectories are often too context-specific to transfer, while tool-level reuse ignores the surrounding context and environment. In this paper, we introduce a hybrid episodic-procedural memory strategy (H-EPM) that enables experience-induced self-evolution of multi-turn tool-use policies by adaptively reusing partially overlapping successful experiences during both inference and training. Inspired by human episodic-procedural integration, we construct a tool graph from accumulated trajectories, where recurring tool-to-tool dependencies capture procedural routines and each edge is augmented with compact episodic summaries of relevant context. At inference time, the agent dynamically balances episodic recall for contextual reasoning with procedural execution for routine steps. Beyond inference, H-EPM introduces a memory-guided reinforcement learning paradigm that directly addresses a core challenge in multi-turn agent reinforcement learning, namely ineffective exploration over long trajectories. By biasing exploration toward historically successful tool transitions, H-EPM learns a stronger policy that generalizes at inference time without relying on domain-specific experience collection. Experiments show that H-EPM consistently delivers substantial inference-time gains over strong baselines across multi-turn tool-use benchmarks, reaching improvements of up to fifty percent. It also improves reinforcement learning policy performance, achieving gains of up to forty percent on out-of-distribution tasks.

Figures

Figures reproduced from arXiv: 2512.07287 by Jiang Bian, Jingjing Fu, Jun Zhang, Lei Song, Rui Wang, Sijia Li, Yuchen Huang, Zifan Liu, Zijian Li.

Figure 1
Figure 1. Figure 1: Building the memory graph in H-EPM. We build the graph based on previous successful trajectories. Except for the summarization tool, all tools invoked in historical trajectories are treated as nodes, and edges are constructed to connect them according to their calling sequential relationships. The state information summarized by the summarization tool is treated as edge information. Each edge must contain … view at source ↗
Figure 2
Figure 2. Figure 2: Adaptive experience retrieval with H-EPM. First, we locate the last tool call in the graph and identify the candidate node it is connected to. Then, the agent decides whether it should summarize the current state. If the decision is yes, the agent calls the summary tool and abstracts the current state as a summary. It then compares the similarity between this summarized state and the stored states in the c… view at source ↗
Figure 3
Figure 3. Figure 3: Comparison with episodic memory and tool graph based methods in multi-turn tool-use tasks. (a) In memory-based methods, memories are stored as discrete items, but these approaches typically incorporate only episodic event memories and retrieve trajectory-level experience before execution. Before the task objective becomes sufficiently clear, this retrieval process may return mismatched or inappropriate exp… view at source ↗
Figure 4
Figure 4. Figure 4: An illustrative trajectory example produced by H-EPM. In the trajectory, when the agent decides to invoke the summarization tool, it compares the current state with state information on connected edges to provide guidance via episodic memory. If the agent decides not to summarize the state at the current step, H-EPM provides guidance based on transition weights, corresponding to procedural memory. 17 [PIT… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective

    cs.CL 2026-05 unverdicted novelty 6.0

    EvoMemBench evaluates 15 memory methods for LLM agents and finds long-context baselines competitive with no single memory approach working consistently across settings.

Reference graph

Works this paper leans on

30 extracted references · 22 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Agashe, S., Han, J., Gan, S., Yang, J., Li, A., and Wang, X. E. Agent s: An open agentic framework that uses computers like a human.arXiv preprint arXiv:2410.08164,

    Pith/arXiv arXiv

  2. [5]

    Tool-star: Empow- ering llm-brained multi-tool reasoner via reinforcement learning.arXiv preprint arXiv:2505.16410,

    Dong, G., Chen, Y ., Li, X., Jin, J., Qian, H., Zhu, Y ., Mao, H., Zhou, G., Dou, Z., and Wen, J.-R. Tool-star: Empow- ering llm-brained multi-tool reasoner via reinforcement learning.arXiv preprint arXiv:2505.16410,

    Pith/arXiv arXiv

  3. [6]

    Group-in-group policy optimization for llm agent training.arXiv preprint arXiv:2505.10978,

    Feng, L., Xue, Z., Liu, T., and An, B. Group-in-group policy optimization for llm agent training.arXiv preprint arXiv:2505.10978,

    Pith/arXiv arXiv

  4. [7]

    Evaluating llm-based agents for multi-turn conversa- tions: A survey.arXiv preprint arXiv:2503.22458,

    Guan, S., Xiong, H., Wang, J., Bian, J., Zhu, B., and Lou, J.-g. Evaluating llm-based agents for multi-turn conversa- tions: A survey.arXiv preprint arXiv:2503.22458,

    arXiv

  5. [8]

    X., Song, Y ., Zhu, C., Zhu, H., and Wen, J.-R

    Jiang, J., Zhou, K., Zhao, W. X., Song, Y ., Zhu, C., Zhu, H., and Wen, J.-R. Kg-agent: An efficient autonomous agent framework for complex reasoning over knowledge graph. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pp. 9505–9523, 2025a. Jiang, X., Li, F., Zhao, H., Qiu, J., Wang, J.,...

    Pith/arXiv arXiv

  6. [9]

    Naviagent: Bilevel planning on tool dependency graphs for function calling.arXiv preprint arXiv:2506.19500, 2025b

    Jiang, Y ., Zhou, H., GU, L., Han, A., and Li, T. Naviagent: Bilevel planning on tool dependency graphs for function calling.arXiv preprint arXiv:2506.19500, 2025b. Lee, K.-H., Chen, X., Furuta, H., Canny, J., and Fischer, I. A human-inspired reading agent with gist memory of very long contexts.arXiv preprint arXiv:2402.09727,

    Pith/arXiv arXiv

  7. [10]

    Toolnet: Connecting large language mod- els with massive tools via tool graph.arXiv preprint arXiv:2403.00839, 2024a

    Liu, X., Peng, Z., Yi, X., Xie, X., Xiang, L., Liu, Y ., and Xu, D. Toolnet: Connecting large language mod- els with massive tools via tool graph.arXiv preprint arXiv:2403.00839, 2024a. Liu, Y ., Zhang, G., Wang, K., Li, S., and Pan, S. Graph-augmented large language model agents: Cur- rent progress and future prospects.arXiv preprint arXiv:2507.21407,

    Pith/arXiv arXiv

  8. [11]

    Controlllm: Augment language models with tools by searching on graphs

    Liu, Z., Lai, Z., Gao, Z., Cui, E., Li, Z., Zhu, X., Lu, L., Chen, Q., Qiao, Y ., Dai, J., et al. Controlllm: Augment language models with tools by searching on graphs. In ECCV (12), 2024b. Lu, J., Holleis, T., Zhang, Y ., Aumayer, B., Nan, F., Bai, F., Ma, S., Ma, S., Li, M., Yin, G., et al. Toolsand- box: A stateful, conversational, interactive evaluati...

    Pith/arXiv arXiv

  9. [12]

    T., Daruki, S., Tang, X., et al

    9 Ouyang, S., Yan, J., Hsu, I., Chen, Y ., Jiang, K., Wang, Z., Han, R., Le, L. T., Daruki, S., Tang, X., et al. Rea- soningbank: Scaling agent self-evolving with reasoning memory.arXiv preprint arXiv:2509.25140,

    Pith/arXiv arXiv

  10. [13]

    and Jadon, A

    Patil, A. and Jadon, A. Advancing reasoning in large lan- guage models: Promising methods and approaches.arXiv preprint arXiv:2502.03671,

    Pith/arXiv arXiv

  11. [14]

    From iso- lated conversations to hierarchical schemas: Dynamic tree memory representation for llms.arXiv preprint arXiv:2410.14052,

    Rezazadeh, A., Li, Z., Wei, W., and Bao, Y . From iso- lated conversations to hierarchical schemas: Dynamic tree memory representation for llms.arXiv preprint arXiv:2410.14052,

    Pith/arXiv arXiv

  12. [15]

    Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y ., Wu, Y ., et al. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  13. [17]

    Chemagent: Self-updating library in large language models improves chemical reasoning.arXiv preprint arXiv:2501.06590,

    Tang, X., Hu, T., Ye, M., Shao, Y ., Yin, X., Ouyang, S., Zhou, W., Lu, P., Zhang, Z., Zhao, Y ., et al. Chemagent: Self-updating library in large language models improves chemical reasoning.arXiv preprint arXiv:2501.06590,

    Pith/arXiv arXiv

  14. [18]

    Enhancing large language model with self-controlled memory framework.arXiv preprint arXiv:2304.13343,

    Wang, B., Liang, X., Yang, J., Huang, H., Wu, S., Wu, P., Lu, L., Ma, Z., and Li, Z. Enhancing large language model with self-controlled memory framework.arXiv preprint arXiv:2304.13343,

    Pith/arXiv arXiv

  15. [20]

    N., Liu, L., et al

    Wang, Z., Wang, K., Wang, Q., Zhang, P., Li, L., Yang, Z., Jin, X., Yu, K., Nguyen, M. N., Liu, L., et al. Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning.arXiv preprint arXiv:2504.20073, 2025b. Wise, S. P. The role of the basal ganglia in procedural memory. InSeminars in Neuroscience, volume 8, pp. 39–46. Elsevier,

    Pith/arXiv arXiv

  16. [21]

    How memory management impacts llm agents: An empirical study of experience-following behavior.arXiv preprint arXiv:2505.16067,

    Xiong, Z., Lin, Y ., Xie, W., He, P., Tang, J., Lakkaraju, H., and Xiang, Z. How memory management impacts llm agents: An empirical study of experience-following behavior.arXiv preprint arXiv:2505.16067,

    arXiv

  17. [22]

    A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110,

    Xu, W., Mei, K., Gao, H., Tan, J., Liang, Z., and Zhang, Y . A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110,

    Pith/arXiv arXiv

  18. [23]

    τ-bench: A benchmark for tool-agent-user interaction in real-world domains.arXiv preprint arXiv:2406.12045,

    Yao, S., Shinn, N., Razavi, P., and Narasimhan, K. τ-bench: A benchmark for tool-agent-user interaction in real-world domains.arXiv preprint arXiv:2406.12045,

    Pith/arXiv arXiv

  19. [24]

    Easytool: Enhancing llm-based agents with concise tool instruction.arXiv preprint arXiv:2401.06201,

    Yuan, S., Song, K., Chen, J., Tan, X., Shen, Y ., Kan, R., Li, D., and Yang, D. Easytool: Enhancing llm-based agents with concise tool instruction.arXiv preprint arXiv:2401.06201,

    Pith/arXiv arXiv

  20. [25]

    Agentevolver: To- wards efficient self-evolving agent system.arXiv preprint arXiv:2511.10395,

    Zhai, Y ., Tao, S., Chen, C., Zou, A., Chen, Z., Fu, Q., Mai, S., Yu, L., Deng, J., Cao, Z., et al. Agentevolver: To- wards efficient self-evolving agent system.arXiv preprint arXiv:2511.10395,

    arXiv

  21. [26]

    Lightthinker: Thinking step- by-step compression.arXiv preprint arXiv:2502.15589, 2025a

    Zhang, J., Zhu, Y ., Sun, M., Luo, Y ., Qiao, S., Du, L., Zheng, D., Chen, H., and Zhang, N. Lightthinker: Thinking step- by-step compression.arXiv preprint arXiv:2502.15589, 2025a. Zhang, Y ., Wang, H., Feng, S., Tan, Z., Han, X., He, T., and Tsvetkov, Y . Can llm graph reasoning generalize beyond pattern memorization?arXiv preprint arXiv:2406.15992,

    arXiv

  22. [27]

    Memengine: A unified and modular library for develop- ing advanced memory of llm-based agents

    Zhang, Z., Dai, Q., Chen, X., Li, R., Li, Z., and Dong, Z. Memengine: A unified and modular library for develop- ing advanced memory of llm-based agents. InCompanion Proceedings of the ACM on Web Conference 2025, pp. 821–824, 2025b. 10 Zhao, W., Wang, X., Ma, C., Kong, L., Yang, Z., Tuo, M., Shi, X., Zhai, Y ., and Cai, X. Mua-rl: Multi-turn user- interac...

    Pith/arXiv arXiv 2025

  23. [28]

    Pro: Process

    11 Table 6.Performance comparison across two evaluation metrics in agent multi-turn task ACEBench (process accuracy and end-to-end accuracy). Pro: Process. EtoE: end-to-end. RB: Reasoning Bank. RT: Retrieval Top-1. METRIC MODEL BASERB TOOLNETRT H-EPM PROACC GPT-4.1-MINI 0.633 0.701 0.627 0.6410.719 GPT-4.1 0.653 0.703 0.664 0.6010.765 GPT-4O 0.664 0.684 0...

    2025

  24. [29]

    encode expert knowledge to guide LLMs in executing multi-step toolchains in scientific domains. Finally, approaches supporting dynamic updates, such as ToolNet (Liu et al., 2024a) and SGC (Wu et al., 2024), organize massive tool collections into weighted directed graphs and update them adaptively based on usage, enabling more efficient navigation and plan...

    2024

  25. [30]

    Suggested next tools: Tool 1, Tool 2

    Guidance from episodic and procedural memory is injected as a system prompt in the form of“Suggested next tools: Tool 1, Tool 2”when the top-2 tools satisfy the selection criteria, which is illustrated in the Appendix B. Details for RL.We specify several key parameters. The Kullback–Leibler (KL) divergence loss coefficient is set to β= 0.001 . The trainin...

    1985

  26. [2004]

    τ2-bench: Evaluating conversational agents in a dual- control environment.arXiv preprint arXiv:2506.07982,

    Barres, V ., Dong, H., Ray, S., Si, X., and Narasimhan, K. τ2-bench: Evaluating conversational agents in a dual- control environment.arXiv preprint arXiv:2506.07982,

    Pith/arXiv arXiv

  27. [2008]

    Beyond browsing: Api-based web agents.arXiv preprint arXiv:2410.16464,

    Song, Y ., Xu, F., Zhou, S., and Neubig, G. Beyond browsing: Api-based web agents.arXiv preprint arXiv:2410.16464,

    Pith/arXiv arXiv

  28. [2023]

    Z., and Wong, K.-F

    Wang, H., Huang, W., Wang, Y ., Xi, Y ., Lu, J., Zhang, H., Hu, N., Liu, Z., Pan, J. Z., and Wong, K.-F. Rethinking stateful tool use in multi-turn dialogues: Benchmarks and challenges.arXiv preprint arXiv:2505.13328, 2025a. Wang, Z., Zeng, X., Liu, W., Li, L., Wang, Y ., Shang, L., Jiang, X., Liu, Q., and Wong, K.-F. Toolflow: Boosting llm tool-calling t...

    Pith/arXiv arXiv

  29. [2024]

    Arigraph: Learning knowledge graph world models with episodic memory for llm agents.arXiv preprint arXiv:2407.04363,

    Anokhin, P., Semenov, N., Sorokin, A., Evseev, D., Kravchenko, A., Burtsev, M., and Burnaev, E. Arigraph: Learning knowledge graph world models with episodic memory for llm agents.arXiv preprint arXiv:2407.04363,

    Pith/arXiv arXiv

  30. [2025]

    Acebench: Who wins the match point in tool usage?arXiv preprint arXiv:2501.12851, 2025a

    Chen, C., Hao, X., Liu, W., Huang, X., Zeng, X., Yu, S., Li, D., Wang, S., Gan, W., Huang, Y ., et al. Acebench: Who wins the match point in tool usage?arXiv preprint arXiv:2501.12851, 2025a. Chen, W., Li, W., Yao, D., Meng, X., Gong, C., and Bi, J. Gtool: Graph enhanced tool planning with large language model.arXiv preprint arXiv:2508.12725, 2025b. Chhik...

    arXiv