Pith. sign in

REVIEW 4 major objections 5 minor 31 references

KV cache reuse errors in agentic memory reduce to a single shared chunk-level offset, and correcting it from a handful of probe tokens recovers near-fresh attention.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 13:51 UTC pith:ZMGQOU6C

load-bearing objection A genuine mean-shift correction for KV reuse with solid empirical support, but the probe 'fresh' computation is underspecified and the theory doesn't match the weighted algorithm; both need fixing before the speedup claims are credible. the 4 major comments →

arxiv 2607.21604 v1 pith:ZMGQOU6C submitted 2026-05-15 cs.AI

AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems

Nilesh Prasad Pandey , Jason Kong , Lanxiang Hu , Quanling Zhao , Yujie Zhao , Onat Gungor , Hao Zhang , Tajana Rosing This is my paper
classification cs.AI
keywords KV cache reuseagentic memoryprobe-guided correctionresidual decompositionLLM prefillattention error boundKV quantizationmemory-augmented agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AgentKVShift claims that when a remembered text chunk is retrieved under a new context, the stale key-value states differ from fresh ones mostly by a common offset shared across the entire chunk, plus small per-token noise. Estimating that offset from a tiny probe set, whose tokens are actually recomputed, and adding the estimate to every other token's cached states, recovers most of the quality of full recomputation. The paper argues this turns the recompute budget into a correction signal for the whole chunk, rather than only refreshing a few tokens. Across several open models and two agentic memory designs, the method stays within 1.5–6% relative F1 of full recompute while refreshing only 10–30% of the cache, with 2–3.5x prefill speedups. This matters because memory-augmented agents pay full re-encoding costs on every retrieval, and existing token-selection reuse methods degrade sharply on structured, metadata-rich memories.

Core claim

The paper's central claim is that the per-memory KV reuse residual, fresh minus cached, decomposes as R_i = μ + ξ_i, where μ is a shared chunk-level offset and ξ_i are centered, sub-Gaussian-style token-wise fluctuations. A small set of probe tokens whose fresh KVs are recomputed at layer 1 provides an estimator for μ at every layer; subtracting that estimate from all reused tokens removes the common bias and leaves only probe-estimation error plus fluctuations. The theoretical analysis (Propositions 1–3) bounds attention error before and after correction and shows the corrected bound is strictly tighter whenever the common-bias term exceeds the probe-estimation term. Empirically, this singl

What carries the argument

The load-bearing object is the residual decomposition r_i = μ + ξ_i for key and value states of every token in a retrieved memory chunk, with μ the shared chunk-level offset and ξ_i the token-wise fluctuation. AgentKVShift selects top-b tokens by layer-1 divergence as probes, recomputes their fresh KVs at every layer, estimates μ as the probe mean, and applies a weighted vector addition (K_reuse + w·μ̂) to every non-probe token. This turns the refresh budget into a global correction rather than a per-token decision. The theoretical engine is the deterministic attention-perturbation bound of Lemma 1, combined with Proposition 3, which shows correction wins when the common bias exceeds the pro

Load-bearing premise

The method stands or falls on the premise that probe tokens selected by layer-1 divergence remain representative of the chunk-level residual offset at every later layer, and that token-wise fluctuations are light-tailed enough for the probe mean to concentrate around the true offset.

What would settle it

Construct or find a retrieved memory chunk where the dominant residual offset rotates strongly across layers, for example a chunk whose summary tokens are semantically stable early but heavily re-contextualized deeper in the network; if layer-1 probes miss the deep-layer offset, corrected attention error would fail to decay as the probe budget grows or would stay above the uncorrected bound. A direct measurement of ∥μ̂_b − μ∥ across layers for both K and V, checking whether it decays at the b^-1/2 sample-mean rate for values in shallow layers, would settle the core assumption.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Near-full accuracy at 10% refresh: on a long-horizon dialogue benchmark, relative F1 drop to full recompute stays within 1.5–6% while refreshing only 10% of tokens.
  • Fivefold lower recompute: AgentKVShift reaches quality that prior token-selection reuse methods achieve only at 45–55% refresh, using roughly 10% instead.
  • Prefill speedups of 2–3.5x over full recompute on a single GPU in the 10–30% refresh regime, growing with context length and batch size.
  • Composition with KV quantization: the probe estimate absorbs average quantization error, retaining over 2x the F1 of prior reuse methods at aggressive 2- and 4-bit settings.
  • Generality: the correction transfers across model scales from 3B to 32B parameters and across both note-based and graph-based agentic memory systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same residual-decomposition argument may extend beyond agentic memory: any dynamic retrieval setting where identical text is re-encoded under different contexts, such as RAG with template-heavy metadata, could benefit from probe-guided mean correction.
  • Because probes are fixed at layer 1 but offsets are re-estimated per layer, a natural extension is layer-dependent probe selection or asymmetric K/V probe budgets; the paper's own diagnostics show values at shallow layers deviate from the sub-Gaussian assumption, so a value-specific correction scheme might close the remaining gap.
  • The cross-chunk reasoning gap reported in the appendix suggests memory-local correction has a ceiling; a state-aware variant that conditions the offset on the agent's current reasoning state could address capabilities like state updating and causal inference.
  • The theory predicts the corrected bound tightens as probe budget b grows; a direct experiment sweeping b and measuring attention error against the b^-1/2 prediction would give a sharper test of the sub-Gaussian core assumption.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. AgentKVShift proposes a training-free KV cache reuse correction for agentic memory systems. The authors observe that the residual between fresh and cached KV states for a retrieved memory unit is dominated by a chunk-level offset, with smaller token-wise fluctuations. The method selects a small probe set (b = ceil(r n)) based on layer-1 KV divergence, estimates the per-layer mean residual over the probe set, and corrects all non-probe tokens by adding a divergence-weighted mean shift. They report F1/ROUGE/BLEU on LoCoMo and AMA-Bench-Recall with four LLMs, showing within 1.5–6% of full recompute at r=0.1/0.3, prefill speedups of 2–3.5×, and gains over CacheBlend/ProphetKV, including under KV quantization. A theoretical analysis bounds attention error before and after correction under a sub-Gaussian assumption.

Significance. If the implementation is sound, the paper makes a useful empirical contribution: it identifies a structured component in KV reuse error for agentic memory and demonstrates that a simple mean-shift correction substantially improves over token-selection-only reuse. The evaluation is broad (two memory systems, two benchmarks, four models, quantization combinations) and the appendix contains honest diagnostics: mean-explained energy, probe-concentration checks, and explicit limitations. The method is training-free and trivial to implement, which is practically attractive. However, the central correctness of the speedup claim depends on how probe 'fresh' states are computed; the manuscript as written is ambiguous on this point.

major comments (4)
  1. [§3.3, Algorithm 1 lines 4–8; §3.1] The procedure 'recompute fresh KV for S^K_C ∪ S^V_C at layer ℓ' is not implementable as stated without a full forward pass. K^ℓ_fresh is defined under full recomputation in the current context; the KV of a token at layer ℓ depends on its hidden state at layer ℓ−1, which is produced by attention over all preceding tokens. If only probe tokens are run through the network with cached KV for other tokens, the resulting probe state is not K^ℓ_fresh but a stale-neighbor state, making the probe mean a biased estimator of µ. This bias is absent from E_b in Proposition 2. If a full forward pass is performed instead, the 2–3.5× speedups in Table 3 are not attributable to refreshing only 10–30% of the cache. Please specify the exact forward operations used to obtain probe states, and include their cost in the speedup accounting.
  2. [§3.4, Props. 2–3 and App. B.3] The theoretical guarantee is proven for an unweighted correction using a uniform random probe set, with the same mean added to all tokens. Algorithm 1 instead uses per-token weights w_{K,j}=min(d_{K,j},1) and selects the top-b divergent probes at layer 1. Proposition 2's bound and Proposition 3's sufficient condition do not cover this weighted, non-random estimator. Thus the statement 'AgentKVShift admits a tighter attention-error bound' is not actually established for the algorithm evaluated in the experiments.
  3. [App. A.2, Table 7] The text states that AgentKVShift 'remains the strongest KV-reuse strategy across all reasoning categories,' but the State Abstraction row of Table 7 shows ProphetKV with F1 0.254 (81.7%) versus 0.235 (75.6%) for AgentKVShift, and LLM-Judge accuracy is tied at 0.118. This contradicts the claim in the Conclusion that AgentKVShift 'outperforms CacheBlend and ProphetKV across all tested configurations.' Please correct the claim or explain the discrepancy.
  4. [§4.1, Tables 1–2] No error bars, standard deviations, or number of seeds are reported. LoCoMo contains only 10 multi-session conversations, so the headline statement of 'within 1.5–6% relative F1' is based on a single run. Given that some cells differ by only 1–2 F1 points (e.g., the weighted-average gap in Table 2 is 0.012 F1), the central quantitative comparison to full recompute requires an estimate of run-to-run variability.
minor comments (5)
  1. [Table 3] The batch-size labels and values are run together (e.g., 'B=14 8 16B=14 ...'), making the table unreadable. Please reformat so each σ? column is clearly labeled.
  2. [Tables 1 and 4] CacheBlend on Qwen2.5-3B, AMem, r=0.1 reports F1=0.178 in Table 1 but 0.150 in Table 4 under 'No Quant'. Since both settings are described as LoCoMo/AMem/Qwen2.5-3B at r=0.1, this inconsistency needs to be resolved.
  3. [Tables 2 and 7] Several 'retention ratio' values exceed 100% (e.g., Software F1 104.7% in Table 2). The caption should clarify that this is a ratio to full recompute and that values above 100% are possible due to noise or metric properties.
  4. [Appendix A.1 vs Table 3 caption] Appendix A.1 says LoCoMo and throughput experiments on 3B models use A100 80GB, but Table 3's caption says A100-40GB for Qwen2.5-3B. Please align the hardware specifications.
  5. [Figure 4] The inset panels with axes labeled '1.0 1.5 2.0' and F1-like values are not described in the caption. Please explain what the insets show and how they were produced.

Circularity Check

0 steps flagged

No significant circularity; probe offsets are empirical estimates of the cached/fresh KV gap on the current chunk, not fits to benchmark answers.

full rationale

The derivation chain is: define the KV reuse residual R = K_fresh - K_reuse; observe via SVD that R is dominated by a shared chunk-level offset; estimate that offset from a small probe set; add the weighted offset to all reused tokens; and test the resulting attention quality on LoCoMo and AMA-Bench. Each step is empirically grounded rather than definitionally equivalent to the conclusion. The probe mean is computed from the current chunk's own cached/fresh KV gap and applied before generation; it is never fitted to benchmark labels or to downstream F1, so the near-full-recompute results are genuine empirical evidence for the residual-decomposition hypothesis rather than a forced consequence of the method's construction. The same-author citation of AMA-Bench [28] is a benchmark dataset, externally defined and not parameterized by AgentKVShift, so it is not load-bearing self-citation. The admitted limitations, such as the sub-Gaussian assumption fitting keys more cleanly than values at shallow layers (Appendix C.2 and Appendix E), weaken the theoretical bounds' applicability but do not make the derivation circular. The underspecified 'recompute fresh KV for probe set' step in Algorithm 1 is a feasibility/correctness concern about whether the stated speedups are attainable, not a circularity: it does not reduce the method's prediction to its inputs. Proposition 3's sufficient condition is an algebraic comparison of the two derived upper bounds; while tautological in form, it is not a claim that a fitted parameter predicts itself. Overall, no circular step was identified under the required evidentiary standard.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The paper's central contribution rests on a data-driven statistical assumption (residual = shared offset + light-tailed fluctuations) and a probe-transfer assumption. These are plausible and partly validated, but they are assumptions, not derived facts. No free parameters are fitted to benchmark labels.

free parameters (3)
  • probe budget b = ceil(r n) = r=0.1 for LoCoMo, r=0.3 for AMA-Bench (10% and 30% of tokens)
    Operating points chosen to claim 'refreshing only 10-30% of the cache'. Not fitted to labels, but the headline accuracy/speed trade-off is evaluated at these hand-picked ratios.
  • per-token weights w = min(divergence, 1)
    Heuristic weight truncation in Algorithm 1; not derived from the theoretical bound, which analyzes unweighted correction (Appendix B.3).
  • check layer ℓ_c = 1 = 1
    Probe selection uses first-layer divergence only; choice follows CacheBlend and is not independently justified for agentic memory.
axioms (5)
  • standard math Softmax is 1/2-Lipschitz in l∞ (Lemma 1 uses [14])
    Needed for the deterministic attention perturbation bound in Appendix B.1.
  • ad hoc to paper KV residual vector r_i = k*_i - k_reuse_i decomposes as shared chunk offset µ plus zero-mean token fluctuation ξ_i
    Central modeling assumption; supported empirically in Figure 2 and Appendix C.1, but not proven.
  • domain assumption Token-wise fluctuations ξ_i are sub-Gaussian with scales σ_K, σ_V
    Used in Propositions 1-3; Appendix C.2 shows K is consistent, V is non-Gaussian at shallow layers.
  • ad hoc to paper Layer-1 divergence-based probe set remains representative of the residual offset at all deeper layers
    Algorithm 1 selects probes once at ℓ_c=1 and reuses them; no direct validation that top-divergence tokens at layer 1 track the chunk mean at layers 2..L.
  • standard math Value vectors are norm-bounded by V_max
    Standard boundedness assumption in Lemma 1.

pith-pipeline@v1.3.0-alltime-deepseek · 22836 in / 12374 out tokens · 105783 ms · 2026-08-02T13:51:11.944644+00:00 · methodology

0 comments
read the original abstract

Memory-augmented LLM agents maintain context across hundreds of interactions through agentic memory systems that actively curate retrieved content with LLM-generated metadata such as summaries, keywords, and tags. From an inference cost standpoint, every retrieval triggers a full re-encoding of these structured memory units into Key-Value (KV) states, which dominates prefill latency. Existing training-free KV reuse methods mitigate this by selectively recomputing a small fraction of tokens, but were designed for RAG-style raw passages and degrade on structured agentic memories. We present AgentKVShift, a training-free, probe-guided KV residual correction method that operates per retrieved memory unit. One of the crucial insights we demonstrate is that the per-memory KV reuse residual decomposes into a shared memory-level offset plus small token-wise fluctuations. Estimating this offset from a small probe set allows us to correct every reused token by a single weighted correction. Unlike prior reuse methods which decide which tokens to recompute and leave the rest of the cache stale, AgentKVShift also corrects the tokens it does not recompute, turning the refresh budget into useful signal across the entire chunk. Across four open-source LLMs spanning 3B to 32B parameters and two long-horizon agentic memory benchmarks (long-term dialogue and agentic applications), AgentKVShift achieves near full recompute performance while refreshing only 10-30% of the cache, outperforming baselines at the same recompute ratio. It requires up to 5x lower recompute to reach this near-full performance, which prior reuse methods only attain at 45-55% refresh. In this regime, AgentKVShift delivers prefill speedups of 2-3.5x over no-KV-reuse on a single A100. AgentKVShift orthogonally composes with KV cache quantization, retaining over 2x the F1 of prior reuse methods under aggressive 2- and 4-bit settings.

Figures

Figures reproduced from arXiv: 2607.21604 by Hao Zhang, Jason Kong, Lanxiang Hu, Nilesh Prasad Pandey, Onat Gungor, Quanling Zhao, Tajana Rosing, Yujie Zhao.

Figure 1
Figure 1. Figure 1: AgentKVShift: KV reuse via residual estimation. The mean residuals [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SVD spectra of R ℓ for K and V. To understand the structure of the reuse error, we analyze the memory-wise KV residuals Rℓ K and Rℓ V at different transformer layers. We compute their singular value spectra and contrast them against the spectra obtained after subtracting the chunk mean from each row ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Speedup vs. prefill-latency Pareto sweep at 4,096 tok/req [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Effect of recompute ratio on F1. The same ordering holds for ROUGE-1 and BLEU-1 as shown in Appendix A.3. This reflects a difference in how each method uses the refresh budget. CacheBlend and ProphetKV select which tokens to recompute and leave the rest of the cache uncorrected, so their accuracy scales with the fraction of tokens refreshed. AgentKVShift instead cor￾rects the reused keys themselves using a… view at source ↗
Figure 5
Figure 5. Figure 5: Recompute-ratio sweep on ROUGE-1 (left) and BLEU-1 (right). The dashed line denotes [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Mean-explained residual energy (dark) and remaining top singular value ratio (light) after [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Empirical diagnostics of the token-wise fluctuation term for K (left) and V (right) across [PITH_FULL_IMAGE:figures/full_fig_p023_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 12 linked inside Pith

  1. [1]

    Taming{Throughput-Latency} tradeoff in {LLM} inference with {Sarathi-Serve}

    Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gula- vani, Alexey Tumanov, and Ramachandran Ramjee. Taming{Throughput-Latency} tradeoff in {LLM} inference with {Sarathi-Serve}. In18th USENIX symposium on operating systems design and implementation (OSDI 24), pages 117–134, 2024

    2024

  2. [2]

    Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Prompt cache: Modular attention reuse for low-latency inference.Proceedings of Machine Learning and Systems, 6:325–338, 2024

    In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. Prompt cache: Modular attention reuse for low-latency inference.Proceedings of Machine Learning and Systems, 6:325–338, 2024

    2024

  4. [4]

    Licomemory: Lightweight and cognitive agentic memory for efficient long-term reasoning.arXiv preprint arXiv:2511.01448, 2025

    Zhengjun Huang, Zhoujin Tian, Qintian Guo, Fangyuan Zhang, Yingli Zhou, Di Jiang, Zeying Xie, and Xiaofang Zhou. Licomemory: Lightweight and cognitive agentic memory for efficient long-term reasoning.arXiv preprint arXiv:2511.01448, 2025

    arXiv 2025

  5. [5]

    Unsupervised dense information retrieval with contrastive learning

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118, 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 6769–6781, 2020

    2020

  7. [7]

    PhD thesis, UC Berkeley, 2025

    Woosuk Kwon.vLLM: An Efficient Inference Engine for Large Language Models. PhD thesis, UC Berkeley, 2025

    2025

  8. [8]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. InProceedings of the 29th symposium on operating systems principles, pages 611–626, 2023

    2023

  9. [9]

    A human-inspired reading agent with gist memory of very long contexts.arXiv preprint arXiv:2402.09727, 2024

    Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. A human-inspired reading agent with gist memory of very long contexts.arXiv preprint arXiv:2402.09727, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    2020

  11. [11]

    Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024

    Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Evaluating very long-term conversational memory of llm agents

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851–13870, 2024

    2024

  13. [13]

    Mistral-7B-Instruct-v0.3

    Mistral AI. Mistral-7B-Instruct-v0.3. https://huggingface.co/mistralai/ Mistral-7B-Instruct-v0.3 , 2024. Instruction fine-tuned version of Mistral-7B-v0.3 with extended vocabulary (32,768), v3 tokenizer, and function calling support. 10

    2024

  14. [14]

    Softmax is 1/2-lipschitz: A tight bound across all lp norms.arXiv preprint arXiv:2510.23012, 2025

    Pravin Nair. Softmax is 1/2-lipschitz: A tight bound across all lp norms.arXiv preprint arXiv:2510.23012, 2025

    arXiv 2025

  15. [15]

    OpenAI. Gpt-4o. https://platform.openai.com/, 2024. OpenAI multimodal large language model

    2024

  16. [16]

    Memgpt: towards llms as operating systems

    Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonza- lez. Memgpt: towards llms as operating systems. 2023

    2023

  17. [17]

    Position: Episodic memory is the missing piece for long-term llm agents.arXiv preprint arXiv:2502.06975, 2025

    Mathis Pink, Qinyuan Wu, Vy Ai V o, Javier Turek, Jianing Mu, Alexander Huth, and Mariya Toneva. Position: Episodic memory is the missing piece for long-term llm agents.arXiv preprint arXiv:2502.06975, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Zep: a temporal knowledge graph architecture for agent memory.arXiv preprint arXiv:2501.13956, 2025

    Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. Zep: a temporal knowledge graph architecture for agent memory.arXiv preprint arXiv:2501.13956, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    The probabilistic relevance framework: Bm25 and beyond.Foundations and Trends in Information Retrieval, 3(4):333–389, 2009

    Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: Bm25 and beyond.Foundations and Trends in Information Retrieval, 3(4):333–389, 2009

    2009

  20. [20]

    Accurate kv cache quantization with outlier tokens tracing

    Yi Su, Yuechi Zhou, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, and Min Zhang. Accurate kv cache quantization with outlier tokens tracing. InProceed- ings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12895–12915, 2025

    2025

  21. [21]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https:// qwenlm.github.io/blog/qwen2.5/

    2024

  22. [22]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  23. [23]

    Prophetkv: User-query-driven selective recomputation for efficient kv cache reuse in retrieval-augmented generation.arXiv preprint arXiv:2602.02579, 2026

    Shihao Wang, Jiahao Chen, Yanqi Pan, Hao Huang, Yichen Hao, Xiangyu Zou, Wen Xia, Wentao Zhang, Haitao Wang, Junhong Li, et al. Prophetkv: User-query-driven selective recomputation for efficient kv cache reuse in retrieval-augmented generation.arXiv preprint arXiv:2602.02579, 2026

    arXiv 2026

  24. [24]

    A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110, 2025

    Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning.arXiv preprint arXiv:2508.19828, 2025

    Sikuan Yan, Xiufeng Yang, Xiaowen Ma, Ercong Nie, Zifeng Ding, Yunpu Ma, Zonggen Li, Zuchao Huang, Hinrich Schütze, and V olker Tresp. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning.arXiv preprint arXiv:2508.19828, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Cacheblend: Fast large language model serving for rag with cached knowledge fusion

    Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, and Junchen Jiang. Cacheblend: Fast large language model serving for rag with cached knowledge fusion. InProceedings of the twentieth European conference on computer systems, pages 94–109, 2025

    2025

  27. [27]

    G- memory: Tracing hierarchical memory for multi-agent systems, 2025.URL https://arxiv

    Guibin Zhang, Muxin Fu, Guancheng Wan, Miao Yu, Kun Wang, and Shuicheng Yan. G- memory: Tracing hierarchical memory for multi-agent systems, 2025.URL https://arxiv. org/abs/2506.07398

    Pith/arXiv arXiv 2025

  28. [28]

    Ama-bench: Evaluating long- horizon memory for agentic applications.arXiv preprint arXiv:2602.22769, 2026

    Yujie Zhao, Boqin Yuan, Junbo Huang, Haocheng Yuan, Zhongming Yu, Haozhou Xu, Lanxiang Hu, Abhilash Shankarampeta, Zimeng Huang, Wentao Ni, et al. Ama-bench: Evaluating long- horizon memory for agentic applications.arXiv preprint arXiv:2602.22769, 2026

    Pith/arXiv arXiv 2026

  29. [29]

    Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37: 62557–62583, 2024

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37: 62557–62583, 2024

    2024

  30. [30]

    Memorybank: Enhancing large language models with long-term memory

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. InProceedings of the AAAI conference on artificial intelligence, volume 38, pages 19724–19731, 2024. 11

    2024

  31. [31]

    yes” or “no

    Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. Mem1: Learning to synergize memory and reasoning for efficient long-horizon agents.arXiv preprint arXiv:2506.15841, 2025. 12 Contents This appendix provides supplementary results, evaluation details, and formal analysis supporti...

    Pith/arXiv arXiv 2025