Pith. sign in

REVIEW 4 major objections 53 references

Chunk-bounded memory with cross-chunk reconciliation and co-located storage improves agent accuracy and serving throughput together.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 03:20 UTC pith:BBKYPCWB

load-bearing objection Solid agent-memory systems paper with real Pareto gains; the efficiency half is less cleanly measured than the accuracy half because auxiliary model calls are not cost-broken-out. the 4 major comments →

arxiv 2607.05708 v1 pith:BBKYPCWB submitted 2026-07-07 cs.AI

Akashic: A Low-Overhead LLM Inference Service with MemAttention

Yang Liu , Zhaokai Luo , Huayi Jin , Ruozhou He , Chenchen Hong , Zhiyong Wang , Yifei Liu , Yunfei Gu
show 2 more authors
Chentao Wu Junhao Hu
This is my paper
classification cs.AI
keywords LLM servingagent memoryMemAttentionchunk compactioncross-chunk reconciliationlocality-aware storagelong-context agentsretrieval-augmented inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-horizon LLM agents pile up multi-turn, tool, and session history until replaying the full context is too slow, too long, and too noisy. This paper argues that the usual fixes—global summarization or independent segment summaries—either grow too expensive or split evidence that must be used together, while also ignoring that co-retrieved memories are often scattered on disk. Akashic’s answer is MemAttention: compact one fixed-size chunk at a time, then use the same model-driven semantic match to reconcile that chunk with a few related prior chunks and to retrieve only the top relevant chunks at inference. A co-designed memory manager then rewrites frequently co-used chunks into the same blocks and reclaims stale copies out of place. On dialogue, software-engineering, browsing, and web-agent workloads across three model sizes, the system sits on the quality–efficiency frontier against full-context serving and prior memory systems, raising accuracy while raising throughput and sustainable concurrent load.

Core claim

Akashic shows that long-horizon agent memory can be both more accurate and cheaper to serve when construction is bounded to chunks, cross-chunk evidence is recovered by model-driven joint reconciliation rather than full-history rewrite, and physical layout co-locates chunks that are likely to be retrieved together. Under that design the system improves task quality and serving metrics simultaneously against whole-context, segment-level, hierarchical, and reflective memory baselines.

What carries the argument

MemAttention: gate-triggered compaction of one bounded active chunk, followed by model selection of a small set of related prior chunks for joint update/delete, the same model-driven top-p selection for query-time retrieval, and an out-of-place, affinity-aware block manager that co-locates co-accessed chunks and garbage-collects high-invalid-ratio blocks.

Load-bearing premise

The design assumes that using the model itself to match chunk metadata for joint compaction, retrieval, and physical co-location is accurate and cheap enough that the extra inference and rewrite work pays for itself in quality and latency.

What would settle it

On the same four workloads and models, disable model-driven joint reconciliation and co-location (or replace them with dense-embedding matching and append-only layout) and check whether the claimed accuracy, throughput, and concurrent-rate gains over the strongest baselines disappear or reverse once extra model calls and rewrite cost are counted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agent memory need not choose between quality and serving cost: bounded chunk maintenance plus cross-chunk recovery can raise both.
  • Semantic relevance alone is insufficient for serving; co-locating co-retrieved chunks reduces blocks touched and cold retrieval latency under load.
  • Fixed global summarization policies are a poor match for workloads whose information density varies across datasets and within a trajectory.
  • Long-horizon serving stacks can keep the GPU decoding path unchanged and still gain by inserting memory retrieval, write-back, and layout maintenance on the control path.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If model-driven affinity is reliable, the same co-location idea could apply to other session-scoped stores beyond vector chunk tables, such as tool traces or multi-agent shared scratchpads.
  • The locality-gap framing suggests agent-memory benchmarks should report blocks-per-request and cold retrieval latency, not only F1 or success rate.
  • When context density is extremely high, the residual value of any compression may be small, so adaptive gates that skip compaction may matter as much as better summaries.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. Akashic is a memory system for long-horizon LLM agent serving built around MemAttention: context is maintained as bounded chunks (default 1024 tokens), each new chunk is reconciled via model-driven cross-chunk inference with a small set of related prior chunks (top-p=5), and inference-time retrieval reuses the same model-driven metadata matching. A Memory Manager then co-locates jointly accessed chunks out-of-place and garbage-collects blocks with invalid ratio ≥0.75 to reduce retrieval I/O. Implemented as control-path extensions to vLLM, Akashic is evaluated on LoCoMo, SWE-bench, BrowseComp, and WebArena with Qwen-8B, OPT-30B, and Llama-70B against Full-context, Mem0, MemGAS, MemGPT, and RMM. The paper reports Pareto gains of up to +10.2 accuracy points, 1.21× throughput, and 1.88× sustainable concurrent request rate over the strongest prior memory baselines, plus locality and storage-amplification measurements and ablations of whole-context, no-joint, and dense-only variants.

Significance. The problem is timely and well-motivated: agent workloads accumulate heterogeneous, non-uniform context, and existing memory systems either pay growing whole-history maintenance cost or fragment cross-chunk evidence, while treating storage locality as an opaque backend. The paper’s joint treatment of chunk-granular maintenance, unified model-driven relevance for reconciliation and retrieval, and affinity-aware physical placement is a coherent systems contribution. Strengths include multi-workload, multi-scale evaluation; explicit locality metrics (blocks/request, p95 retrieval latency) and space/write amplification; and sensitivity sweeps over τc, Bmem, τassoc, and θgc. If the end-to-end efficiency claims hold under full cost accounting of auxiliary model calls, the work would be a useful reference point for production agent memory stacks and for connecting agent memory to locality-aware storage design.

major comments (4)
  1. §4.1–4.3, Algorithms 1–2, and §5.1 leave the cost of model-driven matching under-specified relative to the efficiency half of the central claim. Every retrieval issues LLMSelect over candidate metadata; compaction uses JointCompact; relocation/GC use LLMCoLocate/LLMGroup. §5.1 further states that OPT-30B is used for relevance analysis over memory chunks, which may be a separate model from the served Qwen-8B/Llama-70B. Figures 8–9 report only aggregate tokens/s and s/token with no breakdown of GPU time or tokens spent on auxiliary matching versus main prefill/decode, and no statement whether OPT-30B latency/throughput is included in those aggregates or runs on the same H800s under the concurrency knees of Fig. 9. Because the abstract’s 1.21×/1.88× gains are end-to-end serving claims, the manuscript needs an explicit cost accounting (or a controlled experiment with matching disabled / repl
  2. §5.1 lists five baselines including MemGPT, but §5.2 and Figure 8 report only Full-context, Mem0, MemGAS, and RMM (legend and body text). Either MemGPT results should be added under the same protocol, or the baseline list and claims of comparison to hierarchical external memory should be revised. As written, the multi-baseline Pareto claim is incompletely supported for one of the named systems.
  3. §5.5 Dense-only ablation retains 97.2%–97.3% task quality and 98.7%–99.1% throughput versus full Akashic, while Whole-context hurts throughput more. This is useful, but it weakens the paper’s emphasis on model-driven semantic matching as essential to the efficiency story (§4.1, abstract). The manuscript should clarify what fraction of the reported 1.21×/1.88× gains is attributable to (i) bounded chunk maintenance alone, (ii) cross-chunk reconciliation, (iii) model-driven vs dense retrieval, and (iv) physical co-location (Fig. 10). Without that decomposition, the joint quality–efficiency narrative over-attributes gains to the more expensive model-driven path.
  4. The repeated claim of “hardware–software co-designed” memory placement (abstract, §1, §4.2, conclusion) overstates the contribution relative to the implementation. §4.2–4.3 describe software layout policies (out-of-place rewrite, affinity packing, GC at φ≥0.75) on commodity NVMe behind an unmodified vLLM GPU path; there is no hardware interface change, device-level scheduling, or PCIe/NVMe co-design. Reframe as software locality-aware storage management co-optimized with the memory API, or specify what is co-designed with hardware beyond using local SSD bandwidth numbers in Table 1.

Circularity Check

0 steps flagged

No significant circularity: empirical systems paper with external benchmarks and free design knobs, not a self-referential derivation.

full rationale

Akashic is an empirical LLM-serving systems paper. Its load-bearing claims are measured end-to-end quantities—task accuracy (QA F1, %Resolved, BrowseComp accuracy, WebArena success), throughput (token/s), sustainable request rate (req/s before latency knee), blocks/request, p95 retrieval latency, and space/write amplification—on external workloads (LoCoMo, SWE-bench, BrowseComp, WebArena) against independent baselines (Full-context, Mem0, MemGAS, MemGPT, RMM). These metrics are not defined by, nor algebraically forced by, the design parameters. Chunk gate τc=1024, top-p=5, φ(B)≥0.75, τassoc=0.7, and memory budget are free engineering knobs with sensitivity/ablation plots (§5.5, Figs. 11–12); the paper does not fit them to a subset of the reported accuracy/throughput numbers and then call the same numbers a prediction. MemAttention’s model-driven matching (LLMSelect / LLMCoLocate / LLMGroup) and out-of-place co-location are design mechanisms whose net benefit is tested, not derived from a uniqueness theorem or self-citation chain. The only self-reference (RedKnot [44], open-source merge plan) is implementation context and is not used to justify the Pareto claims. No equation reduces a claimed result to its own input by construction. Score 0 is the correct honest finding.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on standard LLM-serving assumptions plus several paper-chosen thresholds and the postulate that model-judged metadata affinity is a reliable co-retrieval signal. No new physical entities; the invented constructs are software mechanisms. Free parameters are the operational knobs that bound cost and locality.

free parameters (5)
  • chunk_gate_tokens (τc) = 1024
    Default 1024 tokens chosen empirically as quality/overhead balance; sensitivity in §5.5 shows peaks near this value.
  • top_p / selection width = 5
    Default p=5 for both cross-chunk reconciliation and retrieval, following Claude Code practice cited in §4.1.
  • gc_invalid_ratio (θgc / φ threshold) = 0.75 (compaction candidate); sensitivity default 0.3
    Blocks compacted when invalid ratio ≥0.75 (text) / default 0.3 in sensitivity figure; controls space vs write amplification.
  • association threshold τassoc = 0.7
    Default 0.7 for when co-location is aggressive enough; §5.5 shows latency knee near this value.
  • memory_budget_tokens (Bmem) = default near 1536 in sensitivity peak
    Retrieval token budget; quality saturates near default while larger budgets hurt throughput (§5.5).
axioms (4)
  • domain assumption Transformer prefill cost and middle-context degradation make full-history replay impractical for long agent trajectories.
    Background motivation in §2.1 citing Lost-in-the-Middle and long-context serving costs.
  • ad hoc to paper An LLM can perform useful semantic matching over chunk metadata keywords for both maintenance-time reconciliation and query-time retrieval without a separate dense scorer.
    Core of MemAttention Algorithms 1–2 and §4.1; inspired by Claude Code but not independently validated outside this system.
  • ad hoc to paper Observed joint materialization of chunks at inference time is a good predictor of future co-retrieval, so out-of-place co-location reduces |P(Rt)|.
    Locality-gap hypothesis §3.2 and Memory Manager §4.2.
  • domain assumption Out-of-place rewrite with tombstones and background compaction (LSM/log-structured style) is an acceptable storage model for session-scoped agent memory.
    Explicit analogy to LSM-trees/Bigtable/LFS in §4.1–4.2.
invented entities (2)
  • MemAttention no independent evidence
    purpose: Chunk-gated compaction with model-driven cross-chunk reconciliation and unified model-driven retrieval.
    Named core algorithm of the paper; software mechanism, not a physical entity. Independent evidence is only the paper’s own ablations.
  • Akashic Memory Manager (association-aware relocation + GC) no independent evidence
    purpose: Co-locate co-retrieved chunks and reclaim invalid blocks to close the locality gap.
    Systems component introduced to turn semantic affinity into physical layout; evidence is internal microbenchmarks in §5.4.

pith-pipeline@v1.1.0-grok45 · 25422 in / 3566 out tokens · 40130 ms · 2026-07-11T03:20:04.658915+00:00 · methodology

0 comments
read the original abstract

Recent LLM-based agent systems continuously accumulate context across multi-turn interactions, tool invocations, and cross-session workflows. Replaying the full history for every request quickly becomes impractical: long contexts increase prefill cost, may exceed context limits, and often bury task-relevant evidence in irrelevant content, degrading both serving efficiency and output quality. We propose Akashic, a low-overhead memory system built around MemAttention, which organizes context into bounded chunks and models semantic relationships across chunks, preserving cross-chunk evidence without repeatedly rewriting the full history. Akashic further applies hardware-software co-designed memory placement to co-locate likely co-retrieved chunks, reducing retrieval fragmentation and I/O overhead. Across four representative workloads and three model sizes, Akashic improves task accuracy by up to 10.2 points, throughput by up to 1.21x, and sustainable request rate by up to 1.88x over strong prior memory baselines.

Figures

Figures reproduced from arXiv: 2607.05708 by Chenchen Hong, Chentao Wu, HuaYi Jin, Junhao Hu, RuoZhou He, Yang Liu, Yifei Liu, Yunfei Gu, ZhaoKai Luo, Zhiyong Wang.

Figure 1
Figure 1. Figure 1: (a) Existing memory designs occupy different points on the task-quality / serving-efficiency trade-off, whereas Akashic targets the high-quality, high-efficiency regime. (b) On BrowseComp with OPT-30B, Akashic im￾proves both accuracy and throughput over all baselines, out￾performing the strongest prior method by about 2.0 points in accuracy and about 1.35× in throughput. as MemGAS summarize smaller units i… view at source ↗
Figure 2
Figure 2. Figure 2: Context information density varies substantially across datasets and over time. (a) Compared with LoCoMo, SWE-Bench exhibits a higher retention ratio. (b) BrowseC￾omp and WebArena show bursty and alternating retention ratio within trajectories. beyond fixed context limits [25]. Industrial and open-source memory layers such as Mem0 similarly emphasize compress￾ing history into compact memory representations… view at source ↗
Figure 4
Figure 4. Figure 4: Throughput on LoCoMo as concurrency increases. Although memory-augmented inference consistently under￾performs standard inference, its throughput gradually ap￾proaches the no-memory baseline at larger batch sizes. context into the LLM for inference. The resulting memory is then stored in a database, and relevant memory context is retrieved and loaded during the next actual inference. We evaluate SWE-Bench,… view at source ↗
Figure 5
Figure 5. Figure 5: Akashic system overview. Algorithm 1 MemAttention: Incremental Chunk Com￾paction and Model-Driven Retrieval 1: Input: new turn 𝑥𝑡 , active chunk𝐶𝑎, chunk table T [𝑢] [𝑠], gate 𝜏𝑐 , top-𝑝 (𝑝=5) 2: Append 𝑥𝑡 to 𝐶𝑎 ⊲ accumulate fresh context 3: if |𝐶𝑎 | ≥ 𝜏𝑐 then 4: 𝑚 ← Compress(𝐶𝑎) ⊲ compact one bounded chunk 5: 𝐾𝑚 ← BuildMeta(𝑚) ⊲ extract keywords / representative metadata 6: M ← {(cid𝑖 , 𝐾𝑖) | 𝑐𝑖 ∈ T [𝑢] [… view at source ↗
Figure 6
Figure 6. Figure 6: Workflow of MemAttention: each incoming chunk context is incrementally compacted into a structured mem￾ory record with user- and session-scoped metadata, and the most relevant prior chunk memories are retrieved to aug￾ment the next-turn inference. user_id and session_id, respectively, and T [𝑢] [𝑠] denotes the chunk table scoped to that user-session namespace. The active chunk 𝐶𝑎 accumulates newly arrived … view at source ↗
Figure 7
Figure 7. Figure 7: Overview of the Memory Compaction workflow. which allows the system to refine the new chunk and to up￾date or delete stale prior memories when the new chunk con￾tains corrections, refinements, or superseding information. The reconciled chunk is then written back to the chunk table, and a new active chunk is started. Lines 14–20 implement inference-time retrieval. The current context is converted into query… view at source ↗
Figure 8
Figure 8. Figure 8: Throughput–accuracy trade-off under single-request inference (batch size = 1) across three models and three workloads. 5.3 Robustness under Heterogeneous Density and Concurrency We next evaluate Akashic under concurrent serving by in￾creasing the offered request rate and measuring average latency per generated token (s/token). As shown in [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Akashic maintains stable performance as concurrency increases, across models of different scales and diverse workloads. Blocks / request atency(ms) Physical bytes/ live bytes otal bytes written / new live bytes 40 38 34 30 30 27 25 22 21 22 20 18 10 0 LoCoMo WebArena SWE-Bench (b) p95 retrieval latency 2.0 1.82 1.5 1.31 1.34 1.05 1.0 0.5 0.0 No-Reloc. No-GC Agg.-Reloc. Akashic 8 6.8 6 5.9 5.1 4.6 4.1 4 3.4… view at source ↗
Figure 10
Figure 10. Figure 10: Compared with append-only and semantic-only layouts, Akashic reduces the number of blocks touched per request and lowers p95 cold-cache retrieval latency by co-locating jointly accessed chunks. Meanwhile, selective out-of-place relocation with background garbage collection keeps both space amplification and write amplification mod￾erate, showing that improved locality does not come at ex￾cessive storage o… view at source ↗
Figure 12
Figure 12. Figure 12: Sensitivity of Akashic’s storage manager. (a) Sen￾sitivity of p95 retrieval latency to the association threshold 𝜏𝑎𝑠𝑠𝑜𝑐 . (b) Sensitivity of space and write amplification to the GC invalid-ratio threshold 𝜃𝑔𝑐 . relevance signals, whereas Akashic’s model-driven relevance matching further improves selection quality by identifying semantically related chunks more accurately during both reconciliation and ret… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 53 canonical work pages · 16 internal anchors

  1. [1]

    LangChain API Reference: ConversationSummaryMem- ory.https://api.python.langchain.com/en/v0.0.354/memory/langchain

    2024. LangChain API Reference: ConversationSummaryMem- ory.https://api.python.langchain.com/en/v0.0.354/memory/langchain. 12 memory.summary.ConversationSummaryMemory.html. Accessed 2026-03-04

    work page 2024

  2. [2]

    LlamaIndex Documentation: Memory.https://docs.llamaindex

    2026. LlamaIndex Documentation: Memory.https://docs.llamaindex. ai/en/stable/api_reference/memory/memory/. Accessed 2026-03-04

    work page 2026

  3. [3]

    mem0: Universal Memory Layer for AI Agents (GitHub).https: //github.com/mem0ai/mem0

    2026. mem0: Universal Memory Layer for AI Agents (GitHub).https: //github.com/mem0ai/mem0. Accessed 2026-03-04

    work page 2026

  4. [4]

    Sumin An, Junyoung Sung, Wonpyo Park, Chanjun Park, and Paul Hongsuck Seo. 2025. LCIRC: A Recurrent Compression Ap- proach for Efficient Long-form Context and Query Dependent Mod- eling in LLMs. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Lin- guistics: Human Language Technologies (Volum...

    work page doi:10.18653/v1/2025.naacl-long.524 2025

  5. [5]

    Anthropic. 2025. Claude Code: Best Practices for Agentic Coding.https: //www.anthropic.com/engineering/claude-code-best-practices. Ac- cessed 2026-03-04

    work page 2025

  6. [6]

    Hsieh, Deborah A

    Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. 2006. Bigtable: A Distributed Storage System for Structured Data. In7th USENIX Symposium on Operating Systems Design and Implementation (OSDI 06). 205– 218.https://www.usenix.org/conference/osdi-06/bigtable-distr...

    work page 2006

  7. [7]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating Large Language Models Trained on Code.arXiv preprint arXiv:2107.03374(2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  8. [8]

    Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. 2021. SPANN: Highly-efficient Billion-scale Approximate Nearest Neighbor Search. InAdvances in Neural Information Processing Systems, Vol. 34. 5199– 5212.https://proceedings.neurips.cc/paper_files/paper/2021/file/ 299dc35e747eb77177d9cea10a802da2-Paper.pdf

    work page 2021

  9. [9]

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413https://arxiv.org/ abs/2504.19413

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    ChinaSiro. 2026. claude-code-sourcemap.https://github.com/ ChinaSiro/claude-code-sourcemap. GitHub repository; unofficial reconstruction from the public npm package and source-map analysis; accessed 2026-04-02

    work page 2026

  11. [11]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman

    work page

  12. [12]

    Training Verifiers to Solve Math Word Problems.arXiv preprint arXiv:2110.14168(2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  13. [13]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO- Awareness.Advances in Neural Information Processing Systems35 (2022), 16344–16359

    work page 2022

  14. [14]

    Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M. Zhang. 2023. Large Language Models for Software Engineering: Survey and Open Problems.arXiv preprint arXiv:2310.03533(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  15. [15]

    2022.https://github.com/features/copilot

    Github. 2022.https://github.com/features/copilot

    work page 2022

  16. [16]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Aman Pandey, Abhishek Kadian, Ahmed Al-Dahle, Aiesha Letman, Anirudh Mathur, Angelica Schelten, Angela Yang, et al . 2024. The Llama 3 Herd of Models.arXiv preprint arXiv:2407.21783(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  17. [17]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu

    work page

  18. [18]

    InProceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.)

    LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. InProceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguis- tics, Singapore, 13358–13376.https://doi.org/10.18653/v1/2023.emnlp- main.825

    work page doi:10.18653/v1/2023.emnlp- 2023

  19. [19]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?. InThe Twelfth In- ternational Conference on Learning Representations.https://openreview. net/forum?id=VTF8yNQM66

    work page 2024

  20. [20]

    Gonzalez, Hao Zhang, and Ion Sto- ica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Sto- ica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the 29th Symposium on Operating Systems Principles. 611–626.https://doi.org/10.1145/ 3600006.3613165

    work page arXiv 2023

  21. [21]

    Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Com- pressing Context to Enhance Inference Efficiency of Large Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Ka- lika Bali (Eds.). Association for Computational Linguistics, Singapore, 6342–6353.https://doi....

    work page doi:10.18653/v1/2023.emnlp-main.391 2023

  22. [22]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Mid- dle: How Language Models Use Long Contexts.Transactions of the Association for Computational Linguistics12 (2024), 157–173.https: //doi.org/10.1162/tacl_a_00638

    work page doi:10.1162/tacl_a_00638 2024

  23. [23]

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating Very Long- Term Conversational Memory of LLM Agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).https://aclanthology.org/2024.acl-long.747/

    work page 2024

  24. [24]

    Patrick O’Neil, Edward Cheng, Dieter Gawlick, and Elizabeth O’Neil

    work page

  25. [25]

    The Log-Structured Merge-Tree (LSM-Tree).Acta Informatica 33, 4 (1996), 351–385.https://doi.org/10.1007/s002360050048

    work page doi:10.1007/s002360050048 1996

  26. [26]

    OpenAI. [n. d.]. OpenAI Model Docs: GPT-4.1.https://developers. openai.com/api/docs/models/gpt-4.1. Accessed 2026-03-04

    work page 2026

  27. [27]

    OpenAI. 2025. Introducing GPT-4.1 in the API.https://openai.com/ index/gpt-4-1/. Accessed 2026-03-04

    work page 2025

  28. [28]

    MemGPT: Towards LLMs as Operating Systems

    Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2023. MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560https://arxiv.org/abs/2310.08560

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  29. [29]

    Vicky Zhao, Lili Qiu, and Jianfeng Gao

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Jianfeng Gao. 2025. SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents. InThe Thirteenth International Conference on Learning Representations.https://openreview.net/forum? id=xKDZAW0He3

    work page 2025

  30. [30]

    Ousterhout

    Mendel Rosenblum and John K. Ousterhout. 1992. The Design and Implementation of a Log-Structured File System.ACM Transactions on Computer Systems10, 1 (Feb. 1992), 26–52.https://doi.org/10.1145/ 146941.146943

    work page arXiv 1992

  31. [31]

    SGLang Team. 2025. SGLang 0.4.6.post1.https://pypi.org/project/ sglang/0.4.6.post1/. PyPI release. Official GitHub repository:https: //github.com/sgl-project/sglang

    work page 2025

  32. [32]

    Uri Shaham, Maor Ivgi, Avia Efrat, Jonathan Berant, and Omer Levy

    work page

  33. [33]

    InFindings of the Association for Computational Linguistics: EMNLP.https://doi.org/10.18653/v1/2023.findings-emnlp.536

    ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Under- standing. InFindings of the Association for Computational Linguistics: EMNLP.https://doi.org/10.18653/v1/2023.findings-emnlp.536

    work page doi:10.18653/v1/2023.findings-emnlp.536 2023

  34. [34]

    Joobo Shim, Jaewon Oh, Hongchan Roh, Jaeyoung Do, and Sang- Won Lee. 2025. Turbocharging Vector Databases using Modern SSDs.Proceedings of the VLDB Endowment18, 11 (2025), 4710–4722. https://doi.org/10.14778/3749646.3749724 13

    work page doi:10.14778/3749646.3749724 2025

  35. [35]

    Aditi Singh, Suhas Jayaram Subramanya, Ravishankar Krishnaswamy, and Harsha Vardhan Simhadri. 2021. FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search. arXiv:2105.09613https://arxiv.org/abs/2105.09613

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  36. [36]

    Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister

    Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister. 2025. In Prospect and Retrospect: Reflective Memory Management for Long- term Personalized Dialogue Agents. InProceedings of the 63rd Annual Meeting of the Association f...

    work page 2025

  37. [37]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. At- tention Is All You Need.Advances in Neural Information Processing Systems30 (2017)

    work page 2017

  38. [38]

    vLLM Team. 2026. vLLM (release v0.10.0).https://github.com/vllm- project/vllm/tree/releases/v0.10.0. GitHub repository, release branch releases/v0.10.0, accessed 2026-03-31

    work page 2026

  39. [39]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voy- ager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291https://arxiv.org/abs/2305.16291

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  40. [40]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhenhua Wei, and Ji-Rong Wen. 2023. A Survey on Large Language Model based Autonomous Agents. arXiv:2308.11432https: //arxiv.org/abs/2308.11432

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  41. [41]

    Mengzhao Wang, Weizhi Xu, Xiaomeng Yi, Songlin Wu, Zhangyang Peng, Xiangyu Ke, Yunjun Gao, Xiaoliang Xu, Rentong Guo, and Charles Xie. 2024. Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment.Proceedings of the ACM on Management of Data2, 1 (2024), 1–27.https://doi.org/10.1145/3639269

    work page doi:10.1145/3639269 2024

  42. [42]

    Qingyue Wang, Yanhe Fu, Yanan Cao, Shuai Wang, Zhiliang Tian, and Liang Ding. 2023. Recursively Summarizing Enables Long- Term Dialogue Memory in Large Language Models.arXiv preprint arXiv:2308.15022(2023).https://doi.org/10.48550/arXiv.2308.15022

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2308.15022 2023

  43. [43]

    Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. 2025. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. arXiv:2504.12516https://arxiv.org/ abs/2504.12516

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  44. [44]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of- Thought Prompting Elicits Reasoning in Large Language Models.Ad- vances in Neural Information Processing Systems35 (2022)

    work page 2022

  45. [45]

    Derong Xu, Yi Wen, Pengyue Jia, Yingyi Zhang, Wenlin Zhang, Yichao Wang, Huifeng Guo, Ruiming Tang, Xiangyu Zhao, Enhong Chen, and Tong Xu. 2025. Towards Multi-Granularity Memory Association and Selection for Long-Term Conversational Agents. arXiv:2505.19549 https://arxiv.org/abs/2505.19549

    work page arXiv 2025

  46. [46]

    Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-MEM: Agentic Memory for LLM Agents. arXiv:2502.12110https://arxiv.org/abs/2502.12110

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  47. [47]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  48. [48]

    Liu Yang, Luo ZhaoKai, Jin HuaYi, Wang ZhiYong, He RuoZhou, Wang BoYu, Chen Guanjie, and Hu. Junhao. 2026. RedKnot: Efficient Long- Context LLM Serving with Head-Aware KV Reuse and SegPagedAt- tention. arXiv:2606.06256https://arxiv.org/abs/2606.06256arXiv preprint

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  49. [49]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629https://arxiv.org/abs/ 2210.03629

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  50. [50]

    Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A Distributed Serving System for Transformer-Based Generative Models. In16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 521–538. https://www.usenix.org/conference/osdi22/presentation/yu

    work page 2022

  51. [51]

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. OPT: Open Pre-trained Transformer Language Models. arXiv preprint arXiv:2205.01068(2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  52. [52]

    Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, and Zhangyang Wang. 2024. Found in the Middle: How Language Models Use Long Contexts Better via Plug-and- Play Positional Encoding. InAdvances in Neural Information Processing Systems (NeurIPS)

    work page 2024

  53. [53]

    Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig

    Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. WebArena: A Realistic Web Environ- ment for Building Autonomous Agents. InThe Twelfth International Conference on Learning Representations.https://openreview.net/forum? id=oKn9c6ytLx 14

    work page 2024