Pith. sign in

REVIEW 2 major objections 8 minor 158 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Four kinds of LLM memory compression are one rate-distortion problem

2026-07-10 01:18 UTC pith:KH7BNDRO

load-bearing objection Unifies four disjoint memory-compaction literatures under one rate-distortion objective with a layer-agnostic Fano bound; the cross-layer mechanism transfer and benchmark proposal are the real contributions, but the one agent-layer-specific prediction is not supported by the paper's own experiment. the 2 major comments →

arxiv 2607.08032 v1 pith:KH7BNDRO submitted 2026-07-09 cs.LG

What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents

Ashwin Gerard Colaco , Nada Lahjouji This is my paper
classification cs.LG
keywords whatmemoryacrossagentscompactionagentattentionbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This survey argues that four largely disconnected research communities—KV-cache compression, prompt pruning, architectural state bounding, and agent memory consolidation—are all solving the same problem: a rate-distortion tradeoff where context-derived information must be retained or discarded under a memory budget to preserve downstream task accuracy. The authors formalize this as a single optimization objective (Eq. 1) and derive a layer-agnostic lower bound (Eq. 2) from the data-processing inequality and Fano's inequality. The bound shows that below a task's information requirement I*(Q), every layer must err at a rate governed by the same expression, that query-agnostic compaction pays a penalty of H(Q) bits for not knowing the query in advance, and that tasks whose answers carry many bits compress poorly while low-entropy tasks compress well. The survey then uses this lens to build a seven-axis taxonomy classifying roughly seventy methods uniformly, to transfer specific mechanisms between layers that have never been connected (e.g., an agent forgetting curve becomes a KV eviction prior; a no-eviction retrieval discipline becomes an agent-memory design), and to propose a benchmark that places all layers on one budget axis while measuring error accumulation under repeated compaction.

Core claim

The central discovery is that the same failure mode recurs at every layer: methods that fix what they keep before the query arrives and cannot reverse the decision will drop information the query later needs. The bound makes this precise—query-agnostic compaction pays H(Q) bits, and repeated irreversible summarization compounds error super-linearly because errors both accumulate and self-reinforce, whereas reversible retrieval-backed memory stays flat. The authors confirm both predictions in a small reference experiment: KV eviction accuracy collapses once the budget drops below the needle's information content, and an irreversible summarization operator loses roughly half its facts under a

What carries the argument

The compaction objective (Eq. 1) minimizes expected task loss subject to a memory budget, with the information-bottleneck form maximizing I(Z;Y|Q) subject to I(Z;H) ≤ B. The lower bound (Eq. 2) follows because Y−H−Z forms a Markov chain given Q for query-agnostic operators, so the data-processing inequality gives I(Y;Ŷ|Q) ≤ min(I*(Q), B), and Fano's inequality then bounds the error probability. Three properties—reversibility (P-rev), query-conditioning (P-q), and fidelity profile (P-fid)—are promoted from implicit consequences of the objective to first-class design axes that separate methods that fail from those that do not.

Load-bearing premise

The claim that the four layers share one problem depends on the Markov chain Y−H−Z given Q holding for all layers. For agent memory, the compaction operator is itself a fallible LLM call whose errors differ structurally from arithmetic eviction decisions, and the usage operator involves retrieval that may fail for reasons orthogonal to the rate-distortion tradeoff. The bound is derived under the clean Markov assumption, which the paper acknowledges is an approximation at the

What would settle it

If a query-agnostic irreversible compaction method (e.g., H2O eviction or fixed-schedule summarization) were shown to match a query-conditioned reversible method (e.g., Quest retrieval or archival paging) at equal budget on high-I*(Q) tasks under repeated compaction, the bound's predicted penalty of H(Q) and the super-linear error accumulation would be falsified. The reference experiment tests this on a small model; replication at scale is the key falsifier.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the bound holds, a KV evictor keeping 25% of tokens and a quantizer storing every token at 4 bits can be directly compared on a shared bytes-per-token axis, and an agent's summary can be placed on the same axis.
  • If query-agnostic compaction pays H(Q), then methods like LongLLMLingua and Quest that condition on the query should shift the accuracy-versus-budget curve rightward by the mutual-information gap, a testable prediction.
  • If repeated irreversible summarization compounds error super-linearly, then agent benchmarks that test only single-turn recall are missing the regime where compaction actually fails, and reversible retrieval-backed memory should dominate at equal budget over long horizons.
  • If the four layers share one objective, then a mechanism refined at one tier (e.g., Ada-KV's output-error bound) becomes a candidate design for another tier (e.g., a stop rule for agent summarization).

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The bound's cleanest form assumes the Markov chain Y−H−Z given Q, which is natural for KV eviction but strained for agent memory where Z is an LLM-generated summary and U is a retrieval step that may fail for reasons orthogonal to the rate-distortion tradeoff. The formal unification may therefore be tighter for the KV and architectural layers than for agent memory.
  • If I*(Q) is never measured directly but governs the achievable compression ratio, then the most consequential practical output would be a predictive scaling law that estimates I*(Q) from model size, context length, and task type—turning budget selection from empirical tuning into calculation.
  • The composition of orthogonal compression axes (quantization × low-rank × eviction) is largely uncharted; if their errors compound multiplicatively rather than additively, then stacking methods could be far more lossy than any single method's frontier suggests.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. This survey proposes that memory compaction across four distinct layers of the LLM stack—KV cache, prompt/context, architectural state, and agent memory—is a single rate–distortion problem. The authors formalize this with a unified objective (Eq. 1) and a layer-agnostic lower bound derived from the data-processing inequality and Fano's inequality (Eq. 2), from which they derive four falsifiable predictions. They build a seven-axis taxonomy classifying roughly seventy methods, propose a cross-layer benchmark (COMPACT-Bench), and run a small reference experiment on a commodity GPU. The paper also identifies a shared failure mode (query-agnostic, irreversible loss) and transfers specific mechanisms across layers (e.g., Ada-KV's error bound as a stop rule for agent summarization). The scope is ambitious and the unifying lens is a genuinely useful contribution.

Significance. The paper's primary significance lies in providing a shared formal yardstick for four research communities that have operated largely independently. The lower bound (Eq. 2) is a correct application of standard information-theoretic tools, and the identification of the H(Q) penalty for query-agnostic compaction is a clean, transferable insight. The seven-axis taxonomy and the master table (Table 6) are valuable reference artifacts. The COMPACT-Bench proposal, particularly its emphasis on a shared bytes-per-token budget axis and repeated-compaction measurement, addresses a real gap in the evaluation literature. The five design principles (P1–P5) are well-motivated. The reference experiment, while small-scale, demonstrates the methodology the survey advocates and provides falsifiable instances of the formalism.

major comments (2)
  1. §2, §7, §14.2 (Prediction 4 and Figure 6): Prediction 4 states that 'under repeated irreversible summarization, end-task error grows super-linearly in the number of compaction events.' This is the formalism's most distinctive agent-layer claim and the one directly tested in §14.2. However, Figure 6 does not show super-linear growth. The irreversible operator's recall stays between 0.33 and 0.56 across 5 to 25 compaction events—a roughly constant low level, not a curve that accelerates downward. The paper acknowledges the irreversible operator is 'weakest at the highest compaction frequency' but does not note that this mild degradation fails to match the super-linear prediction. If the error curve is approximately flat (just at a lower level than reversible), the self-reinforcing compounding mechanism is not operating as predicted, and the claim that repeated irreversible compaction has a
  2. §2, Eq. (2) (Markov assumption for agent memory): The lower bound in Eq. (2) is derived under the Markov chain Y−H−Z given Q. This is cleanest for KV cache and prompt compression where C is a deterministic arithmetic operation. For agent memory (§7), C is itself a fallible LLM call and U is a retrieval step that may fail for reasons orthogonal to the rate-distortion tradeoff. The paper acknowledges this in §10.4 ('Where the analogy breaks') but the formal bound in Eq. 2 is presented without qualification as holding 'for any compact memory Z, whether a KV cache, a gist vector, a recurrent state, or an agent's note store.' The authors should either (a) state explicitly the conditions under which the Markov assumption holds at the agent tier and where it breaks, or (b) soften the claim that Eq. (2) applies uniformly without additional assumptions at the agent tier.
minor comments (8)
  1. §4.1: The description of StreamingLLM's attention sinks could benefit from a forward reference to §12, where sinks are analyzed more deeply as artifacts of the softmax normalizer. Cross-referencing would strengthen the argument.
  2. Table 3: The 'Reduct.' column reports compression ratios from cited works with heterogeneous setups. While the caption notes they are 'not directly comparable,' adding a brief note on the baseline (e.g., full KV cache size) would help readers interpret these numbers.
  3. §5.2: The claim that 'a plain average-pooling baseline often rivals trained gisting on long inputs' is attributed to [29] but the mechanism for why this happens is only briefly mentioned. A sentence explaining why naive pooling captures most of the information when H is dense would contextualize this.
  4. §8: The distinction between 'natively trainable sparse attention' (NSA, MoBA) and 'calibrated offline' methods (MInference) is clear, but the claim that the former 'sidesteps the H(Q) penalty' could be stronger with a brief note on whether the learned router fully eliminates the penalty or merely reduces it.
  5. §12: The paper notes that 'none predicts the 80–93% KV reductions' observed in practice and calls for a 'predictive compression scaling law.' This is an important open problem, but the discussion could note whether existing scaling laws for model capacity (e.g., [83]) provide any partial leverage.
  6. §14.1, Figure 5: The y-axis label 'needle accuracy (%)' ranges from 0 to 100, but the text refers to accuracy as 1.00 for the full cache. Consistent use of either fraction or percentage would improve clarity.
  7. The paper uses 'rate(·)' to measure memory in layer-appropriate currencies (GPU bytes, tokens, state dimensions, store size). While this abstraction is central to the unification, a brief table mapping each layer to its specific rate(·) unit would aid readability.
  8. Reference [94] (Ran-Milo) is cited for the claim that attention sinks are 'provably necessary' for trigger-conditional tasks. The paper should note whether this result applies to all softmax transformers or a restricted family, as the strength of the claim varies.

Circularity Check

0 steps flagged

No circularity found; formalism derived from standard information theory with no self-citation chain

full rationale

The paper's central derivation chain is self-contained and non-circular. Eq. 1 is a standard rate-distortion objective applied to compaction, with the information-bottleneck form cited to Tishby et al. [109] (external). Eq. 2 is derived from the data-processing inequality and Fano's inequality under an explicitly stated Markov assumption (Y−H−Z given Q), both standard results. The four predictions follow logically: Prediction 1 (KV collapse) from Eq. 2 applied to concentrated I*(Q); Prediction 2 (query-aware shift by H(Q)) from consequence (iii); Prediction 3 (architectural cliff) from Eq. 2 with B=s; Prediction 4 (super-linear error growth) adds a self-reinforcing-error mechanism beyond the bound, but this is an additional hypothesis, not a circular restatement of inputs. No load-bearing step reduces to its inputs by construction. The authors cite no prior work of their own as load-bearing—the formalism rests on external results (Tishby, Fano, DPI, Haris & Onak [44], Wen et al. [117]). The reference experiments test external methods (SnapKV, StreamingLLM, TOVA, etc.), not the authors' own. The taxonomy classifies external methods along stated axes. The 'importance heuristics as distortion surrogates' framing is interpretive rather than derivational—it does not claim to prove that heuristics equal surrogates, only to read them that way. The skeptic's concern about Prediction 4's empirical fit (Figure 6 showing roughly flat rather than super-linear degradation) is a correctness risk, not circularity: the prediction is a falsifiable claim that the data may or may not confirm, which is the opposite of a circular result forced by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 3 invented entities

The formalism relies on standard information-theoretic axioms (data-processing inequality, Fano's inequality) applied under a domain assumption (Markov chain Y−H−Z given Q) that is clean for KV cache but approximate for agent memory. The super-linear compounding claim is ad hoc to the paper and only weakly tested. No free parameters are fitted to make the bound hold; I*(Q) is acknowledged as unmeasured. The invented entities (C_θ, U, COMPACT-Bench, taxonomy) are framework constructs whose value is tested only at toy scale.

free parameters (2)
  • Budget B = varied across experiments
    The memory budget B is the independent variable in the rate-distortion objective, not a fitted parameter. In the reference experiment it is swept across five values for each method.
  • Task-conditioned information content I*(Q) = not directly measured
    I*(Q) is defined as I(Y;H|Q) and is central to the bound, but the paper acknowledges it is never measured directly (§12, §15 item 4). It appears as a theoretical quantity governing the compression ratio but no method estimates it empirically.
axioms (5)
  • standard math Data-processing inequality: I(Y;Ŷ|Q) ≤ min(I*(Q), B) under the Markov chain Y−H−Z given Q
    Invoked in §2 to derive the lower bound (Eq. 2). Standard information theory.
  • standard math Fano's inequality: P_e ≥ (H(Y|Q) - B - 1) / log|Y|
    Invoked in §2 to bound error probability. Standard information theory.
  • domain assumption The Markov chain Y−H−Z given Q holds for query-agnostic compaction operators across all four layers
    Stated in §2. This is the load-bearing assumption: it is clean for KV cache (Z is a subset of H) but less clean for agent memory where Z is an LLM-generated summary and the relationship between H and Z is mediated by a fallible model.
  • domain assumption Importance heuristics (attention scores, perplexity, LLM-judged salience) are surrogates for the distortion D(θ)
    Asserted in §2 and used throughout §4-7. This is an interpretive claim rather than a proven one; the paper argues it by analogy rather than by showing the heuristics converge to D(θ).
  • ad hoc to paper Repeated irreversible compaction compounds error super-linearly
    Prediction (4) in §2. The super-linear claim is tested in §14.2 but the experiment shows a flat gap (Figure 6) rather than a super-linear curve, and no formal model of the compounding is derived.
invented entities (3)
  • Compaction operator C_θ and usage operator U no independent evidence
    purpose: Formalize the retain/discard decision and subsequent model computation as a rate-distortion problem
    These are formal constructs defined for the unification, not physical entities. Their value is in the framework they enable, tested indirectly via the reference experiment.
  • COMPACT-Bench no independent evidence
    purpose: A benchmark protocol placing KV, prompt, architectural, and agent compaction on one bytes-per-token axis with repeated-compaction measurement
    Proposed but not yet independently run by external groups. The reference experiment in §14 is a small-scale first version, not a full benchmark release.
  • Seven-axis taxonomy no independent evidence
    purpose: Classify ~70 compaction methods uniformly across granularity, lifecycle, fidelity, adaptivity, learnability, mechanism, and substrate
    An organizational framework. Its value is in enabling comparison and transfer, not in being a measurable entity.

pith-pipeline@v1.1.0-glm · 47933 in / 3445 out tokens · 361604 ms · 2026-07-10T01:18:22.605507+00:00 · methodology

0 comments
read the original abstract

Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions. Because none of this memory is free, four largely separate research communities have each learned to compact it. They evict or quantize the KV cache, prune or distill prompts, bound architectural state, and consolidate agent memory. We argue that these are instances of one problem: a rate--distortion decision about what context-derived information to retain versus discard, at what fidelity, under a resource budget, so as to preserve downstream task utility. We make this lens precise with a single compaction objective and a layer-agnostic lower bound, use it to build a seven-axis taxonomy that classifies methods from across the stack uniformly, and use it to transfer mechanisms between layers that have never been connected, from serving-stack KV management to agent long-term memory. Two patterns hold across the survey. At every layer the signal that decides what to keep is attention magnitude or recency, and it fails in the same way everywhere, by discarding, before the query is known and with no way to undo it, information the query later needs. And while compression is measured carefully on single-turn long context, the repeated compaction that agents actually perform is almost never measured, and no benchmark holds one budget axis across all the layers at once. We turn both observations into a benchmark proposal, a small reference experiment, and a set of compaction-aware design principles, and we map the open problems.

Figures

Figures reproduced from arXiv: 2607.08032 by Ashwin Gerard Colaco, Nada Lahjouji.

Figure 1
Figure 1. Figure 1: Map of the survey. The four areas of memory compaction act on different substrates and at different times, yet each [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The rate–distortion view of Section 2. Below the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Where compaction acts across the model and agent lifecycle, the lifecycle axis of Section 3. The same retain-versus [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The memory hierarchy of Section 10. Three tiers [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The accuracy–budget frontier on natural-filler nee [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Fact recall against the number of compaction events [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

158 extracted references · 158 canonical work pages · 134 internal anchors

  1. [1]

    Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference

    Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant J. Nair, Ilya Soloveychik, and Purushotham Kamath. 2024. Keyformer: KV Cache Re- duction through Key Tokens Selection for Efficient Generative Inference. arXiv:2403.09054 [cs.LG] https://arxiv.org/abs/2403.09054

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  2. [2]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski,...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  3. [3]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding. arXiv:2308.14508 [cs.CL] https://arxiv.org/abs/2308.14508

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  4. [4]

    Ivana Balažević, Yuge Shi, Pinelopi Papalampidi, Rahma Chaabouni, Skanda Koppula, and Olivier J. Hénaff. 2024. Memory Consolidation Enables Long- Context Video Understanding. arXiv:2402.05861 [cs.CV] https://arxiv.org/abs/ 2402.05861

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Olek- sandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstet- ter, and Sepp Hochreiter. 2024. xLSTM: Extended Long Short-Term Memory. arXiv:2405.04517 [cs.LG] https://arxiv.org/abs/2405.04517

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. 2024. Titans: Learning to Memorize at Test Time. arXiv:2501.00663 [cs.LG] https://arxiv.org/abs/2501. 00663

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Fe- ichtenhofer, and Judy Hoffman. 2023. Token Merging: Your ViT But Faster. arXiv:2210.09461 [cs.CV] https://arxiv.org/abs/2210.09461

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  8. [8]

    Improving language models by retrieving from trillions of tokens

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Ruther- ford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bog- dan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving,...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  9. [9]

    William Brandon, Mayank Mishra, Aniruddha Nrusimha, Rameswar Panda, and Jonathan Ragan Kelly. 2024. Reducing Transformer Key-Value Cache Size with Cross-Layer Attention. arXiv:2405.12981 [cs.LG] https://arxiv.org/abs/ 2405.12981 Colaco and Lahjouji

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  10. [10]

    Aydar Bulatov, Yuri Kuratov, and Mikhail S. Burtsev. 2022. Recurrent Memory Transformer. arXiv:2207.06881 [cs.CL] https://arxiv.org/abs/2207.06881

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  11. [11]

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao. 2025. PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling. arXiv:2406.02069 [cs.CL] https://arxiv.org/abs/2406.02069

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  12. [12]

    Palu: Compressing KV-Cache with Low-Rank Projection

    Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Chong-Yan Chen, Yu-Fang Hu, Pei-Shuo Wang, Ning-Chi Huang, Luis Ceze, Mohamed S. Abdelfattah, and Kai-Chiang Wu. 2024. Palu: Compressing KV-Cache with Low-Rank Projection. arXiv:2407.21118 [cs.AI] https://arxiv.org/abs/2407.21118

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  13. [13]

    xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction

    Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Xilai Dai, Huiqiang Jiang, Yucheng Li, Luis Ceze, Kai-Chiang Wu, and Mohamed S. Abdelfattah. 2026. xKV: Cross-Layer KV-Cache Compres- sion via Aligned Singular Vector Extraction. arXiv:2503.18893 [cs.CL] https: //arxiv.org/abs/2503.18893

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  14. [14]

    Vivek Chari, Guanghui Qin, and Benjamin Van Durme. 2025. KV-Distill: Nearly Lossless Learnable Context Compression for LLMs. arXiv:2503.10337 [cs.CL] https://arxiv.org/abs/2503.10337

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    Guoxin Chen, Zile Qiao, Xuanzhong Chen, Donglei Yu, Haotian Xu, Wayne Xin Zhao, Ruihua Song, Wenbiao Yin, Huifeng Yin, Liwen Zhang, Kuan Li, Min- peng Liao, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. 2026. IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling. arXiv:2511.07327 [cs.AI] https://arxiv.org/abs/2511.07327

    work page arXiv 2026

  16. [16]

    Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, and Chao Huang. 2025. SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator. arXiv:2412.12094 [cs.CL] https://arxiv.org/abs/2412.12094

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  17. [17]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024. An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models. arXiv:2403.06764 [cs.CV] https://arxiv.org/abs/2403.06764

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  18. [18]

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. 2024. Agent- Poison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases. arXiv:2407.12784 [cs.LG] https://arxiv.org/abs/2407.12784

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, and Dongyan Zhao. 2024. xRAG: Extreme Con- text Compression for Retrieval-augmented Generation with One Token. arXiv:2405.13792 [cs.CL] https://arxiv.org/abs/2405.13792

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  20. [20]

    Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. 2023. Adapting Language Models to Compress Contexts. arXiv:2305.14788 [cs.CL] https://arxiv.org/abs/2305.14788

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav

    work page

  22. [22]

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413 [cs.CL] https://arxiv.org/abs/2504.19413

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina, and Stéphane Clin- chant. 2025. Provence: efficient and robust context pruning for retrieval- augmented generation. arXiv:2501.16214 [cs.CL] https://arxiv.org/abs/2501. 16214

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. arXiv:1901.02860 [cs.LG] https://arxiv.org/abs/1901. 02860

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  25. [25]

    Payel Das, Subhajit Chaudhury, Elliot Nelson, Igor Melnyk, Sarath Swami- nathan, Sihui Dai, Aurélie Lozano, Georgios Kollias, Vijil Chenthamarak- shan, Jiří, Navrátil, Soham Dan, and Pin-Yu Chen. 2024. Larimar: Large Language Models with Episodic Memory Control. arXiv:2403.11901 [cs.LG] https://arxiv.org/abs/2403.11901

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  26. [26]

    Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando De Freitas, and Caglar Gulcehre. 2024. Griffin: Mixing Gated Linear Recurrences with Local Attention for Ef...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  27. [27]

    Enrique Queipo de Llano, Álvaro Arroyo, Federico Barbero, Xiaowen Dong, Michael Bronstein, Yann LeCun, and Ravid Shwartz-Ziv. 2026. Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin. arXiv:2510.06477 [cs.LG] https://arxiv.org/abs/2510.06477

    work page arXiv 2026

  28. [28]

    DeepSeek-AI. 2025. DeepSeek-V3.2-Exp: Boosting Long-Context Efficiency with DeepSeek Sparse Attention. Technical report. DeepSeek Sparse Attention (DSA)

    work page 2025

  29. [29]

    DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

    DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Cheng- gang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guant- ing Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Ji...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  30. [30]

    Chenlong Deng, Zhisong Zhang, Kelong Mao, Shuaiyi Li, Xinting Huang, Dong Yu, and Zhicheng Dou. 2024. A Silver Bullet or a Compromise for Full At- tention? A Comprehensive Study of Gist Token-based Context Compression. arXiv:2412.17483 [cs.CL] https://arxiv.org/abs/2412.17483

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. 2025. Memory Injection Attacks on LLM Agents via Query- Only Interaction. arXiv:2503.03704 [cs.LG] https://arxiv.org/abs/2503.03704

    work page arXiv 2025

  32. [32]

    Pengfei Du. 2026. Memory for Autonomous LLM Agents:Mechanisms, Evalua- tion, and Emerging Frontiers. arXiv:2603.07670 [cs.AI] https://arxiv.org/abs/ 2603.07670

    work page arXiv 2026

  33. [33]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130 [cs.CL] https://arxiv.org/abs/ 2404.16130

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  34. [34]

    Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, Will Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, and Christopher Re

    work page

  35. [35]

    Cartridges: Lightweight and general-purpose long context representations via self-study

    Cartridges: Lightweight and general-purpose long context representations via self-study. arXiv:2506.06266 [cs.CL] https://arxiv.org/abs/2506.06266

    work page internal anchor Pith review Pith/arXiv arXiv

  36. [36]

    Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference

    Yuan Feng, Junlin Lv, Yukun Cao, Xike Xie, and S. Kevin Zhou. 2025. Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference. arXiv:2407.11550 [cs.CL] https://arxiv.org/abs/2407.11550

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  37. [37]

    Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee, Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, and Jun Wang. 2025. Human- inspired Episodic Memory for Infinite Context LLMs. arXiv:2407.09450 [cs.AI] https://arxiv.org/abs/2407.09450

    work page arXiv 2025

  38. [38]

    Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, and Pengfei Zuo. 2024. Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention. arXiv:2403.19708 [cs.CL] https://arxiv.org/abs/2403.19708

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  39. [39]

    Yizhao Gao, Zhichen Zeng, Dayou Du, Shijie Cao, Peiyuan Zhou, Jiaxing Qi, Junjie Lai, Hayden Kwok-Hay So, Ting Cao, Fan Yang, and Mao Yang

    work page

  40. [40]

    SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs

    SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs. arXiv:2410.13276 [cs.CL] https://arxiv.org/abs/2410.13276

    work page internal anchor Pith review Pith/arXiv arXiv

  41. [41]

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. 2024. Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs. arXiv:2310.01801 [cs.CL] https://arxiv.org/abs/2310.01801

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  42. [42]

    Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. 2024. In-context Autoencoder for Context Compression in a Large Language Model. arXiv:2307.06945 [cs.CL] https://arxiv.org/abs/2307.06945

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  43. [43]

    Albert Gu and Tri Dao. 2024. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv:2312.00752 [cs.LG] https://arxiv.org/abs/2312. 00752

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  44. [44]

    Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. 2025. When Attention Sink Emerges in Language Models: An Empirical View. arXiv:2410.10781 [cs.CL] https://arxiv.org/abs/2410.10781

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  45. [45]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su

    work page

  46. [46]

    HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

    HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. arXiv:2405.14831 [cs.CL] https://arxiv.org/abs/2405.14831

    work page internal anchor Pith review Pith/arXiv arXiv

  47. [47]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. 2025. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models. arXiv:2502.14802 [cs.CL] https://arxiv.org/abs/2502.14802

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  48. [48]

    Themistoklis Haris and Krzysztof Onak. 2025. Compression Barriers for Au- toregressive Transformers. arXiv:2502.15955 [cs.DS] https://arxiv.org/abs/2502. 15955 What to Keep, What to Forget: A Rate–Distortion View of Memory Compaction in LLMs and Agents

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  49. [49]

    KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. 2025. KVQuant: To- wards 10 Million Context Length LLM Inference with KV Cache Quantization. arXiv:2401.18079 [cs.LG] https://arxiv.org/abs/2401.18079

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  50. [50]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. RULER: What’s the Real Context Size of Your Long-Context Language Models? arXiv:2404.06654 [cs.CL] https://arxiv.org/abs/2404.06654

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  51. [51]

    Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yanbin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, Zhenrong Cheng, Xuanbo Fan, Jiaxin Guo, Xinlei Yu, Zhenhong Zhou, Zewen Hu, Jiahao Huo, Junhao Wang, Yuwei Niu, Yu Wang, Zhe...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  52. [52]

    MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention

    Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. MInference 1.0: Accelerating Pre-filling for Long- Context LLMs via Dynamic Sparse Attention. arXiv:2407.02490 [cs.CL] https: //arxiv.org/abs/2407.02490

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  53. [53]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. arXiv:2310.05736 [cs.CL] https://arxiv.org/abs/2310.05736

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  54. [54]

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. arXiv:2310.06839 [cs.CL] https://arxiv.org/abs/2310.06839

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  55. [55]

    Greg Kamradt. 2023. Needle In A Haystack – Pressure Testing LLMs. https: //github.com/gkamradt/LLMTest_NeedleInAHaystack. Software

    work page 2023

  56. [56]

    Hao Kang, Qingru Zhang, Souvik Kundu, Geonhwa Jeong, Zaoxing Liu, Tushar Krishna, and Tuo Zhao. 2024. GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM. arXiv:2403.05527 [cs.LG] https://arxiv.org/abs/2403.05527

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  57. [57]

    ACON: Optimizing Context Compression for Long-horizon LLM Agents

    Minki Kang, Wei-Ning Chen, Dongge Han, Huseyin A. Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan. 2026. ACON: Optimizing Context Compression for Long-horizon LLM Agents. arXiv:2510.00615 [cs.AI] https://arxiv.org/abs/2510.00615

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  58. [58]

    Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. 2024. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack. arXiv:2406.10149 [cs.CL] https://arxiv.org/abs/2406.10149

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  59. [59]

    Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. 2024. In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss. arXiv:2402.10790 [cs.CL] https://arxiv.org/ abs/2402.10790

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  60. [60]

    Efficient Memory Management for Large Language Model Serving with PagedAttention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv:2309.06180 [cs.LG] https://arxiv.org/abs/2309.06180

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  61. [61]

    Wonbeom Lee, Jungi Lee, Junghwan Seo, and Jaewoong Sim. 2024. InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management. arXiv:2406.19707 [cs.LG] https://arxiv.org/abs/2406.19707

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  62. [62]

    Haoyang Li, Yiming Li, Anxin Tian, Tianhao Tang, Zhanchao Xu, Xuejia Chen, Nicole Hu, Wei Dong, Qing Li, and Lei Chen. 2025. A Survey on Large Language Model Acceleration based on KV Cache Management. arXiv:2412.19442 [cs.AI] https://arxiv.org/abs/2412.19442

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  63. [63]

    Yucheng Li. 2023. Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering. arXiv:2304.12102 [cs.CL] https://arxiv.org/abs/2304.12102

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  64. [64]

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. SnapKV: LLM Knows What You are Looking for Before Generation. arXiv:2404.14469 [cs.CL] https://arxiv.org/abs/2404.14469

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  65. [65]

    SCBench: A KV Cache-Centric Analysis of Long-Context Methods

    Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, and Lili Qiu. 2025. SCBench: A KV Cache-Centric Analysis of Long-Context Methods. arXiv:2412.10319 [cs.CL] https://arxiv.org/abs/2412.10319

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  66. [66]

    Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. 2024. Prompt Com- pression for Large Language Models: A Survey. arXiv:2410.12388 [cs.CL] https://arxiv.org/abs/2410.12388

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  67. [67]

    Zongqian Li, Yixuan Su, and Nigel Collier. 2024. 500xCompressor: Generalized Prompt Compression for Large Language Models. arXiv:2408.03094 [cs.CL] https://arxiv.org/abs/2408.03094

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  68. [68]

    Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedi- gos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, Omri Abend, Raz Alon, Tomer Asida, Amir Bergman, Roman Glozman, Michael Gokhman, Avashalom Manevich, Nir Ratner, Noam Rozen, Erez Shwartz, Mor Zusman, and Yoav Shoham. 2024. Jamba: A Hybrid Transformer-Mamb...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  69. [69]

    Sleep-time Compute: Beyond Inference Scaling at Test-time

    Kevin Lin, Charlie Snell, Yu Wang, Charles Packer, Sarah Wooders, Ion Stoica, and Joseph E. Gonzalez. 2025. Sleep-time Compute: Beyond Inference Scaling at Test-time. arXiv:2504.13171 [cs.AI] https://arxiv.org/abs/2504.13171

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  70. [70]

    Zehao Lin, Xixuan Hao, Renyu Fu, Shaobo Cui, Kai Chen, Chunyu Li, Zhiyu Li, and Feiyu Xiong. 2026. A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle. arXiv:2604.16548 [cs.CR] https://arxiv.org/abs/2604.16548

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  71. [71]

    Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. 2024. MiniCache: KV Cache Compression in Depth Dimension for Large Language Models. arXiv:2405.14366 [cs.CL] https://arxiv.org/abs/2405. 14366

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  72. [72]

    Lost in the Middle: How Language Models Use Long Contexts

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172 [cs.CL] https://arxiv.org/abs/2307.03172

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  73. [73]

    Yanyu Liu, Jingying Fu, Sixiang Liu, Yitian Zou, You Fu, Jiehan Zhou, and Shouhua Zhang. 2025. KV Cache Compression for Inference Efficiency in LLMs: A Review. arXiv:2508.06297 [cs.DC] https://arxiv.org/abs/2508.06297

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  74. [74]

    Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, and Junchen Jiang. 2024. CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving. arXiv:2310.07240 [cs.NI] https://arxiv.org/abs/2310.07240

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  75. [75]

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. 2023. Scissorhands: Ex- ploiting the Persistence of Importance Hypothesis for LLM KV Cache Compres- sion at Test Time. arXiv:2305.17118 [cs.LG] https://arxiv.org/abs/2305.17118

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  76. [76]

    Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. 2024. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache. arXiv:2402.02750 [cs.CL] doi:10.13140/RG.2.2. 28167.37282

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.13140/rg.2.2 2024

  77. [77]

    MoBA: Mixture of Block Attention for Long-Context LLMs

    Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, Zhiqi Huang, Huan Yuan, Suting Xu, Xinran Xu, Guokun Lai, Yanru Chen, Huabin Zheng, Junjie Yan, Jianlin Su, Yuxin Wu, Neo Y. Zhang, Zhilin Yang, Xinyu Zhou, Mingxing Zhang, and Jiezhong Qiu. 2025. MoBA: Mixture of Block Attention for Long...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  78. [78]

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating Very Long-Term Conversational Memory of LLM Agents. arXiv:2402.17753 [cs.CL] https://arxiv.org/abs/2402. 17753

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  79. [79]

    Fanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, and Muhan Zhang. 2025. TransMLA: Multi-Head Latent Attention Is All You Need. arXiv:2502.07864 [cs.LG] https://arxiv.org/abs/2502.07864

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  80. [80]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and Editing Factual Associations in GPT. arXiv:2202.05262 [cs.CL] https: //arxiv.org/abs/2202.05262

    work page internal anchor Pith review Pith/arXiv arXiv 2022

Showing first 80 references.