Pith. sign in

REVIEW 3 major objections 6 minor 44 references

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RestoreKV shows that a small, context-conditioned 'restore cache' added to any query-agnostic KV eviction method recovers most of the accuracy lost under aggressive compression.

desk verdict A well-engineered, budget-matched restore cache gives large gains at 4-11K contexts; the fixed-capacity mechanism looks weaker at 100K, and missing code/data hold it back from full reproducibility. read the letter →

arxiv 2608.01247 v1 pith:ZQP3U7PX submitted 2026-08-02 cs.CL cs.LG

Changwoo Baek , Seungjun Shin , Kyeongbo Kong This is my paper · ORCID
classification cs.CLcs.LG
keywords KVcacheevictionquery-agnosticcompressionlearnedrestoretokensLoRAadaptationself-distillationlong-contextLLMinferenceRULERbenchmarkKVzip
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard formulation of query-agnostic KV cache eviction—choosing which original key-value pairs to keep—is incomplete. It introduces RestoreKV, which supplements the retained pairs with a small, context-conditioned 'restore cache' generated by a few learned tokens that read the full cache once before eviction. The central claim is that while the information lost by eviction is specific to each context, the mechanism that produces a compact complement of that information can be shared across contexts. If that claim holds, any existing eviction method can be plugged into RestoreKV under the same total memory budget and recover most of the accuracy it loses at tight budgets, with negligible one-time cost and no added query-time latency.

What carries the argument

The central mechanism is the restore pass: a set of $n=8$ learnable restore-token embeddings that are processed, immediately after context prefill and before eviction, through one forward pass of the frozen backbone with LoRA adapters enabled. Because the restore tokens have causal access to the full KV cache, each outputs a KV pair per layer and head, forming a 'restore cache' of $nLH$ pairs. Budget matching is the identity that carries the argument: the total budget $B=\lfloor r T L H\rfloor$ is split so the restore cache occupies $nLH$ slots and the unchanged base evictor fills the remaining $B-nLH$ slots, preserving the query-time KV memory exactly. Training uses symmetric KL self-distillation against the full-cache model, and the LoRA adapters are disabled for all queries and decoding, so the only added cost is a single forward pass over eight positions at cache-construction time.

What would settle it

Evaluate RestoreKV at a retention ratio of r=0.01, below the minimum ratio sampled during training (0.025), on RULER-4K with Qwen3-4B. The paper's claim that one shared mechanism supports multiple budgets implies it should still improve over KVzip at this out-of-range ratio; if the improvement vanishes or turns negative, the mechanism only works inside the trained ratio range.

Watch

Extended reading notes

Core claim

RestoreKV's core discovery is that a single shared, learned transformation can turn the full KV cache of any context into a small set of restore KV pairs that compensate for what eviction removes. Concretely, eight learnable restore-token embeddings are appended after the context; in one pass through the frozen backbone equipped with LoRA adapters, they attend to all context key-value pairs and produce $nLH$ new context-conditioned KV pairs. These restore pairs occupy a reserved slice of the fixed KV budget, while the base evictor fills the remaining slots with original context pairs exactly as it would without RestoreKV. The whole system is trained offline by self-distillation: the restored-cache student is optimized to match the frozen full-cache teacher's answer-token distributions, with budget ratios sampled uniformly from $U(0.025,0.25)$. The paper reports that this recovers a large fraction of the accuracy lost under aggressive eviction—for example, raising KVzip from 38.2 to 73.2 on RULER-4K at a 5% KV budget on Qwen3-4B—and that the gain comes primarily from the attention-side LoRA adaptation that generates the context-conditioned restore cache, not from generic information stored in the restore-token embeddings.

Load-bearing premise

The one shared set of restore-token embeddings and LoRA adapters, trained on about 6.2 thousand examples from three text corpora, transfers across unseen contexts, across retention ratios, and to contexts about seven times longer than anything seen in training, without any per-context fitting. If that transfer fails, RestoreKV would need per-context optimization and would lose its efficiency advantage.

Editorial extensions

If this is right

  • Any query-agnostic evictor—KVzip, KVzip+, ContrastKV, SnapKV, H2O, or Fast KVzip—can be upgraded by training one RestoreKV checkpoint per model and evictor, improving 59 of 60 paired budget-matched settings on Qwen3-4B.
  • At a 5% KV budget, RestoreKV raises KVzip on RULER-4K from 38.2 to 73.2, and RestoreKV+ reaches 86.4 RULER accuracy at 16× compression on the KVPress Benchmark with KVzip+.
  • The one-time cache-construction cost is small: about 0.03–0.04 s (under 0.5% of compression time) and 84 MB (0.4% peak memory) on a 32K context with Llama-3.1-8B-Instruct, with no added query-time latency or memory.
  • The gains are driven by the attention-side LoRA adaptation that generates context-conditioned restore states, not by the learnable embeddings themselves; fixed embeddings with LoRA recover 96% of the full improvement.
  • The method generalizes to contexts much longer than the training data: on SCBench's ~104K-token tasks, RestoreKV improves KVzip at all four tested budget ratios despite being trained on contexts up to 15K tokens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • RestoreKV suggests a general design principle for compression: instead of only selecting what to keep, reserve a small fraction of the budget for a learned summary of what was discarded; the same principle could apply to prompt summarization or embedding distillation where a shared reconstruction mechanism replaces per-instance optimization.
  • Because the restore tokens attend to the full cache, RestoreKV's overhead should grow with context length in absolute terms, but the reported 0.04 s at 32K implies the method can remain worthwhile at 100K+ contexts; a direct measurement of restore-pass latency at SCBench lengths would confirm this scaling.
  • The training-inference evictor mismatch results suggest that the restore mechanism adapts to the specific error pattern of the evictor it was trained with; a natural extension is to train RestoreKV jointly with a family of evictors or with an adversarial evictor so a single checkpoint remains effective when the eviction rule changes at deployment.
  • Because the restore cache is generated from the full cache before eviction, it can capture distributed patterns across many tokens that no selection-based method could retain; this hints that RestoreKV may excel on tasks where answers depend on global context rather than a few salient tokens, a hypothesis testable by comparing retrieval-heavy versus aggregation-heavy task categories within RULER.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes RestoreKV, a plug-in for query-agnostic KV cache eviction. After context prefill, n=8 learned restore tokens attend to the full KV cache in a single LoRA-adapted pass, producing nLH context-conditioned KV pairs; these are concatenated with the base evictor's retained pairs under the same total KV budget. The restore-token embeddings and LoRA adapters are trained offline by self-distillation from the frozen full-cache model on 6.2k examples, with the budget ratio sampled uniformly. At inference the adapters are disabled after cache construction, so query processing and decoding use the original frozen backbone. Experiments on Qwen3-0.6B/4B/8B and Llama-3.1-8B-Instruct across RULER-4K, QASPER, QuALITY, and LongHealth (plus LongBench and SCBench in the appendix) report large gains at tight budgets, e.g., RULER-4K KVzip from 38.2 to 73.2 at r=0.05 on Qwen3-4B, and 59 of 60 paired budget-matched improvements over five base eviction methods.

Significance. If the main results hold, RestoreKV is a practically valuable complement to selection-based eviction: it keeps the base importance scorer and eviction rule unchanged, adds negligible one-time cache-construction overhead, and improves five different eviction methods across four backbones. The paper's ablations are a genuine strength: Table 2 isolates attention-side LoRA adaptation as the primary source of recovery, Table 3 controls for full-context conditioning, Appendix A.2 rules out the RoPE-offset confound, Appendix C.5 reports seed sensitivity, and Appendix A.3 tests training-inference evictor mismatch. The self-distillation target is an external full-cache behavior, so the evaluation is not circular. However, the fixed-size restore cache scales poorly to very long contexts: at roughly 104K tokens the reported SCBench gains are far smaller than at 4-11K, which limits the scope of the paper's central claim that compression-induced degradation is substantially reduced.

major comments (3)
  1. [Appendix B.2; Eq. (3), Eq. (6)] The long-context evidence in Appendix B.2 does not support the generalization claim made there. Because the restore cache has size nLH independent of context length T, its share of the total KV budget is n/(rT); at an average context of 104K tokens and r=0.05 this is about 0.15% of the budget, versus about 3.9% at the 4K RULER context. The reported SCBench averages are correspondingly small: 29.4 to 29.7 at r=0.4, 31.6 to 33.1 at r=0.2, 30.0 to 32.4 at r=0.1, and 24.1 to 25.6 at r=0.05, compared with a 34.6-point gain on RULER-4K at r=0.05. The statement that 'the benefit of the restore cache is not limited to short contexts' is technically true, but the benefit shrinks sharply with T, and the abstract's claim of 'substantially reduces compression-induced degradation' is not established at roughly 100K tokens. The authors should either scale n with context length (analyzing the resulting budget trade-off), provide a capacity analysis, or substantially temper the long-context claim.
  2. [Table 4] The non-monotonic effect of the restore-token count prevents the obvious remedy of simply increasing n for long contexts. At r=0.05, accuracy is 65.3 for n=1, 70.4 for n=2, 68.5 for n=4, 73.2 for n=8, and 69.1 for n=16, so the optimal n is not monotonic and n=16 is worse than n=8. This means the capacity bottleneck identified above cannot be resolved by increasing n without further study, and the paper should either report how n interacts with longer contexts (for example, an n=16 run at 32K or 104K) or explain the non-monotonicity.
  3. [Section 4.2; Appendix C.5] The headline empirical claims rest on single-seed runs. Table 1 and Figure 3 report seed-0 results for all settings, and Appendix C.5 reports seed statistics only for RULER-4K on Qwen3-4B. Since one of the 60 paired settings in Table 1 is a regression (SnapKV on QuALITY at r=0.1, -1.2), the consistency claim would be materially stronger if seed variance were reported for at least one additional benchmark, or if the single-seed nature of the main table were stated more prominently in Section 4.
minor comments (6)
  1. [Section 3.3, Eq. (7)-(8)] The notation p_full_i and p_res_i is clear to a specialist, but the equations would benefit from an explicit statement that both distributions are over the vocabulary at answer position i and are conditioned on the same gold answer prefix y_{<i}.
  2. [Section 3.2, Eq. (6)] Please specify how the base evictor recomputes per-layer and per-head budgets when the total budget is reduced from B to B-nLH; if the allocation is proportional, the floor/ceil handling should be stated so that the claim of an unchanged eviction rule is precise.
  3. [Abstract; Section 4.1] The abstract describes 'four long-context benchmarks' while the longest of these is about 11K tokens (LongHealth); the roughly 104K-token SCBench results appear only in the appendix. The context-length coverage of the headline claim should be stated more precisely.
  4. [Appendix C.1] The training mixture includes generic instructions such as 'summarize' and 'explain' alongside source-provided queries; a sentence on how well the generic-instruction examples transfer to the downstream benchmarks would be useful.
  5. [Reproducibility] The paper provides a project page but no code or data release; releasing the training-data construction scripts and the evaluation harness would materially aid reproducibility.
  6. [Figure 6(a)] The overhead percentage is described relative to 'end-to-end compression time' in the text and 'compression time' in the caption; clarify whether this includes the full prefill or only the scoring/eviction phase.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RestoreKV's headline gains come from held-out benchmark accuracy, not from a quantity defined by the training objective or by self-citation.

full rationale

RestoreKV's derivation chain is an empirical pipeline, not a closed-form reduction. The restore cache is generated by Eq. (4) as a function of the full cache C, composed with the base evictor under the budget-matching Eq. (6), and trained by Eq. (8) to match the frozen full-cache teacher's answer-token distributions. The reported claims are benchmark scores (RULER-4K, QASPER, QuALITY, LongHealth, LongBench, SCBench) on contexts disjoint from the LongAlpaca/PG-19/Tulu-3 FLAN training mixture, so the training loss does not define the evaluation metric. The self-distillation target is the same backbone's full-cache output, which is a legitimate reference behavior rather than a circular input: the method must generalize to unseen contexts and tasks to raise task accuracy. There is no load-bearing self-citation: the cited base evictors (KVzip, KVzip+, ContrastKV, SnapKV, H2O), synthesized-cache baselines (Attention Matching, Cartridges, etc.), and infrastructure (LoRA, RULER, SCBench) are external. The n=8 token count and U(0.025,0.25) ratio range are chosen via ablations on RULER-4K, so the headline RULER-4K number is a selected-configuration result rather than a fully out-of-the-box prediction; this is a benchmarking-validity caveat, not a circularity, because n and the ratio range do not mathematically force the 34.6-point gain. The small SCBench gains at ~104K contexts indicate an external-validity/long-context-capacity concern, not a reduction of the output to the input. No step in the paper equates a predicted quantity with a fitted parameter or imports a uniqueness result from the authors' own prior work.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on a handful of standard assumptions about distillation and generalization, plus two trained components (restore tokens and restore cache) that are not independently falsifiable outside the paper's benchmarks. The hyperparameters n=8, LoRA rank, alpha, and the training ratio range are set by hand or validation; they are not fitted to the evaluation benchmarks' target numbers, but they do affect the reported gains.

free parameters (4)
  • Number of restore tokens n = 8
    Chosen by ablation on RULER-4K (Table 4): n=8 gives the best accuracy at r=0.05; n=16 is slightly worse.
  • LoRA rank r_LoRA = 8
    Standard low-rank choice, stated in Section 3.2; not swept.
  • LoRA scaling alpha = 16
    Chosen by hand, stated in Section 3.2 and C.2.
  • Training ratio sampling range U(r_min, r_max) = 0.025 to 0.25
    Selected by validation on RULER-4K (Appendix Table A); this range maximizes accuracy at tight budgets.
assumptions (4)
  • domain assumption The full-cache model is a suitable teacher for distillation; matching its answer-token distribution improves downstream task accuracy.
    Section 3.3 uses self-distillation from the frozen full-cache model as the training signal; assumes that this teacher captures the behavior to preserve.
  • domain assumption The training contexts (LongAlpaca, PG-19, Tulu-3 FLAN) are representative of evaluation contexts across benchmarks.
    Appendix C.1 constructs the 6.2k training mixture; SCBench results (Appendix B.2) test generalization to longer contexts.
  • domain assumption LoRA-generated restore KV states remain compatible with the frozen backbone at inference when LoRA is disabled.
    Section 3.2 and 3.4 rely on this; verified empirically by the reported improvements, but not proven.
  • domain assumption Query-agnostic eviction scores s and rule Evict are fixed and can be treated as auxiliaries.
    Section 3.1 treats the base evictor as a black box; RestoreKV reserves slots and keeps scoring unchanged.
invented entities (2)
  • Restore-token embeddings (n=8 per context)
    purpose: Learned embeddings that, together with LoRA adapters, generate a context-conditioned restore cache Cres to complement the evicted cache.
    The only evidence is the paper's own benchmarks and ablations; there is no falsifiable prediction outside this paper.
  • Restore cache Cres (nLH KV pairs)
    purpose: Compact complement that replaces part of the retained cache under the same total budget.
    It is the output of the trained mechanism; its benefit is demonstrated only in the paper's experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction." pith.science (2026). https://pith.science/paper/ZQP3U7PX

@misc{pith2026260801247,
  author       = {Pith},
  title        = {Pith review of: RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZQP3U7PX}},
  note         = {Machine review of arXiv:2608.01247}
}
abstract

Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained. We introduce RestoreKV, which complements this selection-based formulation with learned restoration under the same total KV budget. Our key insight is that, although the information lost through eviction is context-specific, the mechanism for generating its compact complement can be shared across contexts. After context prefill, a few restore tokens attend to the full KV cache in a single LoRA-adapted pass, generating a compact, context-conditioned restore cache. The base importance scorer and eviction rule remain unchanged, and the adapters are disabled for all subsequent queries and decoding. RestoreKV is trained through parameter-efficient self-distillation from the frozen full-cache model, optimizing only $0.4\%$ of the parameters and requiring no task-specific tuning. Across four backbones and four long-context benchmarks, RestoreKV substantially reduces compression-induced degradation. On Qwen3-4B, it improves 59 of 60 paired, budget-matched settings across five base eviction methods; at a $5\%$ budget, it raises KVzip from $38.2$ to $73.2$ on RULER-4K. Applied to KVzip+, RestoreKV reaches $86.4$ RULER accuracy at $16\times$ compression on the KVPress Benchmark, while adding less than $0.5\%$ one-time cache-construction overhead in a 32K-context evaluation. Our project page is available at https://paper.pnu-cvsp.com/RestoreKV/

Figures

Figures reproduced from arXiv: 2608.01247 by the authors.

Figure 1
Figure 1. RestoreKV narrows the gap to full-cache behav￾ior under aggressive compression at negligible one-time cost. Applied to KVzip+, RestoreKV reaches 86.4 RULER accuracy at 16× compression on the KVPress Benchmark while adding negligible compression-time overhead. et al. 2025; Jegou and Jeblick 2026; Chen et al. 2026; Kim, Han, and Yun 2026), the resulting cache remains composed only of retained original states. The cent… view at source ↗
Figure 2
Figure 2. Overview of RestoreKV. After context prefill, a single LoRA-adapted restore pass processes n restore tokens (n=8 by default) with causal access to the full KV cache, generating nLH context-conditioned KV pairs. These restore states are combined with the retained context states under the same total KV budget. LoRA is active only during restore-cache generation; subsequent query processing and decoding use the origina… view at source ↗
Figure 3
Figure 3. Performance across four model backbones and four benchmarks. RestoreKV and RestoreKV+ are applied to KVzip and KVzip+, respectively, under the same KV budget. Both methods reduce compression-induced performance loss, with larger gains under tighter cache budgets. PRO 6000 GPU, and a full training run for Qwen3-4B takes approximately two hours. 4.2 Main Results [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison with a Synthesized-Cache Baseline on Qwen3-4B. RestoreKV+ consistently improves token￾eviction baselines and remains competitive with AM-fast, a strong synthesized-cache method, at ∼13× lower compres￾sion time on 4K-token contexts. structs a 4K-context cache…
Figure 6
Figure 6. Figure 6: evaluates RestoreKV on Llama-3.1-8B-Instruct with 32K-token contexts using an NVIDIA RTX PRO 6000. Con￾structing the eight-token restore cache before eviction incurs only 0.03–0.04s of additional latency, accounting for less than 0.5% of the end-to-end compression time…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 17 canonical work pages

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  4. [4]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  5. [5]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  6. [6]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =

  7. [7]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  8. [8]

    Efficient Streaming Language Models with Attention Sinks , url =

    Xiao, Guangxuan and Tian, Yuandong and Chen, Beidi and Han, Song and Lewis, Mike , booktitle =. Efficient Streaming Language Models with Attention Sinks , url =

Show all 44 references
  1. [9]

    Advances in Neural Information Processing Systems , volume=

    Kvzip: Query-agnostic kv cache compression with context reconstruction , author=. Advances in Neural Information Processing Systems , volume=

  2. [10]

    SnapKV: LLM Knows What You are Looking for Before Generation , url =

    Li, Yuhong and Huang, Yingbing and Yang, Bowen and Venkitesh, Bharat and Locatelli, Acyr and Ye, Hanchen and Cai, Tianle and Lewis, Patrick and Chen, Deming , booktitle =. SnapKV: LLM Knows What You are Looking for Before Generation , url =. doi:10.52202/079017-0722 , editor =

    doi

  3. [11]

    H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , url =

    Zhang, Zhenyu and Sheng, Ying and Zhou, Tianyi and Chen, Tianlong and Zheng, Lianmin and Cai, Ruisi and Song, Zhao and Tian, Yuandong and R\'. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , url =. Advances in Neural Information Processin...

  4. [12]

    The Fourteenth International Conference on Learning Representations , year=

    Cartridges: Lightweight and general-purpose long context representations via self-study , author=. The Fourteenth International Conference on Learning Representations , year=

  5. [13]

    Zefan Cai and Yichi Zhang and Bofei Gao and Yuliang Liu and Yucheng Li and Tianyu Liu and Keming Lu and Wayne Xiong and Yue Dong and Junjie Hu and Wen Xiao , booktitle=. Pyramid. 2025 , url=

    2025

  6. [14]

    arXiv preprint arXiv:1503.02531 , year=

    Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=

    arXiv

  7. [15]

    Neurocomputing , volume=

    RoFormer: Enhanced transformer with Rotary Position Embedding , author=. Neurocomputing , volume=. 2024 , publisher=

    2024

  8. [16]

    Edward J Hu and yelong shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo. 2022 , url=

    2022

  9. [17]

    International Conference on Learning Representations , volume=

    Razorattention: Efficient kv cache compression through retrieval heads , author=. International Conference on Learning Representations , volume=

  10. [18]

    Advances in Neural Information Processing Systems , volume=

    Keydiff: Key similarity-based kv cache eviction for long-context llm inference in resource-constrained environments , author=. Advances in Neural Information Processing Systems , volume=

  11. [19]

    SCBench: A KV Cache-Centric Analysis of Long-Context Methods , url =

    Li, Yucheng and Jiang, Huiqiang and Wu, Qianhui and Luo, Xufang and Ahn, Surin and Zhang, Chengruidong and Abdi, Amir and Li, Dongsheng and Gao, Jianfeng and Yang, Yuqing and Qiu, Lili , booktitle =. SCBench: A KV Cache-Centric Analysis of Long-Context Methods , url =

  12. [20]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  13. [21]

    Advances in Neural Information Processing Systems , volume=

    Ada-kv: Optimizing kv cache eviction by adaptive budget allocation for efficient llm inference , author=. Advances in Neural Information Processing Systems , volume=

  14. [22]

    IEEE Communications Surveys & Tutorials , volume=

    Mobile edge intelligence for large language models: A contemporary survey , author=. IEEE Communications Surveys & Tutorials , volume=. 2025 , publisher=

    2025

  15. [23]

    Advances in Neural Information Processing Systems , volume=

    Chunkkv: Semantic-preserving kv cache compression for efficient long-context llm inference , author=. Advances in Neural Information Processing Systems , volume=

  16. [24]

    Lookahead

    Jinwoo Ahn and Ingyu Seong and Akhil Kedia and Junhan Kim and Hyemi Jang and Kangwook Lee and Yongkweon Jeon , booktitle=. Lookahead. 2026 , url=

    2026

  17. [25]

    arXiv preprint arXiv:2605.23258 , year=

    A Simple Plug-in for Improving Eviction-Based KV Cache Compression , author=. arXiv preprint arXiv:2605.23258 , year=

    arXiv

  18. [26]

    arXiv preprint arXiv:2503.10337 , year=

    Kv-distill: Nearly lossless learnable context compression for llms , author=. arXiv preprint arXiv:2503.10337 , year=

    arXiv

  19. [27]

    Forty-third International Conference on Machine Learning , year=

    Fast KV Compaction via Attention Matching , author=. Forty-third International Conference on Machine Learning , year=

  20. [28]

    International conference on machine learning , pages=

    The flan collection: Designing data and methods for effective instruction tuning , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  21. [29]

    arXiv preprint arXiv:2602.21221 , year=

    Latent context compilation: Distilling long context into compact portable memory , author=. arXiv preprint arXiv:2602.21221 , year=

  22. [30]

    arXiv preprint arXiv:2606.07878 , year=

    Still: Amortized KV Cache Compaction in a Single Forward Pass , author=. arXiv preprint arXiv:2606.07878 , year=

    arXiv

  23. [31]

    arXiv preprint arXiv:2404.06654 , year=

    RULER: What's the real context size of your long-context language models? , author=. arXiv preprint arXiv:2404.06654 , year=

    arXiv

  24. [32]

    , author=

    LongHealth: A Question Answering Benchmark with Long Clinical Documents. , author=. Journal of Healthcare Informatics Research , volume=

  25. [33]

    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    QuALITY: Question answering with long input texts, yes! , author=. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2022

  26. [34]

    Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    A dataset of information-seeking questions and answers anchored in research papers , author=. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2021

  27. [35]

    L ong B ench: A Bilingual, Multitask Benchmark for Long Context Understanding

    Bai, Yushi and Lv, Xin and Zhang, Jiajie and Lyu, Hongchang and Tang, Jiankai and Huang, Zhidian and Du, Zhengxiao and Liu, Xiao and Zeng, Aohan and Hou, Lei and Dong, Yuxiao and Tang, Jie and Li, Juanzi. L ong B ench: A Bilingual, Multitask Benchmark for Long Context Understa...

    2024 doi

  28. [36]

    Yukang Chen and Shengju Qian and Haotian Tang and Xin Lai and Zhijian Liu and Song Han and Jiaya Jia , booktitle=. LongLo. 2024 , url=

    2024

  29. [37]

    Expected Attention:

    Devoto, Alessio and Jeblick, Maximilian and J. Expected Attention:. arXiv preprint arXiv:2510.00636 , year =

  30. [38]

    Yang, An and Li, Anfeng and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Gao, Chang and others , journal =

  31. [39]

    Grattafiori, Aaron and Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and others , journal =. The

  32. [40]

    International Conference on Learning Representations , year =

    Compressive Transformers for Long-Range Sequence Modelling , author =. International Conference on Learning Representations , year =

  33. [41]

    arXiv preprint arXiv:2601.17668 , year=

    Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction , author=. arXiv preprint arXiv:2601.17668 , year=

  34. [42]

    arXiv preprint arXiv:2601.07891 , year=

    KVzap: Fast, Adaptive, and Faithful KV Cache Pruning , author=. arXiv preprint arXiv:2601.07891 , year=

  35. [43]

    C ontrast KV : Robust KV Cache Eviction via Contrastive Signal Fusion for Multi-Query Generalization

    Chen, Xingchi and Zong, Peiyuan and Gao, Ziqiang and Li, Qing and Jiang, Yong and Zhu, Fa and Li, Hui. C ontrast KV : Robust KV Cache Eviction via Contrastive Signal Fusion for Multi-Query Generalization. Proceedings of the 64th Annual Meeting of the A ssociation for C omputat...

    2026 doi

  36. [44]

    arXiv preprint arXiv:2411.15124 , year=

    Tulu 3: Pushing frontiers in open language model post-training , author=. arXiv preprint arXiv:2411.15124 , year=

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.