Pith. sign in

REVIEW 3 major objections 5 minor 128 references

FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

T0 review · 3 major / 5 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FreqDepthKV compresses long-context LLM key-value caches by sharing low-frequency depth components across layers while keeping sparse high-frequency residuals that protect retrieval and reasoning.

desk verdict Plausible MiniCache extension with a clean frequency+routing idea, but Table 1 is uncheckable until they name the model and clean the paper. read the letter →

arxiv 2607.06519 v1 pith:FDDICGXT submitted 2026-07-07 cs.AI

Anna Córdoba , Adam Puente Tercero , Nerea Angulo Hijo , Mar Linares Tercero , Julia Barrientos , Ainhoa Miranda , Jesús Olivera This is my paper
classification cs.AI
keywords KVcachecompressionlong-contextLLMinferencedepth-frequencyfactorizationattentionheadroutinghigh-frequencyresidualsreconstruction-awarelossinference-time
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-context language model inference is bottlenecked by the memory and bandwidth of key-value caches, yet aggressive compression often erases the layer-specific evidence that retrieval and multi-step reasoning need. This paper introduces FreqDepthKV, an inference-time method that factorizes adjacent-layer caches into shared low-frequency depth components and sparse high-frequency residuals, then uses a lightweight prefill probe to assign each attention head to shared-depth, residual-depth, or exact mode according to how compression would change attention logits. The policy adapts to prompt structure without retraining. On long-context QA, needle retrieval, summarization, and code generation with a 32k-token prefill, it closely matches full-cache accuracy while cutting peak KV memory to 6.2 GB (about 3.9 imes compression) and raising decoding throughput to 70.4 tokens/s. A sympathetic reader cares because many long prompts hide decisive evidence in a few token-head-layer interactions that average-error depth sharing would discard.

What carries the argument

Depth-frequency factorization with reconstruction-aware head routing: adjacent-layer KV states are transformed with a fixed DCT basis into shared low-frequency components and sparse high-frequency residuals; a prefill probe routes each head to shared-depth, residual-depth, or exact mode by minimizing a memory-penalized attention-logit reconstruction loss.

What would settle it

On a long needle-retrieval or multi-step code task, if the prefill probe routes heads to shared-depth mode yet later decoding produces wrong answers that full KV gets right—and restoring residual or exact storage for those same heads recovers correctness—then the prefill logit proxy fails as a stand-in for end-to-end task fidelity.

Watch

Extended reading notes

Core claim

FreqDepthKV establishes that inter-layer KV redundancy is frequency-structured: sharing low-frequency depth components while selectively preserving sparse high-frequency residuals—and exact entries for reconstruction-sensitive heads—lets inference-time compression nearly match full KV accuracy on long-context QA, summarization, and code tasks at roughly 3.9× lower peak KV memory and higher throughput.

Load-bearing premise

A one-shot prefill probe over a small set of query positions is assumed to be a stable enough proxy for which heads must keep residual or exact caches so that answers stay correct throughout decoding without re-routing.

Editorial extensions

If this is right

  • Aggressive depth compression can retain retrieval and code accuracy when high-frequency residuals are kept for logit-sensitive heads.
  • A prefill-only reconstruction-aware router adapts compression to prompt structure without model retraining.
  • Depth-frequency sharing stacks with token eviction and quantization for joint sequence, precision, and depth savings.
  • Mixed layer-block sizes (tighter near model boundaries) balance memory and quality better than uniform blocks.
  • At 32k prefill, peak KV memory can fall to 6.2 GB with 70.4 tokens/s throughput while closely matching full-KV task scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Generation-time re-routing, which the paper only sketches as future work, may be necessary when relevant evidence shifts mid-decode in multi-turn or long-generation settings.
  • Assigning lower bit-width to shared low-frequency coefficients than to sparse residuals could cut bandwidth further without changing the routing logic.
  • The same logit-sensitivity probe could identify heads that are chronically non-compressible across models, guiding architecture choices toward more cache-friendly layers.
  • Prompts with diffuse evidence and weak document boundaries may force more exact-mode heads, shrinking the compression advantage relative to retrieval-heavy workloads.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FreqDepthKV, an inference-time KV-cache compression method that stacks adjacent-layer keys/values, applies a fixed DCT depth transform (Eq. 1), and stores a shared low-frequency component plus sparse high-frequency residuals. An online prefill probe routes each head to shared-depth, residual-depth, or exact mode by minimizing a reconstruction-aware attention-logit loss plus memory penalty (Eq. 2–3). On a 32k prefill setup the method reports 58.3 EM / 63.0 F1 / 32.5 ROUGE-L / 48.1 pass@1, nearly matching Full KV while reducing peak KV memory to 6.2 GB (3.9×) and raising throughput to 70.4 tokens/s (Table 1), with ablations in Table 2 attributing gains to the frequency factorization, residuals, and routing.

Significance. If the empirical claims hold, the work is a useful systems contribution: it refines MiniCache-style depth sharing with frequency residuals and logit-aware head routing, remains training-free, and is complementary to token eviction and quantization. The design is concrete (DCT blocks, three cache modes, residual scoring) and the ablations isolate components. Significance is currently limited by incomplete experimental provenance—the base model is unnamed, budgets and variance are underspecified, and the reference list is unreliable—so the headline Table 1 deltas cannot yet be treated as established facts.

major comments (3)
  1. §4 Experiments never names the base long-context decoder (architecture, size, or checkpoint), only “the same base long-context decoder with a 32k-token prefill window.” Without this, Table 1’s Full-KV and baseline numbers (including the MiniCache comparison that is the main depth-sharing foil) are not reproducible or comparable to the literature, so the central claim that FreqDepthKV nearly matches Full KV at 3.9× cannot be verified.
  2. Table 1 / §4: no error bars, multi-seed runs, or multi-model results are reported, and compression budgets are “tuned on a held-out subset of LongBench” without disclosing the selected budgets or residual rates r_b,h for FreqDepthKV or baselines. Aggregate EM/F1/ROUGE/pass@1 and systems metrics are therefore hard to interpret as stable gains rather than single-run outcomes under an opaque budget protocol.
  3. §3, Eq. (2) and Future Work: the load-bearing assumption that a one-shot prefill probe over |P|=128 positions (recent tokens, document boundaries, high-entropy rows), using key-induced logit reconstruction, is a sufficient proxy for which heads must keep residual/exact storage throughout decoding is asserted but not stress-tested. The paper itself notes routing is fixed after prefill; without generation-time re-routing experiments or failure cases on multi-step/needle tasks, the claim that task answers remain correct under aggressive residual sparsity is only weakly supported.
minor comments (5)
  1. Header claims “37th Conference on Neural Information Processing Systems (NeurIPS 2023)” while the arXiv stamp is 2026; this should be corrected.
  2. References contain many off-topic PMC/sensor/biology/fluid-dynamics entries unrelated to KV cache compression; the bibliography needs a thorough cleanup.
  3. Figure 1 is described as summarizing routing patterns in §4, but the caption only restates the high-level idea; a figure that actually shows head-mode assignments would help.
  4. Notation: B is used both for block size and (implicitly) for the DCT basis F_B; residual rate r_b,h and the exact split of “first coefficient group” vs remaining groups should be stated more precisely.
  5. Systems metrics (tokens/s, TTFT, peak GB) lack hardware/software stack details (GPU, batch size, kernel fusion), which limits interpretability of the 70.4 tokens/s and 6.2 GB claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical systems method with external task metrics, not a self-forcing derivation.

full rationale

FreqDepthKV is an inference-time engineering method, not a first-principles derivation that claims to force task accuracy from its own inputs. The depth-frequency factorization (Eq. 1) and reconstruction-aware routing loss L_b,h(m) (Eq. 2) define how heads are assigned to shared/residual/exact modes; those choices minimize a measured logit-reconstruction-plus-memory objective during prefill. Downstream claims (EM, F1, ROUGE-L, pass@1, tokens/s, TTFT) are then reported as separate empirical measurements on LongBench, Needle-in-a-Haystack, summarization, and code benchmarks against Full KV and prior compressors (Table 1, ablations in Table 2). Nothing in the equations algebraically implies those task scores. Tuning λ to a target compression budget is ordinary systems hyperparameter selection and does not rename a fit as a prediction of accuracy. Citations to MiniCache and token/quantization baselines are external prior art by other authors, not load-bearing self-citation uniqueness theorems. There is no self-definitional loop, no fitted parameter re-labeled as the headline result, and no ansatz smuggled in as a forced theorem. Score 0 is appropriate; experimental unverifiability (unnamed base model, etc.) is a reproducibility concern, not circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 3 invented entities

The central accuracy-under-compression claim rests on empirical design choices (block size, probe, λ budget, residual rate) and domain assumptions about inter-layer redundancy and logit sensitivity, not on free physical constants or formal theorems. Invented entities are algorithmic constructs (modes, residual index sets, routing loss), not new physical objects. No independent external evidence is given beyond the paper's own tables.

free parameters (5)
  • layer block size B (default 4; 2 at boundaries)
    Hand-chosen depth-block width controlling how many layers share low-frequency coefficients; ablated as B=2/8.
  • memory penalty λ
    Tuned so estimated peak KV footprint hits ~3.8–3.9× compression; directly steers mode assignment.
  • residual retention rate r_b,h
    Fraction of tokens keeping high-frequency residuals, chosen from routing loss budget.
  • probe size |P|=128 and probe sampling policy
    Number and selection of prefill query positions used to score reconstruction-sensitive heads.
  • DCT/frequency split (first coefficient group as shared low-frequency)
    Ad hoc partition of depth spectrum into shared vs residual bands without learned basis.
assumptions (4)
  • domain assumption Adjacent transformer layers have substantial low-frequency KV redundancy that a fixed short orthonormal depth transform (DCT) can separate from sparse high-frequency residuals.
    Core premise of §3 Eq. (1) and MiniCache lineage; not proved, only motivated.
  • ad hoc to paper Key-induced attention-logit reconstruction error on a small prefill probe set predicts which heads need residual/exact storage for correct long-context task behavior.
    Routing objective Eq. (2) and mode assignment; load-bearing and not independently validated outside reported tables.
  • ad hoc to paper Budgeted routing decided once after prefill can be reused throughout autoregressive decoding (optional rare refresh).
    Stated in §3; Future Work notes generation-aware routing may be needed.
  • domain assumption Standard transformer attention and KV caching semantics hold; method is inference-only and needs no weight updates.
    Background systems assumption throughout method and experiments.
invented entities (3)
  • shared-depth / residual-depth / exact head cache modes
    purpose: Per-head discrete compression policy adapting to prompt structure.
    Defined by the paper's routing procedure; no external independent definition.
  • depth-frequency factorization of stacked adjacent-layer KV (Z = F_B X)
    purpose: Separate shared low-frequency depth components from sparse high-frequency residuals.
    Algorithmic representation introduced in §3; evidence is only internal ablations.
  • reconstruction-aware routing loss L_b,h(m) over cached attention logits
    purpose: Choose compression mode by measured logit change plus memory cost.
    Paper-specific objective; not a standard external metric with independent calibration.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference." pith.science (2026). https://pith.science/paper/FDDICGXT

@misc{pith2026260706519,
  author       = {Pith},
  title        = {Pith review of: FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FDDICGXT}},
  note         = {Machine review of arXiv:2607.06519}
}
read the original abstract

Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.

Figures

Figures reproduced from arXiv: 2607.06519 by the authors.

Figure 1
Figure 1. Overview of the core idea: FreqDepthKV removes redundant depth information shared [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Architecture of FreqDepthKV: KV states are decomposed across layer depth into shared [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

128 extracted references · 128 canonical work pages

  1. [1]

    2026 , journal =

    PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference , author =. 2026 , journal =

    work page 2026

  2. [2]

    2024 , journal =

    MiniCache: KV Cache Compression in Depth Dimension for Large Language Models , author =. 2024 , journal =

    work page 2024

  3. [3]

    2026 , journal =

    Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction , author =. 2026 , journal =

    work page 2026

  4. [4]

    2026 , journal =

    ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing , author =. 2026 , journal =

    work page 2026

  5. [5]

    2026 , journal =

    The risk of KV cache compression , author =. 2026 , journal =

    work page 2026

  6. [6]

    2025 , journal =

    Attention Is All You Need for KV Cache in Diffusion LLMs , author =. 2025 , journal =

    work page 2025

  7. [7]

    2025 , journal =

    R-KV: Redundancy-aware KV Cache Compression for Reasoning Models , author =. 2025 , journal =

    work page 2025

  8. [8]

    2024 , journal =

    Lossless KV Cache Compression to 2\ author =. 2024 , journal =

    work page 2024

Show all 128 references
  1. [9]

    2026 , journal =

    Knowledge Packs: Zero-Token Knowledge Delivery via KV Cache Injection , author =. 2026 , journal =

    2026

  2. [10]

    2025 , journal =

    The Pitfalls of KV Cache Compression , author =. 2025 , journal =

    2025

  3. [11]

    2025 , journal =

    ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference , author =. 2025 , journal =

    2025

  4. [12]

    2024 , journal =

    A Survey on Large Language Model Acceleration based on KV Cache Management , author =. 2024 , journal =

    2024

  5. [13]

    2024 , journal =

    KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing , author =. 2024 , journal =

    2024

  6. [14]

    2025 , journal =

    Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization , author =. 2025 , journal =

    2025

  7. [15]

    2025 , journal =

    TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization , author =. 2025 , journal =

    2025

  8. [16]

    2026 , journal =

    Compressing Kv Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity , author =. 2026 , journal =

    2026

  9. [17]

    2025 , journal =

    SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers , author =. 2025 , journal =

    2025

  10. [18]

    2024 , journal =

    Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks , author =. 2024 , journal =

    2024

  11. [19]

    2025 , journal =

    Hcattention: Extreme Kv Cache Compression Via Heterogeneous Attention Computing for Llms , author =. 2025 , journal =

    2025

  12. [20]

    2025 , journal =

    EvolKV: Evolutionary KV Cache Compression for LLM Inference , author =. 2025 , journal =

    2025

  13. [21]

    2025 , journal =

    Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques , author =. 2025 , journal =

    2025

  14. [22]

    2026 , journal =

    The kv4 Trade-off Is Workload-Dependent: A Depth-and Workload-Resolved Study of 4-bit KV-Cache Quantization on a 4 GB Turing GPU , author =. 2026 , journal =

    2026

  15. [23]

    2025 , journal =

    TableKV: KV Cache Compression for In-Context Table Processing , author =. 2025 , journal =

    2025

  16. [24]

    2026 , journal =

    Q-KVComm: Efficient Multi-Agent Communication via Adaptive KV Cache Compression , author =. 2026 , journal =

    2026

  17. [25]

    2026 , journal =

    HCAttention: Extreme KV cache compression via heterogeneous attention computing for LLMs , author =. 2026 , journal =

    2026

  18. [26]

    2025 , journal =

    Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression , author =. 2025 , journal =

    2025

  19. [27]

    2024 , journal =

    Eigen Attention: Attention in Low-Rank Space for KV Cache Compression , author =. 2024 , journal =

    2024

  20. [28]

    , author =

    Structural optimization principles for edge AI in motorsport telemetry. , author =. 2026 , journal =

    2026

  21. [29]

    , author =

    Building large-scale English-Romanian literary translation resources with open models. , author =. 2026 , journal =

    2026

  22. [30]

    , author =

    Limit-Cycle Proliferation Under Parametric Delayed Feedback in a Conductance-Based Neuron: Bifurcation Landscape, Orbit Catalog, and Capacity Analysis. , author =. 2026 , journal =

    2026

  23. [31]

    , author =

    Towards sustainable AI knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources. , author =. 2026 , journal =

    2026

  24. [32]

    , author =

    Multi-Hardware Benchmarking of Open-Source Large Language Models with Retrieval-Augmented Generation for Mitsubishi FX-Series PLC Instruction List Code Generation. , author =. 2026 , journal =

    2026

  25. [33]

    , author =

    Infrared Imaging for Autonomous Power Inspection: A Review from Detector to System Integration. , author =. 2026 , journal =

    2026

  26. [34]

    , author =

    A survey on LLM-as-a-judge. , author =. 2026 , journal =

    2026

  27. [35]

    , author =

    The Rise of Small Language Models in Healthcare: A Comprehensive Survey. , author =. 2026 , journal =

    2026

  28. [36]

    , author =

    Development of a Rapid Deep-Learning-Assisted Multiphase Multicomponent Numerical Simulation Protocol. , author =. 2026 , journal =

    2026

  29. [37]

    , author =

    F-Transformer: a federated transformer for efficient and privacy-preserving sequence generation. , author =. 2026 , journal =

    2026

  30. [38]

    2021 , journal =

    The Modern Mathematics of Deep Learning , author =. 2021 , journal =

    2021

  31. [39]

    2023 , journal =

    Learn to Accumulate Evidence from All Training Samples: Theory and Practice , author =. 2023 , journal =

    2023

  32. [40]

    2023 , journal =

    Deep Learning and Computational Physics (Lecture Notes) , author =. 2023 , journal =

    2023

  33. [41]

    2026 , journal =

    Monodense Deep Neural Model for Determining Item Price Elasticity , author =. 2026 , journal =

    2026

  34. [42]

    2017 , journal =

    A multitask deep learning model for real-time deployment in embedded systems , author =. 2017 , journal =

    2017

  35. [43]

    2026 , journal =

    Training Transformers for KV Cache Compressibility , author =. 2026 , journal =

    2026

  36. [44]

    2019 , journal =

    Deep learning observables in computational fluid dynamics , author =. 2019 , journal =

    2019

  37. [45]

    2024 , journal =

    Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving , author =. 2024 , journal =

    2024

  38. [46]

    2025 , journal =

    Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention , author =. 2025 , journal =

    2025

  39. [47]

    2025 , journal =

    SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation , author =. 2025 , journal =

    2025

  40. [48]

    2026 , journal =

    Sub-Token Routing for KV Cache Compression , author =. 2026 , journal =

    2026

  41. [49]

    2024 , journal =

    Hymba: A Hybrid-head Architecture for Small Language Models , author =. 2024 , journal =

    2024

  42. [50]

    2025 , journal =

    KV-RAPTOR: Scalable Tree-Structured Retrieval with KV Cache Compression for Question-Answering Systems , author =. 2025 , journal =

    2025

  43. [51]

    1970 , journal =

    Deep Learning-Based Point Cloud Compression: An In-Depth Survey and Benchmark\_supp1-3594355.pdf , author =. 1970 , journal =

    1970

  44. [52]

    2026 , journal =

    KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference , author =. 2026 , journal =

    2026

  45. [53]

    2025 , journal =

    FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression , author =. 2025 , journal =

    2025

  46. [54]

    2026 , journal =

    FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration , author =. 2026 , journal =

    2026

  47. [55]

    , author =

    Large-Scale Model-Enhanced Vision-Language Navigation: Recent Advances, Practical Applications, and Future Challenges. , author =. 2026 , journal =

    2026

  48. [56]

    , author =

    SS-AdaMoE: Spatio-Spectral Adaptive Mixture of Experts with Global Structural Priors for Graph Node Classification. , author =. 2026 , journal =

    2026

  49. [57]

    , author =

    AI Agents as Universal Task Solvers. , author =. 2026 , journal =

    2026

  50. [58]

    , author =

    Green prompt engineering for sustainable generative AI. , author =. 2026 , journal =

    2026

  51. [59]

    , author =

    Anatomic Interactive Atlas of the Loggerhead Sea Turtle (Caretta caretta) Coelomic Cavity. , author =. 2026 , journal =

    2026

  52. [60]

    , author =

    On Demand Secure Scalable Video Streaming for Both Human and Machine Applications. , author =. 2026 , journal =

    2026

  53. [61]

    , author =

    Compute-in-memory implementation of state space models for event sequence processing. , author =. 2026 , journal =

    2026

  54. [62]

    , author =

    Kilohertz volumetric imaging of in vivo dynamics using squeezed light field microscopy. , author =. 2025 , journal =

    2025

  55. [63]

    Ai agents as universal task solvers

    Achille A and Soatto S. Ai agents as universal task solvers. PMC, 2026. doi:10.3390/e28030332

    2026 doi

  56. [64]

    Anatomic interactive atlas of the loggerhead sea turtle (caretta caretta) coelomic cavity

    Arencibia A, Melian A, and Oros J. Anatomic interactive atlas of the loggerhead sea turtle (caretta caretta) coelomic cavity. PMC, 2026 a . doi:10.3390/ani16050754

    2026 doi

  57. [65]

    On demand secure scalable video streaming for both human and machine applications

    Zain A, Fan Y, and Zhou J. On demand secure scalable video streaming for both human and machine applications. PMC, 2026 b . doi:10.3390/s26041285

    2026 doi

  58. [66]

    Rest-kv: Robust kv cache eviction with layer-wise output reconstruction and spatial-temporal smoothing

    Yongqi An, Chang Lu, Kuan Zhu, Tao Yu, Chaoyang Zhao, Hong Wu, Ming Tang, and Jinqiao Wang. Rest-kv: Robust kv cache eviction with layer-wise output reconstruction and spatial-temporal smoothing. arXiv preprint, 2026

    2026

  59. [67]

    The modern mathematics of deep learning

    Julius Berner, Philipp Grohs, Gitta Kutyniok, and Philipp Petersen. The modern mathematics of deep learning. arXiv preprint, 2021. doi:10.1017/9781009025096.002

    2021 doi

  60. [68]

    Make each token count: Towards improving long-context performance with kv cache eviction

    Ngoc Bui, Hieu Trung Nguyen, Arman Cohan, and Rex Ying. Make each token count: Towards improving long-context performance with kv cache eviction. arXiv preprint, 2026

    2026

  61. [69]

    R-kv: Redundancy-aware kv cache compression for reasoning models

    Zefan Cai, Wen Xiao, Hanshi Sun, Cheng Luo, Yikai Zhang, Ke Wan, Yucheng Li, Yeyang Zhou, Li-Wen Chang, Jiuxiang Gu, Zhen Dong, Anima Anandkumar, Abedelkadir Asi, and Junjie Hu. R-kv: Redundancy-aware kv cache compression for reasoning models. arXiv preprint, 2025

    2025

  62. [70]

    The pitfalls of kv cache compression

    Alex Chen, Renato Geh, Aditya Grover, Guy Van den Broeck, and Daniel Israel. The pitfalls of kv cache compression. arXiv preprint, 2025

    2025

  63. [71]

    Tablekv: Kv cache compression for in-context table processing

    Giulio Corallo, Elia Faure-Rolland, Miriam Lamari, and Paolo Papotti. Tablekv: Kv cache compression for in-context table processing. Journal, 2025. doi:10.18653/v1/2025.trl-1.13

    2025 doi

  64. [72]

    da Silva, Savio S

    Joao Gabriel J. da Silva, Savio S. T. de Oliveira, Lucas Alexandria Alves, Nicolas Eiris, and Arlindo Rodrigues Galvao Filho. Kv-raptor: Scalable tree-structured retrieval with kv cache compression for question-answering systems. Journal, 2025. doi:10.5753/sbbd.2025.247245

    2025 doi

  65. [73]

    Hymba: A hybrid-head architecture for small language models

    Xin Luna Dong, Yonggan Fu, Shizhe Diao, Wonmin Byeon, Zhaoliang Chen, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Matthijs Van Keirsbilck, Min-Hung Chen, Yoshi Suhara, Yingyan Lin, Jan Kautz, and Pavlo Molchanov. Hymba: A hybrid-head architecture for small language models. Jo...

    2024 doi arXiv

  66. [74]

    Deep learning-based point cloud compression: An in-depth survey and benchmark\_supp1-3594355.pdf

    Wei Gao. Deep learning-based point cloud compression: An in-depth survey and benchmark\_supp1-3594355.pdf. Journal, 1970. doi:10.1109/tpami.2025.3594355/mm1

    1970 doi

  67. [75]

    Monodense deep neural model for determining item price elasticity

    Lakshya Garg, Sai Yaswanth, Deep Narayan Mishra, Karthik Kumaran, Anupriya Sharma, and Mayank Uniyal. Monodense deep neural model for determining item price elasticity. arXiv preprint, 2026. doi:10.1109/AAIML67890.2026.11498150

    2026 doi

  68. [76]

    Training transformers for kv cache compressibility

    Yoav Gelberg, Yam Eitan, Michael Bronstein, Yarin Gal, and Haggai Maron. Training transformers for kv cache compressibility. arXiv preprint, 2026

    2026

  69. [77]

    Kv pareto: Systems-level optimization of kv cache and model compression for long context inference

    Sai Gokhale, Devleena Das, Rajeev Patwari, Ashish Sirasao, and Elliott Delaye. Kv pareto: Systems-level optimization of kv cache and model compression for long context inference. Journal, 2026. doi:10.18653/v1/2026.eacl-industry.9

    2026 doi

  70. [78]

    Minicache: Kv cache compression in depth dimension for large language models

    Gholamreza Haffari, Yefei He, Akide Liu, Jing Liu, Zizheng Pan, and Bohan Zhuang. Minicache: Kv cache compression in depth dimension for large language models. Journal, 2024. doi:10.52202/079017-4443

    2024 doi

  71. [79]

    The risk of kv cache compression

    Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno, Sebastian Trimpe, and Marco Pavone. The risk of kv cache compression. arXiv preprint, 2026

    2026

  72. [80]

    A survey on llm-as-a-judge

    Gu J, Jiang X, Shi Z, Tan H, Zhai X, Xu C, Li W, Shen Y, Ma S, Liu H, Wang S, Zhang K, Lin Z, Zhang B, Ni L, Gao W, Wang Y, and Guo J. A survey on llm-as-a-judge. PMC, 2026. doi:10.1016/j.xinn.2025.101253

    2026 doi

  73. [81]

    Key, value, compress: A systematic exploration of kv cache compression techniques

    Neusha Javidnia, Bita Darvish Rouhani, and Farinaz Koushanfar. Key, value, compress: A systematic exploration of kv cache compression techniques. Journal, 2025. doi:10.1109/cicc63670.2025.10983416

    2025 doi

  74. [82]

    Towards efficient large language model serving: A survey on system-aware kv cache optimization

    Jiantong Jiang, PeiYu Yang, Rui Zhang, and Feng Liu. Towards efficient large language model serving: A survey on system-aware kv cache optimization. Journal, 2025. doi:10.18653/v1/2026.findings-acl.1916

    2025 doi

  75. [83]

    Fastkv: Decoupling of context reduction and kv cache compression for prefill-decoding acceleration

    Dongwon Jo, Jiwon Song, Yulhwa Kim, and Jae-Joon Kim. Fastkv: Decoupling of context reduction and kv cache compression for prefill-decoding acceleration. Journal, 2026. doi:10.18653/v1/2026.findings-acl.1610

    2026 doi

  76. [84]

    Q-kvcomm: Efficient multi-agent communication via adaptive kv cache compression

    Boris Kriuk and Logic Ng. Q-kvcomm: Efficient multi-agent communication via adaptive kv cache compression. Journal, 2026. doi:10.1109/icmsci67830.2026.11469367

    2026 doi

  77. [85]

    Development of a rapid deep-learning-assisted multiphase multicomponent numerical simulation protocol

    Fang L, Sun Q, Xu Q, and Li X. Development of a rapid deep-learning-assisted multiphase multicomponent numerical simulation protocol. PMC, 2026. doi:10.1021/acsomega.6c03179

    2026 doi

  78. [86]

    A survey on large language model acceleration based on kv cache management

    Haoyang Li, Yiming Li, Anxin Tian, Tianhao Tang, ZhaoDong Xu, X Chen, Ning Hu, Wei Dong, Qing Li, and Lei Chen. A survey on large language model acceleration based on kv cache management. Journal, 2024. doi:10.48550/arxiv.2412.19442

    2024 doi arXiv

  79. [87]

    Faedkv: Infinite-window fourier transform for unbiased kv cache compression

    Runchao Li, Yao Fu, Mu Sheng, Xianxuan Long, Haotian Yu, and Pan Li. Faedkv: Infinite-window fourier transform for unbiased kv cache compression. Journal, 2025. doi:10.18653/v1/2025.findings-emnlp.914

    2025 doi

  80. [88]

    Minicache: Kv cache compression in depth dimension for large language models

    Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. Minicache: Kv cache compression in depth dimension for large language models. arXiv preprint, 2024 a

    2024

  81. [89]

    Minicache: Kv cache compression in depth dimension for large language models

    Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. Minicache: Kv cache compression in depth dimension for large language models. Journal, 2024 b . doi:10.48550/arxiv.2405.14366

    2024 doi arXiv

  82. [90]

    Lye, Siddhartha Mishra, and Deep Ray

    Kjetil O. Lye, Siddhartha Mishra, and Deep Ray. Deep learning observables in computational fluid dynamics. arXiv preprint, 2019. doi:10.1016/j.jcp.2020.109339

    2019 doi

  83. [91]

    The rise of small language models in healthcare: A comprehensive survey

    Garg M, Raza S, Rayana S, Liu X, and Sohn S. The rise of small language models in healthcare: A comprehensive survey. PMC, 2026 a . doi:10.1016/j.cosrev.2026.100999

    2026 doi

  84. [92]

    Building large-scale english-romanian literary translation resources with open models

    Nadas M, Diosan L, Tomescu A, and Piscoran A. Building large-scale english-romanian literary translation resources with open models. PMC, 2026 b . doi:10.3389/frai.2026.1807431

    2026 doi

  85. [93]

    Compressing kv cache for long-context llm inference with inter-layer attention similarity

    Da Ma, Lu Chen, Situo Zhang, Yuxun Miao, Su Zhu, Zhi Chen, Hongshen Xu, Hanqi Li, Shuai Fan, Lei Pan, and Kai Yu. Compressing kv cache for long-context llm inference with inter-layer attention similarity. Journal, 2026. doi:10.1109/icassp55912.2026.11464826

    2026 doi

  86. [94]

    A multitask deep learning model for real-time deployment in embedded systems

    Miquel Marti and Atsuto Maki. A multitask deep learning model for real-time deployment in embedded systems. arXiv preprint, 2017

    2017

  87. [95]

    Multi-hardware benchmarking of open-source large language models with retrieval-augmented generation for mitsubishi fx-series plc instruction list code generation

    Yeh MF, Luo CC, and Lu CL. Multi-hardware benchmarking of open-source large language models with retrieval-augmented generation for mitsubishi fx-series plc instruction list code generation. PMC, 2026. doi:10.3390/s26113602

    2026 doi

  88. [96]

    Limit-cycle proliferation under parametric delayed feedback in a conductance-based neuron: Bifurcation landscape, orbit catalog, and capacity analysis

    Alhawarat MO, Alnsour AJ, Al-Husainy MAF, and Abdelnaby KM. Limit-cycle proliferation under parametric delayed feedback in a conductance-based neuron: Bifurcation landscape, orbit catalog, and capacity analysis. PMC, 2026. doi:10.3390/e28060678

    2026 doi

  89. [97]

    F-transformer: a federated transformer for efficient and privacy-preserving sequence generation

    Patel N, Brahmbhatt S, Ramoliya F, Vyas O, Nair A, Vyas T, Jadav NK, Tanwar S, and Alabdultif A. F-transformer: a federated transformer for efficient and privacy-preserving sequence generation. PMC, 2026. doi:10.1038/s41598-026-40881-0

    2026 doi

  90. [98]

    Attention is all you need for kv cache in diffusion llms

    Quan Nguyen-Tri, Mukul Ranjan, and Zhiqiang Shen. Attention is all you need for kv cache in diffusion llms. arXiv preprint, 2025

    2025

  91. [99]

    Learn to accumulate evidence from all training samples: Theory and practice

    Deep Pandey and Qi Yu. Learn to accumulate evidence from all training samples: Theory and practice. arXiv preprint, 2023

    2023

  92. [100]

    Polykv: A shared asymmetrically-compressed kv cache pool for multi-agent llm inference

    Ishan Patel and Ishan Joshi. Polykv: A shared asymmetrically-compressed kv cache pool for multi-agent llm inference. arXiv preprint, 2026. doi:10.5281/zenodo.19686729

    2026 doi

  93. [101]

    Knowledge packs: Zero-token knowledge delivery via kv cache injection

    Andrey Pustovit. Knowledge packs: Zero-token knowledge delivery via kv cache injection. arXiv preprint, 2026

    2026

  94. [102]

    Swiftkv: Fast prefill-optimized inference with knowledge-preserving model transformation

    Aurick Qiao, Zhewei Yao, Samyam Rajbhandari, and Yuxiong He. Swiftkv: Fast prefill-optimized inference with knowledge-preserving model transformation. Journal, 2025. doi:10.18653/v1/2025.emnlp-main.1306

    2025 doi

  95. [103]

    Mooncake: A kvcache-centric disaggregated architecture for llm serving

    Ruoyu Qin, Zheming Li, Weiran He, Mingxing Zhang, Yongwei Wu, Weimin Zheng, and Xinran Xu. Mooncake: A kvcache-centric disaggregated architecture for llm serving. Journal, 2024. doi:10.48550/arxiv.2407.00079

    2024 doi arXiv

  96. [104]

    Deep Ray, Orazio Pinti, and Assad A. Oberai. Deep learning and computational physics (lecture notes). arXiv preprint, 2023

    2023

  97. [105]

    Structural optimization principles for edge ai in motorsport telemetry

    Cadiz RJ and Rodriguez-Sela F. Structural optimization principles for edge ai in motorsport telemetry. PMC, 2026. doi:10.1038/s41598-026-49736-0

    2026 doi

  98. [106]

    Green prompt engineering for sustainable generative ai

    Podder S, Date H, and Murthy S. Green prompt engineering for sustainable generative ai. PMC, 2026. doi:10.1016/j.ese.2026.100684

    2026 doi

  99. [107]

    Eigen attention: Attention in low-rank space for kv cache compression

    Utkarsh Saxena, Gobinda Saha, Sakshi Choudhary, and Kaushik Roy. Eigen attention: Attention in low-rank space for kv cache compression. Journal, 2024. doi:10.18653/v1/2024.findings-emnlp.899

    2024 doi

  100. [108]

    The kv4 trade-off is workload-dependent: A depth-and workload-resolved study of 4-bit kv-cache quantization on a 4 gb turing gpu

    Christopher Slothouber. The kv4 trade-off is workload-dependent: A depth-and workload-resolved study of 4-bit kv-cache quantization on a 4 gb turing gpu. Journal, 2026. doi:10.2139/ssrn.6941538

    2026 doi

  101. [109]

    Spindlekv: A novel kv cache reduction method balancing both shallow and deep layers

    Zicong Tang, Shi Luohe, Zuchao Li, Baoyuan Qi, Liu Guoming, Lefei Zhang, and Ping Wang. Spindlekv: A novel kv cache reduction method balancing both shallow and deep layers. Journal, 2025. doi:10.18653/v1/2025.acl-long.1380

    2025 doi

  102. [110]

    Wang, Boxiao Jin, Zhongzhi Yu, and Minjia Zhang

    Z. Wang, Boxiao Jin, Zhongzhi Yu, and Minjia Zhang. Model tells you where to merge: Adaptive kv cache merging for llms on long-context tasks. Journal, 2024. doi:10.48550/arxiv.2407.08454

    2024 doi arXiv

  103. [111]

    Sub-token routing for kv cache compression

    Wei, id\_orcid 0009-0003-9914-2657 Jiang, and Wei Wang. Sub-token routing for kv cache compression. Journal, 2026

    2026

  104. [112]

    Ss-adamoe: Spatio-spectral adaptive mixture of experts with global structural priors for graph node classification

    Kang X, Yu T, Wang L, Guo Y, and Zhang F. Ss-adamoe: Spatio-spectral adaptive mixture of experts with global structural priors for graph node classification. PMC, 2026 a . doi:10.3390/e28030355

    2026 doi

  105. [113]

    Towards sustainable ai knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources

    Shen X, Feng L, Hua S, Liu D, Xie Z, and Liu B. Towards sustainable ai knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources. PMC, 2026 b . doi:10.3389/fpsyg.2026.1843444

    2026 doi

  106. [114]

    Compute-in-memory implementation of state space models for event sequence processing

    Zhang X, Hu M, Lu S, Kim S, Lee EY, Liu Y, and Lu WD. Compute-in-memory implementation of state space models for event sequence processing. PMC, 2026 c . doi:10.1038/s41467-025-68227-w

    2026 doi

  107. [115]

    Chunkkv: Semantic-preserving kv cache compression for efficient long-context llm inference

    Liu Xiang, Zhenheng Tang, Peijie Dong, Zeyu Li, Yue Liu, Bo Li, Xuming Hu, and Xiaowen Chu. Chunkkv: Semantic-preserving kv cache compression for efficient long-context llm inference. Journal, 2025. doi:10.48550/arxiv.2502.00299

    2025 doi

  108. [116]

    Emily Xiao, C. H. Li, Yilin Zhang, Graham Neubig, and Amanda Bertsch. Efficient many-shot in-context learning with dynamic block-sparse attention. Journal, 2025. doi:10.18653/v1/2025.acl-long.1542

    2025 doi

  109. [117]

    Infrared imaging for autonomous power inspection: A review from detector to system integration

    Guo Y, Du Y, Mao R, Zhao Y, and Guo J. Infrared imaging for autonomous power inspection: A review from detector to system integration. PMC, 2026. doi:10.3390/s26113552

    2026 doi

  110. [118]

    Hcattention: Extreme kv cache compression via heterogeneous attention computing for llms

    Dongquan Yang, Yifan Yang, Xiaotian Yu, Xianbiao Qi, and Rong Xiao. Hcattention: Extreme kv cache compression via heterogeneous attention computing for llms. Journal, 2025. doi:10.2139/ssrn.5403127

    2025 doi

  111. [119]

    Hcattention: Extreme kv cache compression via heterogeneous attention computing for llms

    Dongquan Yang, Yifan Yang, Xiaotian Yu, Xianbiao Qi, and Rong Xiao. Hcattention: Extreme kv cache compression via heterogeneous attention computing for llms. Journal, 2026. doi:10.1016/j.neucom.2026.134247

    2026 doi

  112. [120]

    Kvsharer: Efficient inference via layer-wise dissimilar kv cache sharing

    Yifei Yang, Zouying Cao, Qiguang Chen, Libo Qin, Dongjie Yang, Hai Zhao, and Zhi Chen. Kvsharer: Efficient inference via layer-wise dissimilar kv cache sharing. Journal, 2024 a . doi:10.48550/arxiv.2410.18517

    2024 doi arXiv

  113. [121]

    Zhen Yang, J. N. Han, Kan Wu, Ruobing Xie, An Wang, Xingwu Sun, and Zhanhui Kang. Lossless kv cache compression to 2\ arXiv preprint, 2024 b

    2024

  114. [122]

    Lossless kv cache compression to 2\ Journal, 2024 c

    Zhen Yang, Jianwei Han, Kan Wu, Ruobing Xie, An Wang, Xingwu Sun, and Zhan Kang. Lossless kv cache compression to 2\ Journal, 2024 c . doi:10.48550/arxiv.2410.15252

    2024 doi arXiv

  115. [123]

    Tailorkv: A hybrid framework for long-context inference via tailored kv cache optimization

    Dingyu Yao, Bowen Shen, Zheng Lin, Wei Liu, Jian Luan, Bin Wang, and Weiping Wang. Tailorkv: A hybrid framework for long-context inference via tailored kv cache optimization. Journal, 2025. doi:10.18653/v1/2025.findings-acl.1043

    2025 doi

  116. [124]

    Evolkv: Evolutionary kv cache compression for llm inference

    Bohan Yu and Yekun Chai. Evolkv: Evolutionary kv cache compression for llm inference. Journal, 2025 a . doi:10.18653/v1/2025.findings-emnlp.88

    2025 doi

  117. [125]

    Evolkv: Evolutionary kv cache compression for llm inference

    Bohan Yu and Yekun Chai. Evolkv: Evolutionary kv cache compression for llm inference. arXiv preprint, 2025 b

    2025

  118. [126]

    Large-scale model-enhanced vision-language navigation: Recent advances, practical applications, and future challenges

    Li Z, Meng X, He X, Zhang Y, and Yin W. Large-scale model-enhanced vision-language navigation: Recent advances, practical applications, and future challenges. PMC, 2026. doi:10.3390/s26072022

    2026 doi

  119. [127]

    Kilohertz volumetric imaging of in vivo dynamics using squeezed light field microscopy

    Wang Z, Zhao R, Wagenaar DA, Espino D, Sheintuch L, Benshlomo O, Kang W, Zhu E, Lee CK, Schmidt WC, Pammar A, Wang J, Wong GCL, Liang R, Lee S, Lin MZ, Kannan M, Golshani P, Hsiai TK, and Gao L. Kilohertz volumetric imaging of in vivo dynamics using squeezed light field micros...

    2025 doi

  120. [128]

    Enhancing large multimodal models with adaptive sparsity and kv cache compression

    Te Zhang, Yuheng Li, Junxiang Wang, and Lujun Li. Enhancing large multimodal models with adaptive sparsity and kv cache compression. Journal, 2025. doi:10.1109/icme59968.2025.11209796

    2025 doi

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.