Pith. sign in

REVIEW 3 major objections 3 minor 52 references

S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes S4R, a low-rank KV cache compressor that builds key/value subspaces from a sampled subset of prompt tokens and reconstructs only a sparse set during decoding, achieving up to 5× cache compression with near full-cache acc

desk verdict A solid, honest KV-compression paper that may overstate 'near full-cache accuracy' but is worth a serious look. read the letter →

arxiv 2608.00528 v1 pith:UBZHIR5K submitted 2026-08-01 cs.CL

Jialong Han , You Wu , Kewei Tu This is my paper
classification cs.CL
keywords KVcachecompressionlow-rankapproximationsingularvaluedecompositionlong-contextLLMinferenceattentionsinkssparsereconstructionlatent-spacerelevancescoringprompt-aware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

S4R claims that the dominant low-rank subspace of a long prompt's key/value cache can be recovered from a small, deliberately chosen sample of non-sink tokens — no external calibration data, no decomposition of the full prompt. That would dissolve the standard dilemma in low-rank KV compression: fixed post-training methods are cheap at serving time but brittle under distribution shift, while prompt-dependent methods adapt to the current context but pay a heavy prefilling cost. If the paper is right, a cheap sampled SVD at prefilling, followed by latent-space scoring and sparse reconstruction at each decoding step, delivers a KV cache about one-fifth the size of the full cache while staying close to full-cache accuracy. A reader should care because KV cache memory, not model parameters, is the practical bottleneck for long-context LLM serving.

What carries the argument

Truncated SVD over a sampled, token-wise-normalized subset of non-sink key/value states (Eqs. 1–2): the top-r right singular vectors B^{K/V} become the low-rank bases. Every non-sink token is stored as coefficients c = s·B (Eq. 3), so the persistent cache per layer is dominated by L_t·r instead of L_t·D'. Two mechanisms carry decoding: latent-space relevance scoring, which runs softmax attention between a sliding window of query coefficients and cached key coefficients (Eqs. 8–10) to pick reconstruction positions without materializing full keys, and sparse reconstruction, which rebuilds only the local window plus top-k global positions (Eqs. 13–16) before exact attention with uncompressed si

What would settle it

A direct test: over a corpus of long prompts, compare the top-r singular subspace built from S4R's sampled tokens against the subspace of the full prompt's KV states (e.g., via principal angles); then check whether large subspace distance coincides with S4R's accuracy drops. RULER's FWE task, where S4R scores 76.67 versus 83.33 for the full cache, is the paper's own candidate failure point. A behavioral variant: build a task whose answer sits in a low-attention mid-prompt region with atypical key/value norms; if S4R misses it while full-prompt SVD baselines recover it, the sampling representat

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that a prompt-dependent low-rank KV representation does not require the whole prompt. S4R keeps the first s sink tokens in full precision, samples T' ≪ T non-sink tokens by a 7:3 recent-to-uniform split, normalizes their states, and takes a truncated SVD of the sample to get low-rank bases. All non-sink keys and values are stored as r-dimensional coefficients in those bases. At decoding, queries are projected into the key subspace, relevance is scored in latent space over a sliding window, and only the local window plus top-k global positions are reconstructed for exact attention. Reported payoff: up to 5× KV compression with near full-cache LongBench a

Load-bearing premise

The load-bearing premise is that a small sample of non-sink tokens — about η of the prompt, split 7:3 between recent and uniformly spaced positions — spans the same low-rank key/value subspace as the entire prompt; the paper offers no bound, diagnostic, or failure criterion for when that sample misrepresents the cache.

Editorial extensions

If this is right

  • If the central claim holds, long-context serving can carry a persistent KV cache of roughly r/D' of its original size per layer (one-fifth at 5× compression), with prefilling cost far below full-prompt SVD methods such as xKV.
  • The 7:3 recent-to-uniform sampling rule becomes a reusable default: no calibration data and no full-prompt decomposition are needed to get prompt-adaptive bases, and the ablation shows pure-recent sampling (10:0) is measurably worse.
  • Keeping sink tokens uncompressed is load-bearing: the ablation removes them from the low-rank pipeline and loses 1.5–3.5 points on the two smaller models, so any simplification of the method must preserve this step.
  • Aggressive rank reduction survives multi-key and multi-query retrieval: on RULER, MK1, MK2, MQ, and MV all stay at 100% with the full cache at 2.5× compression.
  • Tuning the query window from 32 to 4 costs less than 0.1 points on Qwen3-4B and is the configuration behind the efficiency table, where S4R reaches 8.32 output tokens/s versus 1.96 for the full-prompt baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The FWE gap (76.67 vs 83.33 for full KV) marks where the sampled-subspace premise frays: tasks with scattered, low-prominence evidence are precisely those where latent-space scoring can overlook needed tokens. A diversity-aware or error-feedback sampling rule is the natural next test.
  • Because the paper shows a 1:1.5 key-to-value rank split helps Qwen but not Llama, rank allocation itself could be learned per layer or per head from a handful of prompts, likely buying accuracy at zero memory cost.
  • Nothing in the method couples the sample size to the true intrinsic rank of the KV states; if the subspace is low-rank to begin with, a much smaller T' than η·T should suffice, and the paper's own ablations do not sweep T'. Measuring accuracy versus T' would map the method's real sample efficiency.
  • Quantization is described as orthogonal; if the coefficient caches quantize as cleanly as raw KV states, memory could drop below the r/D' ceiling, but the coefficient distribution is a different object and needs its own calibration study.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes S4R, a two-stage low-rank KV cache compression method. During prefilling it preserves a small number of sink tokens in full precision, samples non-sink tokens with a hybrid 7:3 recent-to-uniform heuristic, normalizes the sampled states, and builds key/value subspaces via truncated SVD (Eqs. 1-2). All non-sink keys and values are then stored as low-rank coefficients (Eq. 3). During decoding it maintains a sliding query-coefficient window, scores cached tokens in latent space (Eqs. 8-10), selects a local window plus top-k global tokens (Eqs. 13-15), and reconstructs only the selected KV entries for exact attention (Eqs. 16-18). Memory analysis (Eqs. 19-24) shows the persistent cache ratio approaches r/D' for long contexts, and Appendix B derives the decoding complexity O(L_t(GWr + ρD'(r+G))). Experiments on LongBench and RULER across Llama and Qwen models report near full-cache accuracy under up to 5× compression, with substantially lower prefilling and decoding cost than the xKV baseline.

Significance. The central idea—replacing full-prompt SVD with SVD over a deliberately sampled subset of tokens—is attractive and, if validated, would improve the accuracy-efficiency trade-off of prompt-dependent low-rank KV compression. The memory and complexity analyses are straightforward and correct, and the evaluation is broad: two benchmarks, four model families, and extensive ablations. The method also makes a concrete, testable efficiency claim: it avoids the large prefilling cost of full-prompt online low-rank methods while retaining prompt adaptivity. However, the headline accuracy claim rests on an unproven representativeness assumption about the sampled subspace, and the RULER FWE result plus the Limitations section provide concrete evidence that the assumption can fail. The contribution is significant but conditional; strengthening the empirical or theoretical support for the sampled subspace and latent-space scoring would make the paper solid.

major comments (3)
  1. [§3.1.2, Eq. (2)] The load-bearing premise is that the rank-r subspace computed from T'=η·T sampled tokens with a fixed 7:3 recent/uniform split faithfully represents the full prompt's KV subspace. No sample-complexity bound, subspace-distance estimate, or diagnostic is provided. This is not merely a formal gap: Table 2 shows S4R at 76.67 on RULER FWE versus 83.33 for full KV and xKV, and the Limitations section concedes that 'tasks with scattered evidence (e.g., FWE) can still degrade when latent scoring misses relevant positions.' Because every LongBench and RULER result depends on this construction, the 'near full-cache accuracy' claim is conditional on an unquantified representativeness assumption. Please add a theoretical guarantee, a quantitative diagnostic (e.g., subspace coverage or agreement of selected tokens with full-KV selection), or an empirical demonstration that this failure mode is rare a
  2. [§3.2.2, Eq. (8)] The substitution of true attention logits by latent-space inner products Z = C_Q(C_K)^T / sqrt(r) is a second load-bearing assumption. The paper asserts that these latent scores rank tokens similarly to true attention, but no analysis or measurement of this agreement is given. The RULER FWE gap again suggests the approximation can fail precisely when relevant evidence is scattered. I ask for at least an empirical sanity check—e.g., the agreement rate between latent top-k and exact-attention top-k on a set of caches—and preferably a monitoring mechanism during decoding. Without such evidence, the sparse reconstruction set can silently omit tokens needed for the task.
  3. [§4.1, Tables 1-2] The paper does not report the target retention ratio η (equivalently the compression ratio) for S4R on each experimental setting. Section 4 states that 'the rank r is obtained by solving Eq. (21) for the target retention ratio η,' but Table 1 has no η column, and the RULER setting is described only as '2.5× compression.' Consequently the central 'up to 5×' claim and the memory ratio in Eq. (21) cannot be verified per model or per benchmark. Please report η, the resulting rank r, and the reconstruction ratio ρ for every configuration, including the efficiency experiment in Table 3.
minor comments (3)
  1. [§3.1.2] The notation eI and eQ for complements of index sets and for recent queries is nonstandard and visually confusing; please use explicit superscripts or a different symbol (e.g., complement notation I^c).
  2. [§4.3 / Tables 7-10] Several default hyperparameters (W=32, 7:3 split, pooling/aggregation choices, ρ=η) are selected based on LongBench ablations on the same benchmark reported as the main result. The ablations show robustness, but a separate validation split or a brief discussion of selection bias would strengthen the generalizability claim.
  3. [References / Tables] Reference 'LI et al., 2025' should be formatted as 'Li et al., 2025' for consistency. In Table 7, the entries in the W column are run together with the score column (e.g., '164.83' for W=1); please fix the table formatting so the window size is clearly separated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: accuracy claims are measured on external benchmarks, rank is set by the memory target (Eq. 21), and the only self-citation (GRKV) is a non-load-bearing related-work note.

full rationale

The paper's derivation chain is not circular. The core construction—sampling T' non-sink tokens, normalizing them, truncated SVD (Eq. 2), projecting all tokens into low-rank coefficients (Eq. 3), latent-space relevance scoring (Eq. 8), and sparse reconstruction (Eq. 16)—is an algorithmic definition, not a prediction derived from its own outputs. All accuracy claims are measured against external benchmarks (LongBench, RULER) and full-KV baselines. The compression ratio is an input target, not a fitted output: rank r is solved from Eq. (21) for a target retention ratio η, so the 'up to 5×' number is a chosen operating point. The only self-citation, GRKV (Peng et al., 2026), appears in Appendix C.1 as a related-work comparison ('GRKV formulates KV-cache merging as a global regression problem') and is not load-bearing for any S4R claim. The acknowledged limitation—'tasks with scattered evidence (e.g., FWE) can still degrade when latent scoring misses relevant positions'—with RULER FWE 76.67 vs. 83.33 full KV, is an empirical correctness risk about sample representativeness, not a circularity. Hyperparameter choices (W, 7:3 split) are tuned on the same benchmark and could inflate absolute numbers, but that is selection/overfitting, not definitional circularity. Therefore no specific circular step can be quoted; score 0 reflects the absence of circular derivation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. It relies on standard linear-algebra constructs (SVD, low-rank projection) and on empirical assumptions about KV structure and token relevance. The free parameters are collectively tuned on the evaluation benchmarks, which is the main factor keeping soundness at 5.

free parameters (6)
  • rank r = obtained by solving Eq. (21) for target retention ratio η
    Set to match a memory budget, not fit to accuracy, but it is a free design choice that trades compression against reconstruction error.
  • query window W = 32 (default), 4 in latency setting
    Chosen via ablation on LongBench (Table 7); the default is justified as near-optimal, and W=4 is used only in the efficiency table.
  • pooling kernel κ = 7
    Taken from SnapKV, but it is a hand-tuned smoothing hyperparameter that affects token selection.
  • recent-to-uniform sampling split = 7:3
    Chosen via ablation on LongBench (Table 9); models differ on the best split and no single split is universally optimal.
  • reconstruction ratio ρ = ρ = η (0.2 or 0.3)
    Set equal to the retention ratio by default, but the ablation in Table 10 shows performance varies with ρ and the default is a safety choice.
  • sink token count s = 4
    From the default setting of Xiao et al. (2024); the ablation in Table 4 shows the value affects accuracy.
assumptions (4)
  • domain assumption The KV cache exhibits exploitable low-rank structure.
    Cited to Yu et al. (2024) in Section 2; all low-rank methods rely on this empirical premise.
  • ad hoc to paper The low-rank subspace computed from a sample of T' tokens approximates the subspace of the full prompt.
    Section 3.1.2 constructs bases from a selective sample; no bound is given. This is the load-bearing assumption of the method.
  • ad hoc to paper Latent-space inner products Z = C_Q (C_K)^T / sqrt(r) rank tokens similarly to true attention scores.
    Section 3.2.2 uses this approximation for top-k selection; the FWE degradation shows it can fail.
  • domain assumption The first s tokens act as attention sinks and must be preserved in full precision.
    Follows from Xiao et al. (2024), and is verified by the ablation in Section 4.3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching." pith.science (2026). https://pith.science/paper/UBZHIR5K

@misc{pith2026260800528,
  author       = {Pith},
  title        = {Pith review of: S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UBZHIR5K}},
  note         = {Machine review of arXiv:2608.00528}
}
abstract

The growth of context window lengths in Large Language Models (LLMs) significantly enhances their long-context capabilities but incurs prohibitive memory costs due to the Key-Value (KV) cache. Although low-rank compression of KV cache is a promising remedy, existing methods face a dilemma: offline approaches depend on external calibration data, whereas online approaches incur substantial compute for full-prompt decomposition and reconstruction. In this paper, we propose S$^4$R, which builds low-rank subspaces from selectively sampled tokens and computes attention over a sparsely reconstructed KV representation. S$^4$R uses prompt-aware initialization to build initial key/value bases from a representative prompt subset, trading off calibration-data dependence against prefilling cost. Because fully reconstructing the cache at every decoding step is prohibitively expensive and hurts throughput, we further adopt sparse reconstruction to retain only informative positions during decoding. Extensive experiments on LongBench and RULER with Llama and Qwen model families show that S$^4$R achieves up to 5$\times$ KV compression with near full-cache accuracy, combining the efficiency of fixed compression with the adaptability of prompt-dependent methods.

Figures

Figures reproduced from arXiv: 2608.00528 by the authors.

Figure 1
Figure 1. Performance–speed trade-off on Qwen3-4B. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 26 canonical work pages

  1. [1]

    Kevin , booktitle =

    Feng, Yuan and Lv, Junlin and Cao, Yukun and Xie, Xike and Zhou, S. Kevin , booktitle =. Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference , url =

    work page

  2. [2]

    2024 , editor =

    Zhang, Yuxin and Du, Yuxuan and Luo, Gen and Zhong, Yunshan and Zhang, Zhenyu and Liu, Shiwei and Ji, Rongrong , booktitle =. 2024 , editor =

    work page 2024

  3. [3]

    2025 , eprint=

    CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing , author=. 2025 , eprint=

    work page 2025

  4. [4]

    Xiao, Guangxuan and Tang, Jiaming and Zuo, Jingwei and Guo, Junxian and Yang, Shang and Tang, Haotian and Fu, Yao and Han, Song , booktitle =

    work page

  5. [5]

    2025 , eprint=

    DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphones , author=. 2025 , eprint=

    work page 2025

  6. [6]

    2024 , eprint=

    Effectively Compress KV Heads for LLM , author=. 2024 , eprint=

    work page 2024

  7. [7]

    Proceedings of the 29th Symposium on Operating Systems Principles , pages =

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph and Zhang, Hao and Stoica, Ion , title =. Proceedings of the 29th Symposium on Operating Systems Principles , pages =. 2023 , isbn =. doi:10.1145/3600006.3613165 , abstract =

    arXiv 2023

  8. [8]

    Efficiently Scaling Transformer Inference , url =

    Pope, Reiner and Douglas, Sholto and Chowdhery, Aakanksha and Devlin, Jacob and Bradbury, James and Heek, Jonathan and Xiao, Kefan and Agrawal, Shivani and Dean, Jeff , booktitle =. Efficiently Scaling Transformer Inference , url =

    work page

Show all 52 references
  1. [9]

    The Twelfth International Conference on Learning Representations , year=

    Efficient Streaming Language Models with Attention Sinks , author=. The Twelfth International Conference on Learning Representations , year=

  2. [10]

    2024 , editor =

    Kang, Hao and Zhang, Qingru and Kundu, Souvik and Jeong, Geonhwa and Liu, Zaoxing and Krishna, Tushar and Zhao, Tuo , booktitle =. 2024 , editor =

    2024

  3. [11]

    H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , url =

    Zhang, Zhenyu and Sheng, Ying and Zhou, Tianyi and Chen, Tianlong and Zheng, Lianmin and Cai, Ruisi and Song, Zhao and Tian, Yuandong and R\'. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , url =. Advances in Neural Information Processin...

    2023

  4. [12]

    2024 , editor =

    Liu, Zirui and Yuan, Jiayi and Jin, Hongye and Zhong, Shaochen and Xu, Zhaozhuo and Braverman, Vladimir and Chen, Beidi and Hu, Xia , booktitle =. 2024 , editor =

    2024

  5. [13]

    2024 , eprint=

    KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head , author=. 2024 , eprint=

    2024

  6. [14]

    and Shao, Yakun Sophia and Keutzer, Kurt and Gholami, Amir , booktitle =

    Hooper, Coleman and Kim, Sehoon and Mohammadzadeh, Hiva and Mahoney, Michael W. and Shao, Yakun Sophia and Keutzer, Kurt and Gholami, Amir , booktitle =. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization , url =. doi:10.52202/079017-0040 , editor =

    doi

  7. [15]

    and Yun, Sangdoo and Song, Hyun Oh , booktitle =

    Kim, Jang-Hyun and Kim, Jinuk and Kwon, Sangwoo and Lee, Jae W. and Yun, Sangdoo and Song, Hyun Oh , booktitle =. KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction , url =. 2025 , address =

    2025

  8. [16]

    Loki: Low-rank Keys for Efficient Sparse Attention , url =

    Singhania, Prajwal and Singh, Siddharth and He, Shwai and Feizi, Soheil and Bhatele, Abhinav , booktitle =. Loki: Low-rank Keys for Efficient Sparse Attention , url =. 2024 , address =. doi:10.52202/079017-0532 , editor =

    2024 doi

  9. [17]

    2024 , eprint=

    LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy , author=. 2024 , eprint=

    2024

  10. [18]

    2024 , eprint=

    Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks , author=. 2024 , eprint=

    2024

  11. [19]

    2024 , author =

    Proceedings of the 62nd. 2024 , author =. doi:10.18653/v1/2024.acl-long.428 , url =

    2024 doi

  12. [20]

    2026 , eprint=

    OjaKV: Context-Aware Online Low-Rank KV Cache Compression , author=. 2026 , eprint=

    2026

  13. [21]

    Abdelfattah and Kai-Chiang Wu , booktitle=

    Chi-Chih Chang and Wei-Cheng Lin and Chien-Yu Lin and Chong-Yan Chen and Yu-Fang Hu and Pei-Shuo Wang and Ning-Chi Huang and Luis Ceze and Mohamed S. Abdelfattah and Kai-Chiang Wu , booktitle=. Palu:. 2025 , url=

    2025

  14. [22]

    2025 , eprint=

    PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling , author=. 2025 , eprint=

    2025

  15. [23]

    2024 , editor =

    Tang, Jiaming and Zhao, Yilong and Zhu, Kan and Xiao, Guangxuan and Kasikci, Baris and Han, Song , booktitle =. 2024 , editor =

    2024

  16. [24]

    2025 , eprint=

    ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration , author=. 2025 , eprint=

    2025

  17. [25]

    SALS: Sparse Attention in Latent Space for KV Cache Compression , url =

    Mu, Junlin and Huang, Hantao and Zhang, Jihang and Yu, Minghui and Wang, Tao and Li, Yidong , booktitle =. SALS: Sparse Attention in Latent Space for KV Cache Compression , url =. 2025 , address =

    2025

  18. [26]

    Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time , url =

    Liu, Zichang and Desai, Aditya and Liao, Fangshuo and Wang, Weitao and Xie, Victor and Xu, Zhaozhuo and Kyrillidis, Anastasios and Shrivastava, Anshumali , booktitle =. Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time , url =

  19. [27]

    2025 , editor =

    Sun, Hanshi and Chang, Li-Wen and Bao, Wenlei and Zheng, Size and Zheng, Ningxin and Liu, Xin and Dong, Harry and Chi, Yuejie and Chen, Beidi , booktitle =. 2025 , editor =

    2025

  20. [28]

    SnapKV: LLM Knows What You are Looking for Before Generation , url =

    Li, Yuhong and Huang, Yingbing and Yang, Bowen and Venkitesh, Bharat and Locatelli, Acyr and Ye, Hanchen and Cai, Tianle and Lewis, Patrick and Chen, Deming , booktitle =. SnapKV: LLM Knows What You are Looking for Before Generation , url =. 2024 , address =. doi:10.52202/0790...

    2024 doi

  21. [29]

    2025 , editor =

    Jie, Shibo and Tang, Yehui and Han, Kai and Deng, Zhi-Hong and Han, Jing , booktitle =. 2025 , editor =

    2025

  22. [30]

    A Survey on Large Language Model Acceleration based on

    Haoyang LI and Yiming Li and Anxin Tian and Tianhao Tang and Zhanchao Xu and Xuejia Chen and Nicole HU and Wei Dong and Li Qing and Lei Chen , journal=. A Survey on Large Language Model Acceleration based on. 2025 , url=

    2025

  23. [31]

    doi:10.48550/arXiv.2506.04642 , url =

    Joshi, Vinay and Brahma, Pratik Prabhanjan and Liu, Zicheng and Barsoum, Emad , date =. doi:10.48550/arXiv.2506.04642 , url =. 2506.04642 , eprinttype =

    doi arXiv

  24. [32]

    2025 , eprint=

    xKV: Cross-Layer SVD for KV-Cache Compression , author=. 2025 , eprint=

    2025

  25. [33]

    2024 , eprint=

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. 2024 , eprint=

    2024

  26. [34]

    2024 , eprint=

    GPT-4 Technical Report , author=. 2024 , eprint=

    2024

  27. [35]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  28. [36]

    2025 , eprint=

    Qwen2.5-1M Technical Report , author=. 2025 , eprint=

    2025

  29. [37]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  30. [38]

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , url =

    Lewis, Patrick and Perez, Ethan and Piktus, Aleksandra and Petroni, Fabio and Karpukhin, Vladimir and Goyal, Naman and K\". Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , url =. Advances in Neural Information Processing Systems , editor =. 2020 , address =

    2020

  31. [39]

    , title =

    Park, Joon Sung and O'Brien, Joseph and Cai, Carrie Jun and Morris, Meredith Ringel and Liang, Percy and Bernstein, Michael S. , title =. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , articleno =. 2023 , isbn =. doi:10.1145/3586183.36...

    2023

  32. [40]

    A survey on large language model based autonomous agents , volume=

    Wang, Lei and Ma, Chen and Feng, Xueyang and Zhang, Zeyu and Yang, Hao and Zhang, Jingsen and Chen, Zhiyuan and Tang, Jiakai and Chen, Xu and Lin, Yankai and Zhao, Wayne Xin and Wei, Zhewei and Wen, Jirong , year=. A survey on large language model based autonomous agents , vol...

    doi

  33. [41]

    2024 , url=

    Cheng-Ping Hsieh and Simeng Sun and Samuel Kriman and Shantanu Acharya and Dima Rekesh and Fei Jia and Boris Ginsburg , booktitle=. 2024 , url=

    2024

  34. [42]

    doi:10.5281/zenodo.12608602 , url =

    Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and...

    doi

  35. [43]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  36. [44]

    Bowen Peng and Jeffrey Quesnelle and Honglu Fan and Enrico Shippole , booktitle=. Ya. 2024 , url=

    2024

  37. [45]

    Peng, Junjie and Wu, You and Wu, Haoyi and Han, Jialong and Xie, Xiaohua and Tu, Kewei and Lai, Jianhuang , journal=

  38. [46]

    L ong B ench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks

    Bai, Yushi and Tu, Shangqing and Zhang, Jiajie and Peng, Hao and Wang, Xiaozhi and Lv, Xin and Cao, Shulin and Xu, Jiazheng and Hou, Lei and Dong, Yuxiao and Tang, Jie and Li, Juanzi. L ong B ench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multita...

    2025 doi

  39. [47]

    GQA : Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

    Ainslie, Joshua and Lee-Thorp, James and de Jong, Michiel and Zemlyanskiy, Yury and Lebron, Federico and Sanghai, Sumit. GQA : Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. Proceedings of the 2023 Conference on Empirical Methods in Natural La...

    2023 doi

  40. [48]

    Eigen Attention: Attention in Low-Rank Space for KV Cache Compression

    Saxena, Utkarsh and Saha, Gobinda and Choudhary, Sakshi and Roy, Kaushik. Eigen Attention: Attention in Low-Rank Space for KV Cache Compression. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024.findings-emnlp.899

    2024 doi

  41. [49]

    L ong B ench: A Bilingual, Multitask Benchmark for Long Context Understanding

    Bai, Yushi and Lv, Xin and Zhang, Jiajie and Lyu, Hongchang and Tang, Jiankai and Huang, Zhidian and Du, Zhengxiao and Liu, Xiao and Zeng, Aohan and Hou, Lei and Dong, Yuxiao and Tang, Jie and Li, Juanzi. L ong B ench: A Bilingual, Multitask Benchmark for Long Context Understa...

    2024 doi

  42. [50]

    P yramid I nfer: Pyramid KV Cache Compression for High-throughput LLM Inference

    Yang, Dongjie and Han, Xiaodong and Gao, Yan and Hu, Yao and Zhang, Shilin and Zhao, Hai. P yramid I nfer: Pyramid KV Cache Compression for High-throughput LLM Inference. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.195

    2024 doi

  43. [51]

    T a DA : Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering

    Joshi, Vinay and Brahma, Pratik Prabhanjan and Liu, Zicheng and Barsoum, Emad. T a DA : Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: I...

    2025 doi

  44. [52]

    TALE : Token-Adaptive Low-Rank KVC ache Approximation with Reconstruction Elimination

    Lee, Jaeseong and Hwang, Seung-won and Qiao, Aurick and Campos, Daniel and Yao, Zhewei and He, Yuxiong. TALE : Token-Adaptive Low-Rank KVC ache Approximation with Reconstruction Elimination. Transactions of the Association for Computational Linguistics. 2025. doi:10.1162/tacl.a.39

    2025 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.