Pith. sign in

REVIEW 1 major objections 4 minor 3 cited by

Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries

T0 review · 1 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Lexico claims that a single ~4,000-atom dictionary per model can reconstruct LLM key-value caches as sparse combinations, compressing below 2-bit quantization.

desk verdict Novel sparse-coding KV cache method with promising results, but peak-memory and abstract claims need fixing. read the letter →

arxiv 2412.08890 v1 pith:VSIEYU2P submitted 2024-12-12 cs.LG

Junhyuck Kim , Jongho Park , Jaewoong Cho , Dimitris Papailiopoulos This is my paper · ORCID
classification cs.LG
keywords KVcachecompressionsparsecodingdictionarylearningorthogonalmatchingpursuitlargelanguagemodelslow-memoryinferencequantizationlongcontext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the key-value cache a Transformer must remember during generation is highly redundant: every key and value vector can be approximated as a sum of a few vectors drawn from one small dictionary shared by all inputs. Lexico trains that dictionary once per model on ordinary text, then uses orthogonal matching pursuit to write each cached vector as a sparse combination of dictionary atoms, storing only a handful of indices and coefficients. If the claim is right, LLM inference can operate at cache sizes that 2-bit quantization cannot reach, while keeping most of the model's accuracy on reasoning and long-context tasks. The practical payoff is that memory-limited GPUs can serve longer contexts and larger batches, and the compression ratio is tunable by simply choosing how many atoms per vector to allow.

What carries the argument

The load-bearing object is the universal per-layer dictionary $D \in \mathbb{R}^{m \times N}$ with $N \approx 4096$ unit-norm atoms, trained by gradient descent through the OMP encoder on WikiText-103 key/value activations. OMP greedily selects the $s$ atoms that minimize the relative $\ell_2$ reconstruction error $\|k - Dy\|_2 / \|k\|_2$, and the sparse coefficient vector $y$ is stored in CSR form as FP8 values plus int16 indices, costing $3s + 2$ bytes per vector instead of $2m$ bytes. The reconstruction identity $\hat K = K_{\mathrm{csr}}D_k^\top$ (likewise for values) is what converts the sparse cache back into usable attention states, and the factored attention computation $q_tD_kK_{\mathrm{csr}}^\top$ is what keeps decoding efficient. Because the dictionary is input-agnostic, its memory is constant and does not grow with batch size or sequence length.

What would settle it

Measure Lexico's relative reconstruction error $\|k - Dy\|_2 / \|k\|_2$ with sparsity $s = 32$ on a corpus far from WikiText-103, such as heavily formatted source code or non-English text. If the average error exceeds roughly 0.3 or downstream accuracy on that corpus falls below the 90-95% retention range reported for GSM8K, the universal-dictionary claim fails.

Watch

Extended reading notes

Core claim

The central discovery is empirical and structural: KV vectors generated from different prompts, tasks, and model families cluster near a small number of low-dimensional subspaces, so a per-layer overcomplete dictionary of roughly 4,096 atoms trained once on WikiText-103 can serve as a universal basis. Lexico encodes each key or value vector as a sparse linear combination of $s$ atoms found by orthogonal matching pursuit, stores the result as compressed sparse rows with 8-bit coefficients and 16-bit indices, and keeps a small buffer of recent tokens in full precision. Reconstruction is $\hat K = K_{\mathrm{csr}}D_k^\top$ and $\hat V = V_{\mathrm{csr}}D_v^\top$, and attention is computed through $q_tD_kK_{\mathrm{csr}}^\top$ so full keys need not be materialized. The reported result is that at 15-25% of full KV-cache memory, Lexico retains 90-95% of GSM8K accuracy across Mistral, Llama 3, and Qwen2.5, and in the low-memory regime it beats 2-bit quantization by up to 1.7x compression at similar accuracy.

Load-bearing premise

Every deployment's key and value vectors lie close to the union of low-dimensional subspaces spanned by one per-layer dictionary trained once on WikiText-103; otherwise reconstruction error rises and task accuracy drops sharply.

Editorial extensions

If this is right

  • Sparsity $s$ directly sets the cache budget, so one system can trade memory for accuracy continuously instead of jumping between 4-bit, 2-bit, and integer precisions.
  • In the low-memory regime below 20% of full cache, Lexico stays usable where 2-bit quantization cannot even represent the cache, extending deployable context lengths on a fixed GPU.
  • The universality claim means a dictionary trained once can be applied off-the-shelf to new prompts and tasks without per-input retraining, and the dictionary's constant memory is shared across all concurrent sessions.
  • Because Lexico keeps every token in compressed form rather than evicting tokens, long-context tasks that require attention over old tokens suffer less than eviction-based methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the subspace hypothesis holds for the target distribution, Lexico should combine multiplicatively with token eviction: dropping low-information tokens first and then sparse-coding the survivors would compound savings, a combination the paper only mentions as future work.
  • The error-threshold ablation suggests a testable refinement: allocate sparsity per token based on reconstruction error rather than a fixed $s$, which could hold a quality target while using less memory on easy tokens.
  • A cross-model experiment with a dictionary trained on one base model and applied to a different architecture would sharpen the universality claim, since current results show generalization across tasks but do not test transferring one model's dictionary to another model family.
  • The adaptive dictionary extension indicates a path to input-conditioned compression, where rare tokens get extra atoms at inference time; whether the accuracy gain outweighs the added cache is a direct open question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. The paper introduces Lexico, a post-training KV-cache compression method based on sparse coding over a learned universal dictionary. Per-layer key and value dictionaries of ~4k atoms are trained once on WikiText-103, and at inference each KV vector is encoded with Orthogonal Matching Pursuit into a sparse combination of atoms, stored in CSR format with FP8 values and int16 indices, together with a small full-precision buffer of recent tokens. The authors claim that this method compresses the KV cache to 15-25% of its original size while preserving 90-95% of original model performance, and that it can operate below 2-bit quantization memory levels. They evaluate on LongBench, GSM8K, MMLU-Pro and reconstruction error across multiple Llama/Mistral/Qwen models, comparing against KIVI, ZipCache, per-token quantization, SnapKV and PyramidKV, and report Pareto improvements. The paper also includes ablations on error thresholding, buffer sizing, and an adaptive dictionary extension, plus latency measurements.

Significance. The core idea—treating KV vectors as lying near a union of low-dimensional subspaces captured by a universal dictionary—is original and, if the memory claims survive scrutiny, practically important for serving long-context LLMs on memory-constrained GPUs. The paper is honest about the dictionary-learning details, provides a released implementation, and evaluates on a reasonable set of models and tasks. The central hypothesis (dictionary universality across inputs) is directly tested on out-of-domain corpora, which is a strength. However, the abstract overstates the operating points where 'high accuracy' is maintained, and the decoding algorithm as written appears to materialize a full-precision value cache, which would undermine the advertised memory savings. These need to be fixed before the paper is publishable.

major comments (1)
  1. [Section 4.1 and Figure 6] The claim that Lexico 'outperforms any other baseline' in low-memory regimes is supported only by a narrow set of tasks: 8 LongBench tasks, GSM8K, and 2 MMLU-Pro subjects. The universality claim in the abstract and Section 1 is stronger than the evidence: the dictionary is trained on WikiText-103 alone, and while Table 1 shows reconstruction generalization on four corpora, downstream generalization is demonstrated on a small set of benchmarks. This is not a fatal flaw, but the paper should temper the 'universal' language or add a broader task suite, especially since the only MMLU-Pro results are in Figure 6 for two subjects and the improvements over quantization baselines there are marginal at best. Please either add more diverse evaluation tasks or soften the universality claim to match the evidence.
minor comments (4)
  1. [Throughout] There are several typos and grammatical issues: 'applicaiton' (Section 1), 'the having to maintain' (Section 1), 'identitcal' (Section 1), 'accross' (Section 1 and 3.2), 'ℓ2 norm' spacing, and 'a key-value cache' missing article. A careful proofread is advised.
  2. [Table 2] The table header lists 'Llama-3.1-8B-Instruct' and 'Mistral-7B-Instruct-v0.3', but the first row for each model is labeled 'Full Cache' without specifying FP16; this is inconsistent with the caption and other tables (e.g., Table 3 says 'Full cache is in FP16'). Please make the precision explicit in all tables.
  3. [Section 3.3] The dictionary size is N=4096 in most experiments, but Table 1 and Figure 7 use N=1024. The paper reports memory for N=1024 (16.8MB) but not for N=4096. Since N=4096 is the default in the main results, please report the dictionary storage overhead for N=4096 as well, or clarify that the overhead is negligible relative to the KV cache savings.
  4. [Section 4.2.4, Table 6] The 'w/o Adaptation' row reports KV size 34.4% for both Llama and Mistral, but the same configuration (N=1024, s=16, nb=128, FP16 values) yields different KV sizes depending on the sequence length distribution of GSM8K. Please specify the average sequence length used for memory reporting in GSM8K, or provide a formula so the numbers can be reproduced.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: dictionary is trained on reconstruction loss and evaluated on held-out tasks at memory-matched sparsity; no prediction reduces to a fitted parameter or self-citation.

full rationale

The paper's derivation chain is self-contained and empirically evaluated rather than circular. The dictionaries are trained once on WikiText-103 by minimizing the l2 reconstruction error of KV vectors with an OMP encoder (Section 3.3), and the same encoder is then applied at inference. The central claim—that a small per-model dictionary can approximate KV caches across tasks—is tested on held-out corpora (Table 1: CNN/DailyMail, IMDB, TweetEval) and on downstream tasks (LongBench, GSM8K, MMLU-Pro) that are not used in dictionary training. Crucially, the sparsity level s is explicitly set to match the KV size of the baseline (Section 4, 'Hyperparameter settings'), so the reported accuracy at a given memory budget is not obtained by tuning sparsity to maximize downstream accuracy; it is a comparison at matched memory. No equation defines the claimed 'compression' or 'universality' in terms of the training objective itself, and no fitted parameter is renamed as a prediction. The paper contains no load-bearing self-citations: the reference list includes no prior works by the present authors that are invoked to justify the method. The skeptical concern in the reader's take—that Algorithm 2 materializes Dv*Vcsr and therefore transient peak memory may exceed the advertised persistent cache size—is a potential correctness/implementation issue about memory accounting, not a circularity of the derivation. It does not make the result equivalent to its inputs, so it is outside the circularity score. Given that the evaluation is external, memory-matched, and the method is a straightforward application of dictionary learning with a held-out test protocol, the appropriate circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method rests on a learned dictionary (its primary fitted object), several hand-chosen hyperparameters (sparsity, buffer size, dictionary size, approximation window, coefficient precision), and the domain assumption that KV vectors live near low-dimensional subspaces shared across inputs. No new physical or mathematical entities are introduced; the universal dictionary is a learned parameter matrix listed above.

free parameters (6)
  • Dictionary atoms D_k, D_v per layer = N = 4096 atoms of dimension head size (128)
    Learned on WikiText-103 KV activations via OMP and gradient descent to minimize reconstruction error; this is the primary fitted object of the method.
  • Sparsity s = Varies per experiment (4, 8, 10, 14, 16, 20, 24, 32)
    Chosen to match the KV size of the quantization baseline at each comparison point; acts as the compression control knob.
  • Buffer size n_b = 128 for main experiments; varied in ablation
    Hand-picked hyperparameter; ablations (Table 5, Figure 7) show performance depends strongly on it.
  • Dictionary size N = 1024 for ablations, 4096 for main experiments
    Chosen by hand; no systematic ablation of N on task performance is reported.
  • Approximation window n_a = 1 for main experiments, 8 in latency analysis
    Controls how many oldest buffer tokens are compressed per decode step; chosen for implementation convenience.
  • CSR precision = FP8(E4M3) values, int16 indices
    Design choice to reduce coefficient memory from FP16 to 8 bits; paper states minimal degradation.
assumptions (4)
  • standard math OMP provides a sufficiently accurate solution to the NP-hard sparse approximation problem (Equation 3).
    The paper uses OMP without optimality guarantees; if OMP's greedy selection fails for some KV vectors, compression quality degrades.
  • domain assumption Keys and values across different inputs lie in a union of low-dimensional subspaces that can be captured by one universal per-layer dictionary.
    Stated as a hypothesis in Section 3.2 and used to justify training a single dictionary on WikiText-103 for all downstream tasks.
  • domain assumption Small l2 reconstruction error in keys and values translates to small loss in downstream task accuracy.
    Implicit in Sections 3.3 and 3.4 and the choice of loss function; if attention is highly sensitive to certain reconstruction errors, the method could fail even with low l2 error.
  • domain assumption The full-precision buffer of the most recent tokens is retained, and its memory is included in the reported KV size.
    Ablations in Section 4.2.3 show the buffer is critical; the method's memory claims depend on buffer size being small relative to sequence length.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries." pith.science (2026). https://pith.science/paper/VSIEYU2P

@misc{pith2026241208890,
  author       = {Pith},
  title        = {Pith review of: Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VSIEYU2P}},
  note         = {Machine review of arXiv:2412.08890}
}
read the original abstract

We introduce Lexico, a novel KV cache compression method that leverages sparse coding with a universal dictionary. Our key finding is that key-value cache in modern LLMs can be accurately approximated using sparse linear combination from a small, input-agnostic dictionary of ~4k atoms, enabling efficient compression across different input prompts, tasks and models. Using orthogonal matching pursuit for sparse approximation, Lexico achieves flexible compression ratios through direct sparsity control. On GSM8K, across multiple model families (Mistral, Llama 3, Qwen2.5), Lexico maintains 90-95% of the original performance while using only 15-25% of the full KV-cache memory, outperforming both quantization and token eviction methods. Notably, Lexico remains effective in low memory regimes where 2-bit quantization fails, achieving up to 1.7x better compression on LongBench and GSM8K while maintaining high accuracy.

Figures

Figures reproduced from arXiv: 2412.08890 by the authors.

Figure 1
Figure 1. Memory usage vs. performance of Lexico compared to other key-value (KV) cache compression methods on GSM8K. The figure illustrates the relationship between KV cache size and the performance of Lexico on Llama models on GSM8K 5-shot evaluation. For Lexico, we use a dictionary size of N = 4096 atoms and keep the last 128 tokens in full-precision (buffer size nb = 128). Lexico consistently outperforms both eviction-bas… view at source ↗
Figure 2
Figure 2. (a) Prefilling: Following attention computation, Lexico uses OMP to find sparse repre￾sentations of the KV vectors (3-8× smaller). (b) Decoding: Key cache consists of the compressed sparse key cache, Kcsr, and an full-precision buffer, Kbuffer, for the most recent tokens. qt, kt rep￾resent the query, key vectors for the newly generated token. Computation is reduced by computing the query-dictionary product, qtDk, th… view at source ↗
Figure 3
Figure 3. Left shows a pairwise cosine similarity matrix between key vectors generated from one input text from all heads in Layer 10 of Llama-3.1-8B-Instruct. Keys are sorted by similarity to demonstrate the clusters. Right shows the similarity matrix between key vectors from two different input texts. These plots indicate that there may exist a mixture of low-dimensional subspaces in the space of all possible keys, a hypoth… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Dictionary Learning of Lexico. We train a linear layer D (our dictionary) that minimizes ℓ2-reconstruction error of KV cache. The KV cache of layer i are used as training data for dictionary D(i) . Each step, we apply OMP with fixed D to represent KV as a vector of spa…
Figure 5
Figure 5. Figure 5: Memory usage vs. performance of Qwen2.5- 14B-Instruct with Lexico on GSM8K. We compare the performance of Lexico against quantization methods on Qwen2.5- 14B-Instruct, with its weights quantized to 4 bits. For Lexico, we use N = 4096 as the dictionary size and nb = 128…
Figure 7
Figure 7. Figure 7: Memory usage vs. performance of Lexico with and without buffer on LongBench and GSM8K. The figure illustrates the impact of removing the buffer on the performance of Lexico when evaluated on the Llama-3.1-8B-Instruct and Mistral-7B-Instruct models for LongBench (left) …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PolarQuant: Quantizing KV Caches with Polar Transformation

    cs.LG 2025-02 conditional novelty 7.0 of 10

    PolarQuant achieves around 4x KV cache compression for LLMs by quantizing angles after a recursive polar transform with random preconditioning, with LongBench scores close to the full-precision model and above prior c...

  2. Cartridges: Lightweight and general-purpose long context representations via self-study

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A per-corpus trained KV cache, called a Cartridge, matches full-context in-context learning quality on long-document benchmarks while using up to 38.6x less serving memory.

  3. ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution

    cs.LG 2026-07 conditional novelty 5.0 of 10

    ZUNA1.1, an open-source 380M EEG diffusion autoencoder, reconstructs variable-length, flexibly masked EEG at least as well as its predecessor and far better than spherical spline interpolation.

Reference graph

Works this paper leans on

25 extracted references · 2 canonical work pages · cited by 3 Pith papers

  1. [1]

    Gqa: Training generalized multi-query transformer models from multi-head check- points

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head check- points. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro- cessing, pp. 4895–4901,

    2023

  2. [3]

    Longformer: The long-document transformer

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150,

    arXiv 2004

  3. [4]

    Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, et al. Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling. arXiv preprint arXiv:2406.02069,

    arXiv

  4. [8]

    Zipcache: Accurate and efficient kv cache quantization with salient token identification

    Yefei He, Luoming Zhang, Weijia Wu, Jing Liu, Hong Zhou, and Bohan Zhuang. Zipcache: Accurate and efficient kv cache quantization with salient token identification. arXiv preprint arXiv:2405.14256,

    arXiv

  5. [9]

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami

    13 Preprint. Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. Kvquant: Towards 10 million context length llm inference with kv cache quantization. arXiv preprint arXiv:2401.18079,

    arXiv

  6. [10]

    Gear: An efficient kv cache compression recipefor near-lossless generative inference of llm

    Hao Kang, Qingru Zhang, Souvik Kundu, Geonhwa Jeong, Zaoxing Liu, Tushar Krishna, and Tuo Zhao. Gear: An efficient kv cache compression recipefor near-lossless generative inference of llm. arXiv preprint arXiv:2403.05527,

    arXiv

  7. [11]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

    arXiv

  8. [12]

    Snapkv: Llm knows what you are looking for before generation

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: Llm knows what you are looking for before generation. arXiv preprint arXiv:2404.14469,

    arXiv

Show all 25 references
  1. [13]

    Infinite-llm: Efficient llm service for long context with distattention and distributed kvcache

    Bin Lin, Tao Peng, Chen Zhang, Minmin Sun, Lanbo Li, Hanyu Zhao, Wencong Xiao, Qi Xu, Xiafei Qiu, Shen Li, et al. Infinite-llm: Efficient llm service for long context with distattention and distributed kvcache. arXiv preprint arXiv:2401.02669,

    arXiv

  2. [14]

    Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time.Advances in Neural Information Processing Systems, 36, 2024a

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time.Advances in Neural Information Processi...

    arXiv

  3. [15]

    K-sparse autoencoders.arXiv preprint arXiv:1312.5663,

    Alireza Makhzani and Brendan Frey. K-sparse autoencoders.arXiv preprint arXiv:1312.5663,

    arXiv

  4. [16]

    arXiv preprint arxiv:2407.00079,

    arXiv

  5. [17]

    Fast transformer decoding: One write-head is all you need

    Noam Shazeer. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150,

    1911 arXiv

  6. [19]

    You only cache once: Decoder-decoder architectures for language models

    Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, and Furu Wei. You only cache once: Decoder-decoder architectures for language models. arXiv preprint arXiv:2405.05254,

    arXiv

  7. [20]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, p...

    2017

  8. [22]

    Chunkattention: Efficient self-attention with prefix-aware kv cache and two-phase partition

    Lu Ye, Ze Tao, Yong Huang, and Yang Li. Chunkattention: Efficient self-attention with prefix-aware kv cache and two-phase partition. arXiv preprint arXiv:2402.15220,

    arXiv

  9. [24]

    Wkvquant: Quantizing weight and key/value cache for large language models gains more

    Yuxuan Yue, Zhihang Yuan, Haojie Duanmu, Sifan Zhou, Jianlong Wu, and Liqiang Nie. Wkvquant: Quantizing weight and key/value cache for large language models gains more. arXiv preprint arXiv:2402.12065,

    arXiv

  10. [2006]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168,

    arXiv

  11. [2017]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al

    14 Preprint. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574, 2024a. Zh...

    arXiv

  12. [2019]

    Loki: Low-rank keys for efficient sparse attention

    Prajwal Singhania, Siddharth Singh, Shwai He, Soheil Feizi, and Abhinav Bhatele. Loki: Low-rank keys for efficient sparse attention. arXiv preprint arXiv:2406.02542,

    arXiv

  13. [2020]

    APPENDIX A I MPLEMENTATION DETAILS Algorithm 1 illustrates a naive implementation of OMP for understanding

    15 Preprint. APPENDIX A I MPLEMENTATION DETAILS Algorithm 1 illustrates a naive implementation of OMP for understanding. In Lexico, we adopt the implementation of OMP v0 proposed by (Zhu et al., 2020), which minimizes computational complexity using efficient inverse Cholesky f...

    2020

  14. [2021]

    Deepseekmoe: Towards ultimate expert specialization in mixture- of-experts language models

    Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, et al. Deepseekmoe: Towards ultimate expert specialization in mixture- of-experts language models. arXiv preprint arXiv:2401.06066,

    arXiv

  15. [2022]

    Effectively compress kv heads for llm

    Hao Yu, Zelan Yang, Shen Li, Yong Li, and Jianxin Wu. Effectively compress kv heads for llm. arXiv preprint arXiv:2406.07056,

    arXiv

  16. [2023]

    Longbench: A bilingual, multitask benchmark for long context understanding

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding. arXiv preprint arXiv:2308.14508,

    arXiv

  17. [2024]

    A simple and effective l 2 norm-based strategy for kv cache compression

    Alessio Devoto, Yu Zhao, Simone Scardapane, and Pasquale Minervini. A simple and effective l 2 norm-based strategy for kv cache compression. arXiv preprint arXiv:2406.11430,

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.