Pith. sign in

REVIEW 2 references

Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers

T0 review · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Keyless Attention eliminates the key projection, computing softmax(QV^T)V and reducing KV-cache memory by exactly 50%, with comparable or better performance on small-scale models.

arxiv 2606.21848 v3 pith:7WBXE6Z5 submitted 2026-06-20 cs.CL cs.AI

Xin Gao , Xingming Xu This is my paper
classification cs.CLcs.AI
keywords attentionkeylessroutingcachememoryprojectionvalue-spacearchitectures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard transformers use three projections—queries, keys, and values—to compute attention. The keys determine which tokens are relevant to the current token, and the values carry the content that gets mixed. During generation, both keys and values are stored in a cache, which takes up memory and bandwidth. Keyless Attention removes keys entirely, using only the values to compute attention scores: each token's query is compared directly with all past value vectors, and those values are then weighted and summed. This halves the cache size because only values are stored, not keys.

The paper shows a linear-algebra result: in some cases, any standard attention layer can be exactly simulated by a keyless layer by adjusting the query projection. But that equivalence requires a mathematical condition that is not guaranteed in real multi-head models, so the main support is empirical. The authors trained five small to medium language models (up to 1.5B parameters) from scratch on a tiny 30M-token subset of WikiText-103, comparing keyless versus standard attention. They report that keyless matches or slightly outperforms the baseline on perplexity and on four of five zero-shot reasoning tasks, with the cache size cut in half. They also observe that keyless attention overfits less after the best epoch, a claim they attribute to 'gradient entanglement' between query and value projections, though they leave a direct test of this mechanism for future work.

Extended reading notes

Core claim

Keyless Attention matches or outperforms standard QKV attention on perplexity in 4 out of 5 models, and outperforms on 4 out of 5 zero-shot downstream benchmarks, while reducing KV cache memory by exactly 50% during autoregressive inference.

Load-bearing premise

The multi-head equivalence theorem (Theorem 2) requires the subspace condition col(Ω_h^T) ⊆ col(W_V_h) for each head, which is not generically satisfied in practical multi-head attention. The empirical results additionally assume that behavior on a 30M-token subset of WikiText-103 transfers to realistic large-scale pretraining—an assumption not validated in this paper.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. It relies on standard linear algebra and an untested scaling assumption.

free parameters (1)
  • factorization depth m = 3
    The paper chooses m=3 (QVV) after an ablation, but treats m as a tunable hyperparameter; it is not derived from first principles.
assumptions (2)
  • ad hoc to paper Full rank of W_V (single-head) or the subspace condition col(Ω_h^T) ⊆ col(W_V_h) (multi-head) for equivalence theorems.
    Theorem 1 requires W_V square and full rank; Theorem 2 requires condition (7). These conditions do not hold generically in multi-head attention with d_k < d.
  • domain assumption Training on a 30M-token subset of WikiText-103 approximates real pretraining conditions.
    All experiments use this tiny subset; the abstract's claim about 'industrial settings' is unsupported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers." pith.science (2026). https://pith.science/paper/7WBXE6Z5

@misc{pith2026260621848,
  author       = {Pith},
  title        = {Pith review of: Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7WBXE6Z5}},
  note         = {Machine review of arXiv:2606.21848}
}
read the original abstract

Transformer architectures form the foundation of modern natural language processing, yet the Key-Value (KV) cache introduces substantial memory and bandwidth overhead during long-context generation, increasingly bottlenecking large-scale deployment. We propose Keyless Attention, a novel attention mechanism that replaces the conventional key projection with a dedicated value-space routing projection, eliminating key representations from the attention computation entirely and yielding a Value-Only Cache that reduces KV-cache memory by 50% while improving decode throughput. Experiments across multiple models and architectures demonstrate that Keyless Attention achieves comparable perplexity and downstream task performance to standard QKV attention, while consistently reducing KV-cache memory by 50%. Furthermore, Keyless Attention exhibits slower validation loss degradation after the best epoch, indicating improved robustness against overfitting. Ablation studies confirm that the dedicated value-space routing projection is critical, with Keyless Attention outperforming KV-sharing methods that eliminate the key cache without replacing its routing role. Experiments in the pretraining regime further confirm the viability of Keyless Attention in industrial settings.

Figures

Figures reproduced from arXiv: 2606.21848 by the authors.

Figure 1
Figure 1. Training dynamics of QKV and QVV(3) across GPT-2 model depths. Top row: 12-layer (280M parameters); [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Validation loss comparison of QVV and QKV variants across factorization depths [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Perplexity over 4 epochs across three GQA architectures. Keyless Attention matches or outperforms standard [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Decode throughput and KV cache size for Keyless vs. QKV attention under GQA. Left: decode throughput (mean ˘ std over 3 seeds) as a function of context length; Keyless exceeds QKV. Right: KV cache size vs Value-Only Cache size; Keyless reduces cache memory by exactly 5…

Review history (3 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 linked inside Pith

  1. [1]

    Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

    AI at Meta (2024). Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. Meta AI Blog. Ainslie, J., Lee-Thorp, J., de Jong, M., Zemlyanskiy, Y ., Lebrón, F., and Sanghai, S. (2023). GQA: Training generalized multi-query transformer models from multi-head checkpoints. InProceedings of the 2023 Conference on Empirical Methods in Natu...

    arXiv 2024

  2. [5]

    Prabhu, R., Nayak, A., Mohan, J., Ramjee, R., and Panwar, A. (2024). vAttention: Dynamic memory management for serving LLMs without PagedAttention.arXiv preprint arXiv:2405.04437. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language models are unsupervised multitask learners.OpenAI Blog, 1(8):9. Shazeer, N. (2019). Fast...

    arXiv 2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.