A method that uses a handful of 'semantic retrieval heads' instead of all attention heads to decide which key-value cache entries can be dropped, plus layer-wise cache budgeting, reportedly beats prior KV compression baselines on LongBench and Needle-in-a-Haystack.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation
A method that uses a handful of 'semantic retrieval heads' instead of all attention heads to decide which key-value cache entries can be dropped, plus layer-wise cache budgeting, reportedly beats prior KV compression baselines on LongBench and Needle-in-a-Haystack.