DiffKV compresses LLM KV caches with per-head, per-token, per-precision differentiation plus a parallel GPU memory manager, achieving 2.7x-5.7x compression at near-lossless accuracy.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
DiffKV compresses LLM KV caches with per-head, per-token, per-precision differentiation plus a parallel GPU memory manager, achieving 2.7x-5.7x compression at near-lossless accuracy.