DiffKV compresses LLM KV caches with per-head, per-token, per-precision differentiation plus a parallel GPU memory manager, achieving 2.7x-5.7x compression at near-lossless accuracy.
https://huggingface.co/datasets/ AI-MO/aimo-validation-aime , 2024
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
dataset 1
citation-polarity summary
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1roles
dataset 1polarities
use dataset 1representative citing papers
citing papers explorer
-
DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
DiffKV compresses LLM KV caches with per-head, per-token, per-precision differentiation plus a parallel GPU memory manager, achieving 2.7x-5.7x compression at near-lossless accuracy.