XQuant caches quantized layer inputs and rematerializes Keys and Values on the fly, achieving up to 12.5x KV cache memory savings with about 0.1 perplexity degradation.
Gqa: Training generalized multi-query transformer models from multi-head checkpoints
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerialization
XQuant caches quantized layer inputs and rematerializes Keys and Values on the fly, achieving up to 12.5x KV cache memory savings with about 0.1 perplexity degradation.