A hierarchical KV cache for sparse-attention LLM serving bounds each request's GPU memory by a small LRU cache and fetches misses from host memory, raising long-context decoding throughput up to 4.7x with unchanged outputs.
MagicPIG: LSH sampling for efficient LLM generation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management
A hierarchical KV cache for sparse-attention LLM serving bounds each request's GPU memory by a small LRU cache and fetches misses from host memory, raising long-context decoding throughput up to 4.7x with unchanged outputs.