A hierarchical KV cache for sparse-attention LLM serving bounds each request's GPU memory by a small LRU cache and fetches misses from host memory, raising long-context decoding throughput up to 4.7x with unchanged outputs.
FlashAttention-2: Faster attention with better parallelism and work partitioning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.DC 1years
2026 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management
A hierarchical KV cache for sparse-attention LLM serving bounds each request's GPU memory by a small LRU cache and fetches misses from host memory, raising long-context decoding throughput up to 4.7x with unchanged outputs.