LeoAM reports a 3.46x average latency speedup for long-context LLM inference on one commodity GPU by adaptively chunking KV data and loading compact key abstracts from disk instead of full KV values.
2025.{IMPRESS}: An{Importance-Informed}{ Multi-Tier} Prefix{KV} Storage System for Large Language Model Inference
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.OS 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU
LeoAM reports a 3.46x average latency speedup for long-context LLM inference on one commodity GPU by adaptively chunking KV data and loading compact key abstracts from disk instead of full KV values.