RetroInfer introduces the wave index and wave buffer to realize sparse KV-cache attention for long-context LLM inference with up to 4.4X throughput gains while matching full-attention accuracy.
23 Herve Jegou, Matthijs Douze, and Cordelia Schmid
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
citation-role summary
baseline 1
citation-polarity summary
roles
baseline 1polarities
baseline 1representative citing papers
Unifies LSH and LSF for angular ANN, designs an LSF data structure, and proves its optimality via streamlined analysis of prior techniques.
citing papers explorer
-
RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference
RetroInfer introduces the wave index and wave buffer to realize sparse KV-cache attention for long-context LLM inference with up to 4.4X throughput gains while matching full-attention accuracy.
-
A Tour of Locality Sensitive Filtering on the Sphere
Unifies LSH and LSF for angular ANN, designs an LSF data structure, and proves its optimality via streamlined analysis of prior techniques.