TailorKV combines 1-bit quantization in shallow attention layers with dynamic Top-K token retrieval in deeper layers to serve 128k-context Llama-3.1-8B on a single 24GB GPU with a small accuracy loss.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization
TailorKV combines 1-bit quantization in shallow attention layers with dynamic Top-K token retrieval in deeper layers to serve 128k-context Llama-3.1-8B on a single 24GB GPU with a small accuracy loss.