Block-GTQ performs RoPE-aware greedy bit allocation on KV caches using per-block energy scores, cutting logit MAE 32-80% versus uniform TQ-MSE and lifting long-context task scores substantially at 2-3 bits per dimension.
Accurate and efficient 2-bit kv cache quantization with dynamic channel-wise precision boost
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
fields
cs.LG 6years
2026 6representative citing papers
KVarN uses Hadamard rotation plus dual-axis variance normalization on K and V matrices to cut token-scale errors and error accumulation in KV-cache quantization, reaching new SOTA at 2-bit on MATH500, AIME24 and HumanEval.
OSCAR achieves near-BF16 accuracy for 2-bit KV cache quantization by using offline spectral covariance-aware rotations aligned with attention, plus a custom deployable INT2 kernel compatible with paged serving.
Spherical KV combines angle-domain attention using spherical key codes with rate-distortion retention to cut KV cache residency and HBM traffic while keeping a paged, fusion-friendly decode path.
RateQuant uses per-quantizer distortion calibration and reverse waterfilling from rate-distortion theory to optimally allocate mixed-precision bits across KV cache heads, resolving a failure mode where mismatched distortion models invert allocation order.
Token-wise INT4 KV-cache quantization plus block-diagonal Hadamard rotation recovers nearly all accuracy lost by naive INT4 while adding zero end-to-end overhead under paged serving constraints.
citing papers explorer
-
RoPE-Aware Bit Allocation for KV-Cache Quantization
Block-GTQ performs RoPE-aware greedy bit allocation on KV caches using per-block energy scores, cutting logit MAE 32-80% versus uniform TQ-MSE and lifting long-context task scores substantially at 2-3 bits per dimension.
-
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
KVarN uses Hadamard rotation plus dual-axis variance normalization on K and V matrices to cut token-scale errors and error accumulation in KV-cache quantization, reaching new SOTA at 2-bit on MATH500, AIME24 and HumanEval.
-
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
OSCAR achieves near-BF16 accuracy for 2-bit KV cache quantization by using offline spectral covariance-aware rotations aligned with attention, plus a custom deployable INT2 kernel compatible with paged serving.
-
SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inference
Spherical KV combines angle-domain attention using spherical key codes with rate-distortion retention to cut KV cache residency and HBM traffic while keeping a paged, fusion-friendly decode path.
-
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
RateQuant uses per-quantizer distortion calibration and reverse waterfilling from rate-distortion theory to optimally allocate mixed-precision bits across KV cache heads, resolving a failure mode where mismatched distortion models invert allocation order.
-
SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
Token-wise INT4 KV-cache quantization plus block-diagonal Hadamard rotation recovers nearly all accuracy lost by naive INT4 while adding zero end-to-end overhead under paged serving constraints.