A post-training quantization scheme that uses per-group leading-one detection to keep four salient bits from an 8/16-bit integer base, achieving 4-bit weights, activations, and KV cache without fine-tuning or rotation.
A Note on Approximate Hadamard Matrices
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
A Hadamard matrix is a scaled orthogonal matrix with $\pm 1$ entries. Such matrices exist in certain dimensions: the Hadamard conjecture is that such a matrix always exists when $n$ is a multiple of 4. A conjecture attributed to Ryser is that no circulant Hadamard matrices exist when $n > 4$. Recently, Dong and Rudelson proved the existence of approximate Hadamard matrices in all dimensions: there exist universal $0< c < C < \infty$ so that for all $n \geq 1$, there is a matrix $A \in \left\{-1,1\right\}^{n \times n}$ satisfying, for all $x \in \mathbb{R}^n$, $$ c \sqrt{n} \|x\|_2 \leq \|Ax\|_2 \leq C \sqrt{n} \|x\|_2.$$ We observe that, as a consequence of the existence of flat Littlewood polynomials, circulant approximate Hadamard matrices exist for all $n \geq 1$.
citation-role summary
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
method 1polarities
support 1representative citing papers
citing papers explorer
-
Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoring
A post-training quantization scheme that uses per-group leading-one detection to keep four salient bits from an 8/16-bit integer base, achieving 4-bit weights, activations, and KV cache without fine-tuning or rotation.