Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs

Cyril Moineau; Fabrice Rastello; Lucas Maisonnave; Olivier Bichler

REVIEW 3 major objections 6 minor 36 references

Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A perplexity-guided gradual binary search over per-projection clipping ratios makes 3-bit weights, activations, and KV-cache quantization practical, lifting average benchmark accuracy by about 40% over rotation-based baselines.

desk verdict A novel and empirically promising per-projection clipping search (GBS) for 3-bit WAKV quantization, with large reported gains that are probably real, but the paper needs reproducibility fixes and some internal corrections before the numbers can be taken at face value. read the letter →

arxiv 2504.13989 v2 pith:CYKXSHP2 submitted 2025-04-18 cs.LG cs.AIcs.CL

Lucas Maisonnave , Cyril Moineau , Olivier Bichler , Fabrice Rastello This is my paper · ORCID

classification cs.LGcs.AIcs.CL

keywords post-trainingquantizationlargelanguagemodelsHadamardmatricesactivationoutliersclippingratioperplexityoptimizationbinarysearchKVcache

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that 3-bit quantization of large language model weights, activations, and key-value caches is practical when each quantized projection is given its own clipping ratio — the fraction of the activation range kept before rounding — chosen automatically by a gradual binary search that minimizes the model's perplexity. On the QuaRot rotation-based pipeline, this search lifts the average six-benchmark accuracy of 3-bit Mistral-7B-Instruct from 22.06% to 61.32% (FP16: 71.62%) and cuts perplexity from 38.28 to 7.04; similarly large gains appear on LLaMA and Qwen models. The paper also proves that Hadamard matrices reduce activation outliers more than random orthogonal rotations (by $1/\sqrt{n}$ versus $\sqrt{2\log n}/\sqrt{n}$ at the level of the largest entry), and it extends rotations to non-power-of-2 embedding dimensions by zero-padding to a Paley-constructible size. A sympathetic reader would take the central insight to be that low-bit LLM quantization fails mainly because of badly chosen clipping ratios, not because of the rotation or the quantizer itself.

What carries the argument

The central mechanism is the Gradual Binary Search (GBS) of Algorithm 1: the model's linear projections are processed one by one; for each, the projection is quantized while the rest of the model remains in FP16 (or uses already chosen ratios), and the search probes clipping ratios in $[0,1]$, evaluating perplexity on 10% of WikiText2 at each probe and halving the interval toward the lower-perplexity side until the interval width is below $\epsilon$. The search assumes the perplexity-versus-ratio curve is convex with one minimum. Around it, the supporting machinery is the Hadamard rotation (an orthogonal matrix with entries $\pm 1$, applied with the fast Hadamard transform), the Paley construction that builds a Hadamard matrix of order $p+1$ when $p$ is prime and $p \equiv 3 \pmod 4$, and dimension expansion: zero-padding weights from dimension $n$ to $n+d$ so a Hadamard rotation exists, with Lemma 4.1 bounding $d \le n(b-b')/b'$ so the padded multiplication uses no more bit-operations than the unpadded one at a higher bitwidth.

What would settle it

On a fixed model and projection, compute perplexity over a fine grid of clipping ratios in [0,1] at 3-bit WAKV; if the resulting curve shows two separated minima, or a non-convex valley, the unimodality assumption fails and GBS can return a suboptimal ratio — observing such a curve for any model in the paper would directly challenge the method's claimed near-optimality.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that 3-bit quantization of LLM weights, activations, and KV caches is attainable when each quantized projection receives its own clipping ratio, chosen to minimize model perplexity rather than quantization error. The strongest evidence is the 3-bit WAKV result for Mistral-7B-Instruct: QuaRot alone reaches 22.06% average accuracy over six benchmarks with perplexity 38.28, and adding the Gradual Binary Search raises this to 61.32% with perplexity 7.04, compared with FP16's 71.62% and 5.49. The paper also claims a theoretical proof that a Hadamard matrix reduces a single outlier by a factor $1/\sqrt{n}$ while a random orthogonal rotation reduces it by $\sqrt{2\log n}/\sqrt{n}$, so Hadamard matrices attain the optimal bound among all orthogonal matrices. Finally, it claims that zero-padding weights to a dimension that admits a Paley-constructed Hadamard matrix generalizes rotations to non-power-of-2 architectures such as Qwen.

Load-bearing premise

The load-bearing premise is that perplexity as a function of a single projection's clipping ratio is convex with a single minimum, so the gradual binary search in Algorithm 1 cannot get stuck in a local minimum.

Editorial extensions

If this is right

3-bit WAKV quantization becomes usable on several 7-8B model families: the best 3-bit result (Mistral-7B-Instruct at 61.32 average accuracy) comes within about 10 points of FP16 while spending roughly a third of the bits on weights, activations, and KV cache.
GBS is rotation-agnostic: applying it to SpinQuant and DFRot improves their 4-bit and 3-bit results, so the ratio-search benefit is a general add-on to rotation-based quantizers, not a QuaRot-specific fix.
Dimension expansion plus Paley Hadamard construction removes the power-of-2 restriction, making rotation-based quantization available to architectures like Qwen with 1536- or 3584-dimensional embeddings.
Perplexity, not mean-squared quantization error, is the right guiding objective for low-bit post-training quantization; the paper's plots show lower perplexity tracking higher benchmark accuracy across all runs.
Expanding dimensions improves accuracy monotonically up to the Lemma 4.1 bit-operations budget, after which the padded computation costs as much as quantizing with one extra bit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: since the paper's final clipping ratios vary widely by projection (some near 1, some below 0.3), a cheap proxy — a small per-layer-type lookup table of ratios — could test how much of GBS's gain comes from per-projection tuning versus simply clipping aggressively.
Editorial inference: the unimodality assumption is the fragile point; if perplexity landscapes turn out to have multiple minima on other architectures or calibration sets, a multi-start or coarse-grid pre-pass would preserve the method while removing its main failure mode.
Editorial inference: the Hadamard-optimality proof for a single-outlier vector suggests a broader message: outlier reduction in LLM quantization is an incoherence problem, and any orthogonal matrix whose entries are all near $1/\sqrt{n}$ in absolute value should behave similarly to Hadamard rotations.
Editorial inference: the paper's benchmarks are six next-token-prediction tasks; testing GBS-quantized models on instruction-following or generation-quality metrics would show whether the perplexity gains translate beyond perplexity-like tasks.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing you should know about this paper is that the core method is genuinely new: a per-projection binary search over clipping ratios, driven by perplexity, applied gradually one projection at a time. The reported 3-bit WAKV results are striking—QuaRot's average accuracy jumps from 22.06 to 61.32 on Mistral-7B-Instruct, for example—and the improvements are consistent across Mistral, LLaMA, and Qwen families, and across three different rotation baselines (QuaRot, SpinQuant, DFRot). That breadth is the strongest evidence that the central claim is real, and the evaluation is not circular: the ratios are fit on 10% of WikiText2 train, while perplexity is reported on test and benchmark accuracies are on separate tasks.

What is actually new: the gradual binary search itself, and the Paley-based dimension expansion for non-power-of-2 hidden sizes. The Hadamard optimality result is elementary (the max entry of an orthogonal matrix is at least 1/√n, and Hadamard achieves it), but it is correct, and it gives a clean explanation for why Hadamard rotations reduce outliers more than random rotations. The paper also honestly acknowledges that dimension expansion was independently developed in [10].

The soft spots are real but mostly fixable. First, no code or data release, and no error bars; for a method that relies on a search procedure, that is a nontrivial reproducibility gap. Second, there is a direct contradiction about inference cost: Section 4.2 says dimension expansion adds "no additional computational cost at inference," but Lemma 4.1 and the surrounding text explicitly analyze the increased BitOps. The "no cost" statement is simply wrong—expanded activations still need to be rotated. Third, the arithmetic in the example is off: for n=4096, b=4, b'=3, Lemma 4.1 gives d ≤ 1365.33, so the claim that d=1366 is within the limit is a small error. Fourth, the "40% increase in accuracy" headline is cherry-picked from the best Mistral case; the mean improvement is smaller, though still substantial.

The stress-test concern about the unimodality assumption is fair but not fatal. The paper states "assuming a single minimum and a convex landscape" and then later says it "appears to hold true," but provides no direct evidence. The one-dimensional perplexity curve could have multiple minima, and the binary search would then stop at a local optimum. However, the empirical results are consistent across many models and three baselines, which suggests the search is working in practice even if the theoretical justification is loose. I would ask the authors to add a sensitivity analysis or at least plot a few representative perplexity curves.

Overall: this is a solid, useful engineering contribution that deserves peer review. I would send it to a serious referee and ask for code release, correction of the cost contradiction, and a more careful discussion of the unimodality assumption. The central claim is likely to survive.

Serious thinker: yes. The work is coherent, cites the right literature, and acknowledges its limitations. I would bring it to a reading group focused on LLM quantization.

Referee Report

3 major / 6 minor

Summary. This paper proposes GBS (Gradual Binary Search), a post-training quantization method that optimizes per-projection clipping ratios using perplexity as the objective, and combines it with Hadamard rotations and a dimension-expansion scheme (based on Paley Hadamard matrices) to handle non-power-of-2 embedding dimensions. The method targets 3-bit and 4-bit quantization of weights, activations, and KV cache in LLMs. The paper reports large improvements in perplexity and six zero-shot benchmarks relative to QuaRot, SpinQuant, and DFRot across Mistral, LLaMA, and Qwen model families, and also presents theoretical results claiming that Hadamard matrices reduce outlier amplitude more effectively than random orthogonal rotations.

Significance. If the empirical results hold, this would be a substantial advance: 3-bit WAKV quantization with only a modest perplexity increase (e.g., Mistral-7B Instruct from 5.49 to 7.04) and a large retained benchmark average would have clear practical value. The paper's strengths include the breadth of experiments (six models, three baselines), the consistency and size of the reported gains, and a clean theoretical explanation for the advantage of Hadamard rotations over random rotations. However, the central search rests on an unverified unimodality assumption, the paper contains a direct contradiction about inference cost, and the experimental section lacks code/data release and error bars; these issues mean the generality of the method is not yet fully established. With revision, the paper could become a solid contribution to the quantization literature.

major comments (3)

[Section 4.1 / Algorithm 1 / Appendix C] The algorithm's convergence to a near-optimal clipping ratio relies on the assertion in Section 4.1 that the perplexity landscape has a single minimum and is convex, and Section 5.2.1 merely states that this 'appears to hold true.' The evidence in Appendix C contradicts this: Figure 5 shows that running GBS from a model initialized in 4-bit precision converges to a configuration with many clipping ratios at 1 and high variance, while initialization from FP16 produces a different configuration, with test PPL 7.69 vs 7.40. That is direct evidence of initialization dependence, which is incompatible with a single-global-minimum landscape. Please provide a direct test of unimodality (e.g., grid evaluation of PPL versus clipping ratio for several representative projections) and a sensitivity analysis with respect to projection order and initialization; without this, the reported 3-bit gains cannot be robustly attributed to a global optimum search.
[Section 4.2 / Eq. (2) / Lemma 4.1] There is a contradiction about inference cost. The paragraph after Eq. (2) states that dimension expansion 'increases the model size and computational cost' and then, two sentences later, claims 'there is no additional computational cost at inference.' Lemma 4.1 itself bounds d so that BitOps(A'B') <= BitOps(AB), which only matters if the expanded product is computationally more expensive unless the bound is enforced. Please state precisely, for the fused architecture used in the experiments, whether the number of FLOPs per token changes with d. If it does increase, report the accuracy-versus-FLOPs trade-off (Figure 2 already mentions a 'very high computational cost') and compare against the alternative of using one more bit instead of expansion.
[Section 5 / Tables 1-4] The paper reports single-run numbers without error bars, confidence intervals, or a code/data release. The central quantitative claim, e.g., the Mistral-7B Instruct improvement from 22.06 to 61.32 average accuracy in Table 2, is dramatic and could be sensitive to the calibration subset (10% of WikiText2 train) and to hyperparameters such as epsilon, expansion size d, and projection order. Please provide a stability analysis over at least 3-5 calibration draws and report the spread of final PPL/AVG, and report sensitivity to epsilon and to the calibration fraction (the Limitations section itself suggests 10% may be excessive). This is important for supporting the 'general method' claim.

minor comments (6)

[Abstract / Conclusion] The phrase '40% increase in accuracy' is a 40-percentage-point improvement on a 0-100 scale, not a 40% relative increase; for the cited Mistral-7B Instruct case the relative increase is approximately 178%. Please rephrase to avoid misleading readers.
[Section 4.2] The text says 'We first chose to add 8 dimensions, reaching 1542' but 1536+8=1544. The value 1542 would not satisfy the Paley condition (1542-1=1541, which is composite), whereas 1544-1=1543 is prime and 1543 ≡ 3 (mod 4). Please correct the number or the arithmetic.
[Table 2, Qwen2.5 7B Inst row] In the QuaRot+ row for Qwen2.5 7B Inst, PIQA, HS, and ARC-E are all reported as exactly 1.14; this appears to be a copy-paste error and should be checked.
[Algorithm 1] The pseudocode does not make explicit how the model state evolves across the for loop: are projections 1..i-1 kept quantized with their selected ratios when projection i is being evaluated, and is the quantization of earlier projections re-applied or frozen when the clipping ratios are updated? Please clarify.
[Throughout] There are several typographical issues: 'demonsatrate' in Section 3.2, 'Figue' in Appendix A.1.2, and 'W AKV' in the table captions should be 'WAKV'.
[Figure 2 / Lemma 4.1] The text says the computational limit for LLaMA3-8B in 3 bits is d=1366, but Figure 2's x-axis extends to 3500 and shows a 'Computational limit' line. Please clarify how the limit is computed for each model and which bit budget is used for the comparison, since the curves appear to extend beyond the stated limit.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; GBS tuning is evaluated on held-out data and the Hadamard bound is derived from explicit lemmas.

full rationale

The derivation chain is self-contained. GBS (Algorithm 1) selects per-projection clipping ratios by minimizing perplexity on 10% of the WikiText2 training split, while the paper's headline numbers are WikiText2 test perplexity and six zero-shot benchmarks; the tuned ratios are not renamed as predictions of those benchmarks, so the evaluation is not forced by construction. The Hadamard superiority claim is derived in Theorem 3.1 from explicit Lemmas 3.1 and 3.2 using extreme-value calculations for random orthogonal matrices, with assumptions stated; no target result is inserted as an input. The unimodality assumption ('assuming a single minimum and a convex landscape', Section 4.1) is asserted and only described as 'appears to hold true' (Section 5.2.1), but a failure of unimodality would be a correctness or robustness defect of the search, not a circular reduction. The paper cites two of its own works ([21] and [22]) for background claims about quantization-error metrics and outlier visualization; [21] supports the choice of perplexity as objective, yet the paper independently validates that choice via Appendix D and the direct GBS comparisons, so the self-citations are not load-bearing. No uniqueness theorem is imported from the authors' prior work, and no fitted parameter is presented as a prediction. The main risks are empirical (unimodality, calibration overfitting), not circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The main fitted quantities are per-projection clipping ratios and the expansion dimension. The theoretical section relies on a simplified outlier model and a Gaussian approximation for random rotations. No new physical entities are introduced.

free parameters (4)

Per-projection clipping ratios = not reported numerically; determined by GBS for each model and projection
The central mechanism of the method: each quantized projection's clipping ratio is optimized to minimize perplexity on 10% of WikiText2 train, and the reported accuracy gains depend on these fitted values.
Dimension expansion size d = 8 for Qwen2.5-1.5B (text says 1542, should be 1544), 32 for Qwen2.5-7B MLP, up to 2036 for Llama3-8B
Chosen per model via accuracy and compute trade-offs (Figure 2); directly affects the BitOps bound in Lemma 4.1 and the practical cost of inference.
Binary search convergence threshold epsilon = not specified
Algorithm 1 stops when b-a > epsilon; the value determines how finely clipping ratios are tuned and is not reported.
Calibration set fraction = 10% of WikiText2 train
GBS evaluates perplexity on this subset; the paper acknowledges it may be excessive, making it a free choice of the method.

assumptions (5)

standard math Random orthogonal matrix entries are approximately i.i.d. N(0,1/n) in high dimension.
Used in Lemma 3.2 to estimate the maximum entry of a random rotation; the proof sketch conflates the max over n and n^2 entries, but the conclusion is a standard high-dimensional probability result.
domain assumption LLM activation outliers can be represented as a vector with one large coordinate and small noise coordinates.
Theorem 3.1 is proven only for x=(c,epsilon,...,epsilon), not for the richer outlier patterns observed in real LLMs; the empirical relevance is plausible but not established.
domain assumption Perplexity is a convex, unimodal function of the clipping ratio for each projection.
Section 4.1 says 'assuming a single minimum and a convex landscape'; this justifies the binary search but no evidence of convexity is provided.
domain assumption Perplexity on 10% of WikiText2 train is a reliable proxy for six downstream benchmark accuracies.
GBS uses this objective; Appendix D shows a correlation between lower perplexity and higher average benchmark score, but only on the evaluated points, not as a guarantee.
ad hoc to paper The expanded dimensions satisfy the Paley construction condition that n-1 is prime and congruent to 3 mod 4.
Section 5.2.2 claims 1542 is reachable, but 1541 is not prime and 1536+8=1544; the condition is standard, but the paper's stated values violate it, which is an error in the method description.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs." pith.science (2026). https://pith.science/paper/CYKXSHP2

@misc{pith2026250413989,
  author       = {Pith},
  title        = {Pith review of: Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYKXSHP2}},
  note         = {Machine review of arXiv:2504.13989}
}

read the original abstract

Large language models (LLMs) have become pivotal in artificial intelligence, demonstrating strong capabilities in reasoning, understanding, and generating data. However, their deployment on edge devices is hindered by their substantial size, often reaching several billion parameters. Quantization is a widely used method to reduce memory usage and inference time, however LLMs present unique challenges due to the prevalence of outliers in their activations. In this work, we leverage the theoretical advantages of Hadamard matrices over random rotation matrices to push the boundaries of quantization in LLMs. We demonstrate that Hadamard matrices are more effective in reducing outliers, which are a significant obstacle in achieving low-bit quantization. Our method based on a gradual binary search enables 3-bit quantization for weights, activations, and key-value (KV) caches, resulting in a 40% increase in accuracy on common benchmarks compared to SoTA methods. We extend the use of rotation matrices to support non-power-of-2 embedding dimensions, similar to the Qwen architecture, by employing the Paley algorithm. We theoretically demonstrates the superiority of Hadamard matrices in reducing outliers.We achieved 3-bit quantization for weights, activations, and KV cache, significantly enhancing model performance. Our experimental results on multiple models family like Mistral, LLaMA, and Qwen demonstrate the effectiveness of our approach, outperforming existing methods and enabling practical 3-bit quantization.

Figures

Figures reproduced from arXiv: 2504.13989 by the authors.

**Figure 2.** Effect of expanding dimensions on 6 benchmarks average (AVG) for different models in 3 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Maximum absolute value as a function of dimension for a randomly drawn rotation matrix [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: PPL vs Layer during Gradual Binary Search on 10% of Train WikiText2 for a LLaMA3-8B [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Final configurations obtained with GBS started in 4 bits and in FP16 for a LLaMA3-8B [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

**Figure 6.** Figure 6: AVG over Perplexity for all results obtained in Tab 2 and 1 [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 14 canonical work pages

[10]

Improving Quantization with Post-Training Model Expansion, March 2025

Giuseppe Franco, Pablo Monteagudo-Lago, Ian Colbert, Nicholas Fraser, and Michaela Blott. Improving Quantization with Post-Training Model Expansion, March 2025. URL http: //arxiv.org/abs/2503.17513. arXiv:2503.17513 [cs]

arXiv 2025
[1]

Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman

Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. SliceGPT: Compress Large Language Models by Deleting Rows and Columns, February 2024. URL http://arxiv.org/abs/2401.15024. arXiv:2401.15024

arXiv 2024
[2]

Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot: Outlier-Free 4- Bit Inference in Rotated LLMs, October 2024. URL http://arxiv.org/abs/2404.00456. arXiv:2404.00456 [cs]

arXiv 2024
[3]

QuIP: 2-Bit Quantization of Large Language Models With Guarantees

Jerry Chee, V olodymyr Kuleshov, and Yaohui Cai. QuIP: 2-Bit Quantization of Large Language Models With Guarantees

work page
[4]

PrefixQuant: Static Quantization Beats Dynamic through Prefixed Outliers in LLMs, October 2024

Mengzhao Chen, Yi Liu, Jiahao Wang, Yi Bin, Wenqi Shao, and Ping Luo. PrefixQuant: Static Quantization Beats Dynamic through Prefixed Outliers in LLMs, October 2024. URL http://arxiv.org/abs/2410.05265. arXiv:2410.05265

arXiv 2024
[5]

PACT: Parameterized Clipping Activation for Quantized Neural Networks, July 2018

Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I.-Jen Chuang, Vijayalak- shmi Srinivasan, and Kailash Gopalakrishnan. PACT: Parameterized Clipping Activation for Quantized Neural Networks, July 2018. URL http://arxiv.org/abs/1805.06085. arXiv:1805.06085 [cs]

arXiv 2018
[6]

Extreme Value Theory

Laurens De Haan and Ana Ferreira. Extreme Value Theory. Springer Series in Operations Research and Financial Engineering. Springer, New York, NY , 2006. ISBN 978-0-387-23946-0 978-0-387-34471-3. doi: 10.1007/0-387-34471-3. URL http://link.springer.com/10. 1007/0-387-34471-3

doi:10.1007/0-387-34471-3 2006
[7]

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

work page

Show all 36 references

[8]

Differentiable Model Compression via Pseudo Quantization Noise, October 2022

Alexandre Défossez, Yossi Adi, and Gabriel Synnaeve. Differentiable Model Compression via Pseudo Quantization Noise, October 2022. URL http://arxiv.org/abs/2104.09987. arXiv:2104.09987 [cs, stat]

2022 arXiv
[9]

Esser, Jeffrey L

Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha. Learned Step Size Quantization, May 2020. URL http://arxiv. org/abs/1902.08153. arXiv:1902.08153 [cs, stat]

2020 arXiv
[11]

GPTQ: Accurate Post- Training Quantization for Generative Pre-trained Transformers, March 2023

Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate Post- Training Quantization for Generative Pre-trained Transformers, March 2023. URL http: //arxiv.org/abs/2210.17323. arXiv:2210.17323 [cs]

2023 arXiv
[12]

Mahoney, and Kurt Keutzer

Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A Survey of Quantization Methods for Efficient Neural Network Inference, June 2021. URL http://arxiv.org/abs/2103.13630. arXiv:2103.13630 [cs]

2021 arXiv
[13]

A Survey on Methods and Theories of Quantized Neural Networks, December

Yunhui Guo. A Survey on Methods and Theories of Quantized Neural Networks, December
[14]

K. J. Horadam. Hadamard Matrices and Their Applications. Princeton University Press, January
[15]

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization, July 2024

Xijie Huang, Zechun Liu, Shih-Yang Liu, and Kwang-Ting Cheng. RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization, July 2024. URL http://arxiv.org/abs/2407.08044. arXiv:2407.08044 [cs]. 11

2024 arXiv
[16]

LightRot: A Light-weighted Rotation Scheme and Architecture for Accurate Low-bit Large Language Model Inference

Sangjin Kim, Yuseon Choi, Jungjun Oh, Byeongcheol Kim, and Hoi-Jun Yoo. LightRot: A Light-weighted Rotation Scheme and Architecture for Accurate Low-bit Large Language Model Inference. IEEE Journal on Emerging and Selected Topics in Circuits and Systems , pages 1–1, 2025. ISSN...

2025
[17]

DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs

Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei. DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs
[18]

Defensive Quantization: When Efficiency Meets Robustness, April 2019

Ji Lin, Chuang Gan, and Song Han. Defensive Quantization: When Efficiency Meets Robustness, April 2019. URL http://arxiv.org/abs/1904.08444. arXiv:1904.08444 [cs, stat]

2019 arXiv
[19]

SpinQuant: LLM quantization with learned rotations, May 2024

Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. SpinQuant: LLM quantization with learned rotations, May 2024. URL http://arxiv.org/abs/2405.16406. arXiv:2405.16406 [cs]

2024 arXiv
[20]

The Era of 1-bit LLMs: All Large Lan- guage Models are in 1.58 Bits, February 2024

Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. The Era of 1-bit LLMs: All Large Lan- guage Models are in 1.58 Bits, February 2024. URL http://arxiv.org/abs/2402.17764. arXiv:2402.17764 [cs]

2024 arXiv
[21]

Applying maximum entropy principle on quantized neural networks correlates with high accuracy

Lucas Maisonnave, Cyril Moineau, Olivier Bichler, and Fabrice Rastello. Applying maximum entropy principle on quantized neural networks correlates with high accuracy
[22]

Precision Where It Matters: A Novel Spike Aware Mixed-Precision Quantization Strategy for LLaMA-based Lan- guage Models, April 2025

Lucas Maisonnave, Cyril Moineau, Olivier Bichler, and Fabrice Rastello. Precision Where It Matters: A Novel Spike Aware Mixed-Precision Quantization Strategy for LLaMA-based Lan- guage Models, April 2025. URL http://arxiv.org/abs/2504.21553. arXiv:2504.21553 [cs]

2025 arXiv
[23]

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization, April 2024

Aniruddha Nrusimha, Mayank Mishra, Naigang Wang, Dan Alistarh, Rameswar Panda, and Yoon Kim. Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization, April 2024. URL http://arxiv.org/abs/2404.03605. arXiv:2404.03605 [cs] version: 1

2024 arXiv
[24]

Pre- fixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization

Seungwoo Son, Wonpyo Park, Woohyun Han, Kyuyeun Kim, and Jaeho Lee. Pre- fixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization. June 2024. URL https://www.semanticscholar.org/paper/ 1601ad7616681ed9d7e1b9a04b64c1ad9c7196c7

2024
[25]

High-Dimensional Probability: An Introduction with Applica- tions in Data Science

Roman Vershynin. High-Dimensional Probability: An Introduction with Applica- tions in Data Science . Cambridge Series in Statistical and Probabilistic Math- ematics. Cambridge University Press, Cambridge, 2018. ISBN 978-1-108-41519-

2018
[26]

BitNet: Scaling 1-bit Transformers for Large Language Models, October 2023

Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. BitNet: Scaling 1-bit Transformers for Large Language Models, October 2023. URL http://arxiv.org/abs/2310.11453. arXiv:2310.11453 [cs]

2023 arXiv
[27]

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling, October 2023

Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, and Xianglong Liu. Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling, October 2023. URL http://arxiv.org/abs/ 2304.09145. arXiv:23...

2023 arXiv
[28]

URL https://www.cambridge.org/core/books/ highdimensional-probability/797C466DA29743D2C8213493BD2D2102

doi: 10.1017/9781108231596. URL https://www.cambridge.org/core/books/ highdimensional-probability/797C466DA29743D2C8213493BD2D2102

doi
[29]

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning, pages 38087–38099. 12 PMLR, July 202...

2023
[30]

Post- Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting, December

Dawei Yang, Ning He, Xing Hu, Zhihang Yuan, Jiangyong Yu, Chen Xu, and Zhe Jiang. Post- Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting, December
[31]

DFRot: Achieving Outlier-Free and Massive Activation- Free for Rotated LLMs with Refined Rotation, December 2024

Jingyang Xiang and Sai Qian Zhang. DFRot: Achieving Outlier-Free and Massive Activation- Free for Rotated LLMs with Refined Rotation, December 2024. URL http://arxiv.org/ abs/2412.00648. arXiv:2412.00648 [cs]

2024 arXiv
[32]

Chenzhuo Zhu, Song Han, Huizi Mao, and William J. Dally. Trained Ternary Quantization, February 2017. URL http://arxiv.org/abs/1612.01064. arXiv:1612.01064 [cs]. A Theoretical proofs A.1 Hadamard matrices A.1.1 Proof Proof of Lemma 3.1. We define a Hadamard matrix as a rotatio...

2017 arXiv
[35]

FracBits: Mixed Precision Quantization via Fractional Bit-Widths

Linjie Yang and Qing Jin. FracBits: Mixed Precision Quantization via Fractional Bit-Widths. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12):10612–10620, May 2021. ISSN 2374-3468, 2159-5399. doi: 10.1609/aaai.v35i12.17269. URL https://ojs.aaai. org/index.p...

2021 doi
[2012]

Google-Books-ID: oR_HDgAAQBAJ

ISBN 978-1-4008-4290-2. Google-Books-ID: oR_HDgAAQBAJ
[2018]

arXiv:1808.04752 [cs, stat]

URL http://arxiv.org/abs/1808.04752. arXiv:1808.04752 [cs, stat]

arXiv
[2023]

arXiv:2312.10588 [cs]

URL http://arxiv.org/abs/2312.10588. arXiv:2312.10588 [cs]

arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [10]

Improving Quantization with Post-Training Model Expansion, March 2025

Giuseppe Franco, Pablo Monteagudo-Lago, Ian Colbert, Nicholas Fraser, and Michaela Blott. Improving Quantization with Post-Training Model Expansion, March 2025. URL http: //arxiv.org/abs/2503.17513. arXiv:2503.17513 [cs]

arXiv 2025

[2] [1]

Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman

Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. SliceGPT: Compress Large Language Models by Deleting Rows and Columns, February 2024. URL http://arxiv.org/abs/2401.15024. arXiv:2401.15024

arXiv 2024

[3] [2]

Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot: Outlier-Free 4- Bit Inference in Rotated LLMs, October 2024. URL http://arxiv.org/abs/2404.00456. arXiv:2404.00456 [cs]

arXiv 2024

[4] [3]

QuIP: 2-Bit Quantization of Large Language Models With Guarantees

Jerry Chee, V olodymyr Kuleshov, and Yaohui Cai. QuIP: 2-Bit Quantization of Large Language Models With Guarantees

work page

[5] [4]

PrefixQuant: Static Quantization Beats Dynamic through Prefixed Outliers in LLMs, October 2024

Mengzhao Chen, Yi Liu, Jiahao Wang, Yi Bin, Wenqi Shao, and Ping Luo. PrefixQuant: Static Quantization Beats Dynamic through Prefixed Outliers in LLMs, October 2024. URL http://arxiv.org/abs/2410.05265. arXiv:2410.05265

arXiv 2024

[6] [5]

PACT: Parameterized Clipping Activation for Quantized Neural Networks, July 2018

Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I.-Jen Chuang, Vijayalak- shmi Srinivasan, and Kailash Gopalakrishnan. PACT: Parameterized Clipping Activation for Quantized Neural Networks, July 2018. URL http://arxiv.org/abs/1805.06085. arXiv:1805.06085 [cs]

arXiv 2018

[7] [6]

Extreme Value Theory

Laurens De Haan and Ana Ferreira. Extreme Value Theory. Springer Series in Operations Research and Financial Engineering. Springer, New York, NY , 2006. ISBN 978-0-387-23946-0 978-0-387-34471-3. doi: 10.1007/0-387-34471-3. URL http://link.springer.com/10. 1007/0-387-34471-3

doi:10.1007/0-387-34471-3 2006

[8] [7]

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

work page