Pith. sign in

REVIEW 2 major objections 5 minor 52 references

Quantizing both factors of a matrix product before multiplying introduces error; this paper derives the exact expected size of that error and shows that the best rescaling of the shared dimension — a 'fold' — is the solution of a provably s

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Choosing a pre-quantization contraction gauge by minimizing a derived product-error identity gives certified optimal diagonal folds and empirically validated gains for rotations and grouping, tested on quantized matrix products in an image classifier.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection The exact dither-model identity and the shared-fold GP are solid and genuinely useful; the RTN-transfer claims rest on one tiny classifier and need broader validation. the 2 major comments →

arxiv 2607.18745 v1 pith:RNUDDPVO submitted 2026-07-21 cs.LG cs.ITcs.NAmath.ITmath.NA

Contraction-Gauge Preconditioning for Quantized Matrix Multiplication

Piyush Sao , Narasinga Miniskar , Pedro Valero-Lara , Keita Teranishi , Sudip Seal This is my paper
classification cs.LG cs.ITcs.NAmath.ITmath.NA MSC 15A6065G5090C25
keywords quantized matrix multiplicationproduct-error identitycontraction gaugediagonal foldsgeometric programmingsubtractive ditherround-to-nearesttransform selection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to put low-precision matrix multiplication on a single quantitative footing: it derives an exact formula for the expected squared error of a quantized product, then shows that one whole family of error-reducing tricks — rescaling the shared dimension before quantization (a 'fold') — can be chosen by convex optimization with a certified global optimum. The formula matters because the error of a quantized product is not the entrywise error of either factor but that error propagated through the other factor's energy: an outlier in one operand is harmless exactly when the opposite factor has little energy in that coordinate. The identity holds exactly under subtractive dither and independent stochastic rounding; for the hardware-standard deterministic round-to-nearest it is an approximation, so the paper measures how well its predictions transfer. On twelve products of a trained three-block image classifier, the dither-model predictions rank measured round-to-nearest errors with median within-product correlation 0.937 at 8 bits and 0.918 at 4 bits, and the optimized fold cuts held-out product error by 18.0% and 20.5% relative to the identity fold.

Core claim

The paper's central claim is an exact identity for the expected squared error of a quantized matrix product under independent zero-mean entrywise noise (exact for subtractive dither and stochastic rounding, approximate for round-to-nearest): the error equals each operand's quantization variances charged against the opposite factor's row/column energies, plus a bilinear simultaneous-error term. On top of this identity the paper proves that choosing a diagonal contraction gauge — a per-coordinate rescaling (A,B)↦(AD,D⁻¹B) that preserves the product — over a shared output domain is a geometric program with a convex log-domain form and certified global optimum, and that a linear program decides

What carries the argument

The load-bearing object is the product-error identity (Theorem 3.3): an exact finite-dimensional formula expressing the expected squared error of the quantized product as a function of the entrywise variance fields v^A and v^B, weighted by opposite-factor row and column energies, plus a bilinear cross term for simultaneous errors in both factors. It converts every design decision — transform, sharing pattern, grouping, bit allocation, clipping, rounding rule — into one scalar objective. The second mechanism is the contraction-gauge equivalence AB = (AT)(T⁻¹B): a diagonal gauge (a 'fold') makes the shared-domain objective a geometric program, convex in log coordinates, so the optimum is certi

Load-bearing premise

The certified optima — the GP fold, the identity-fold test, the selection statistics — are exact for the stochastic dither model with variance v = c·range²; transferring that ranking to deterministic round-to-nearest, which has input-dependent bias and correlated errors, is verified empirically on one small classifier, and if the transfer fails on other models the 'global optimum' would be optimizing a surrogate.

What would settle it

Reproduce the twelve-product selection protocol on a different architecture, such as a pretrained language model with real token activations, and measure the median within-product Spearman correlation between dither-predicted and round-to-nearest product errors; if it falls well below 0.9, or if the GP fold is beaten by a grid-tuned baseline on a majority of products, the surrogate-to-round-to-nearest transfer that carries the practical claims fails. A faster check is the paper's characteristic-function diagnostic Ξ: activations whose Ξ sits far above the uniform-residue null are warning signs

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Diagonal fold selection for a shared output domain stops being a heuristic: the log-domain geometric program gives a certified global optimum of the range-law objective, and the linear-program test decides exactly when the identity fold is already optimal.
  • One product-error objective ranks candidates across transform families, so comparisons like flat rotation versus hierarchy reduce to evaluating a single statistic such as the slice-energy overlap g Σ p_r q_r.
  • A fixed bit budget between the two operands should be split asymmetrically as b*_A − b*_B = ½ log₂(P_A/P_B); the paper's experiments show the predicted 9/7 split beats the symmetric 8/8 split by 1.7× in squared relative error.
  • Transfer of the dither model to deterministic RTN is measurable: on a trained classifier the dither predictions rank realized errors with median within-product correlation above 0.9, and the GP fold beats both the identity fold and a grid-tuned scaling baseline on ten of twelve products at each precision, lowering composed logit MSE by 15.4% and 26.4%.
  • Transform reuse is made explicit through the quantized-copy count n_opp: a shared gauge costs one transformed copy of the opposite factor, block-specific gauges cost up to one per block, and the measured copy/error trade-off on heavy-tailed operands shows three extra copies buy 24.6% lower error.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the certified optima apply to the dither variance law v = c·range²; an immediate testable upgrade would feed the same geometric program with variance fields empirically calibrated from round-to-nearest residuals, which would make the 'global optimum' certified for the actual deployment quantizer.
  • Editorial extension: the weighted-norm generalization (Corollary 3.4) means the same selection machinery can target residual- or energy-weighted objectives in iterative solvers; the paper points at this application but does not demonstrate it.
  • Editorial extension: the bit-allocation rule offers a cheap audit for existing quantized inference stacks — compute P_A/P_B per layer on real activations and compare the deployed split with ½ log₂(P_A/P_B); where they disagree, the paper's experiments suggest up to ~1.7× error is on the table.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies low-precision computation of C=AB when both factors are quantized. It derives an exact identity (Theorem 3.3) for the expected squared Frobenius product error under independent, zero-mean entrywise errors, valid for non-overloading subtractive dither and for independent stochastic rounding. Using the product-preserving equivalence AB=(AT)(T^{-1}B), it introduces contraction-gauge preconditioning. For positive diagonal gauges (folds), Theorem 4.2 shows that the domain-shared fold selection problem is a geometric program with a convex log-domain formulation, and Proposition 4.3 gives an LP test for identity-fold optimality. For rotations, partial rotations, and hierarchies, the paper derives computable selection statistics and upper bounds (Theorems 6.1, 7.1, 8.1–8.2). Quantizer-side results include asymmetric bit allocation (Theorem 9.1) and a clipped product-error identity (Theorem 9.4). The empirical section tests the dither-model predictions against deterministic RTN on synthetic problems and on twelve internal products from a trained three-block ViT-like image classifier, reporting median within-product rank correlations of 0.937 (8-bit) and 0.918 (4-bit) and geometric-mean held-out error reductions of 18.0% and 20.5% relative to the identity fold.

Significance. The mathematical core is a genuine contribution: an exact finite-dimensional accounting of simultaneous quantization errors, a globally optimal formulation for shared diagonal folds within the stated dither model, and a clean LP test for identity-fold optimality. The proofs in the appendix are careful, and the controlled experiments are designed to isolate each mechanism. The main value of the paper, if it holds, is a common objective and decision statistics for comparing gauge families under stochastic quantization. However, the headline applied claims concern deterministic RTN, and those claims rest on an empirical transfer from the dither model that is validated on only a single small image classifier. The paper is honest about this gap in Remark 9.3, Section 11.1, and Section 12, but the abstract and conclusion present the RTN transfer as a working feature of the framework. The soundness of the core theorems is not in question; the breadth of the applied claim is.

major comments (2)
  1. [Remark 9.3; Section 11.1; Section 10.9] The certified results (Theorem 4.2 and Proposition 4.3) are proved for the dither variance law v_G = c R_G^2. Deterministic RTN is explicitly outside the model: Section 11.1 notes input-dependent bias and inter-coordinate correlations, and Remark 9.3 gives an explicit pair (A=[1 1 1 0], B=I_4, H_4) on which the dither objective ranks H_4 above I_4 while INT8 RTN ranks I_4 above H_4. Therefore the abstract's statement that the GP fold provides a 'common objective for evaluating reusable transform candidates under RTN' is an empirical hypothesis, not a consequence of the certified optimum. The only transfer evidence is Section 10.9: twelve internal products from one trained three-block width-64 ViT-like classifier on 8x8 digit images. This is too narrow to support the general applied claim. Please either broaden the validation (e.g., pretrained language models, as already listed in Section
  2. [Theorem 4.2; Sections 10.2 and 10.9] The 'globally optimal shared fold' claim is global only after imposing a compactification such as finite box bounds h_low <= h_k <= h_high (or an alternative scale-gauge constraint). The experiments report numerically fitted GP folds but do not state the box bounds used, nor whether the resulting solution lies in the interior of the box. Without this information, the reader cannot verify that the deployed fold is the global minimizer of eq. (9) rather than a boundary solution of an arbitrary compactification. Since the headline comparisons against identity and SmoothQuant use these solved folds, the numerical setup needs to be specified, and boundary activity should be checked, for the 'globally optimal' claim to be meaningful in the experimental sections.
minor comments (5)
  1. [Figure 11d and Section 10.9] The median Spearman correlations are given as 0.937 and 0.918 in the text but are rounded to 0.94 and 0.92 in the figure caption. Please make the values consistent.
  2. [Appendix F] The artifact is described as complete with checksums and one-command regeneration, but the repository is private pending clearance. The reproducibility claim cannot be verified by reviewers. Please make the code public before final acceptance, or clearly mark the artifact as unavailable and provide the full numerical outputs in the supplement.
  3. [Sections 4.2 and 10.4] The log-magnitude regularizer tau_log is a free parameter and is fixed in advance in the experiments. Please report the chosen value and, ideally, a sensitivity analysis or a selection rule, since Theorem 5.2's transfer to raw spread requires the condition tau_log <= epsilon a_min,k.
  4. [Table 1] The reuse descriptors n_gauge and n_opp are central to the paper, but the subscript formatting is inconsistent in the text (e.g., 'nopp' vs 'n opp'). Please use a consistent notation, e.g., n_opp, in Table 1 and in all figures that reference copy counts.
  5. [Section 10.2] The identity-to-GP modeled objective ratio of 161.2x is a ratio of modeled dither objectives, not of realized RTN errors. The text does say this, but the figure could mislead readers; consider making this distinction more prominent in the panel label or caption.

Circularity Check

0 steps flagged

No circularity: the product-error identity, GP fold optimality, and selection statistics are derived from stated assumptions; RTN transfer limits are acknowledged and tested, not used as inputs.

full rationale

Walking the derivation chain: Theorem 3.3 expands ||ÂB̂−AB||_F^2 under the independent zero-mean model eq.(2) and proves the variance-weighted identity; the variance fields are inputs, not fitted outputs. Theorem 4.2 defines a geometric program whose objective is exactly the dither range-law leading error plus the cross term, and its global optimality is a mathematical consequence of convexity in log coordinates — not an imported uniqueness result. Proposition 4.3 similarly derives the identity-fold LP from the same objective. The coherence, partial-rotation, and hierarchy bounds are proven in the paper from explicit random-matrix and spectral arguments; they are not self-citations. The trained-classifier claims are empirical transfer tests: the GP fold is selected on calibration dither predictions and then measured on held-out deterministic-RTN products, so the 18.0% / 20.5% reductions are independent measurements, not quantities forced by construction. Remark 3.2 and Section 11.1 explicitly state that deterministic RTN introduces input-dependent bias and inter-coordinate correlations, and Remark 9.3 gives a concrete instance where dither-model and RTN orderings disagree; Section 12 lists pretrained-language-model validation as future work. These are honest limitations of the surrogate-to-RTN transfer, not circularity. No fitted parameter is renamed as a prediction, and no load-bearing step reduces to its own input by definition.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 0 invented entities

The central claims rest on standard dither-quantization assumptions and on the validity of the dither variance law as a proxy for RTN. No new physical entities are introduced; the paper's novel objects (gauge domains, n_opp) are descriptive constructs rather than new postulates.

free parameters (2)
  • log-magnitude regularizer τ_log = 1e-3 (fixed in advance; no tuning)
    Used in Theorem 5.2 and clustering experiments to avoid log(0); results depend mildly on the choice, and the paper states it can be chosen on a grid.
  • GP compactification box bounds {h_low, h_high} = unspecified
    Introduced in Theorem 4.2 to guarantee attainment of the infimum for the shared-fold GP; without them degenerate inputs can fail to attain a finite optimum.
axioms (6)
  • domain assumption Independent zero-mean entrywise quantization errors with known variance fields (eq. 2)
    The entire product-error identity and all derived optima are conditional on this independence and zero-mean structure.
  • domain assumption Non-overloading subtractive dither yields uniform, input-independent errors with variance c R_G^2 (Section 3.1, Remark 3.1)
    This law is what turns quantization groups into variance fields and makes the GP objective exact for dither.
  • domain assumption Deterministic RTN error is adequately ranked by the dither variance law (Section 11.1)
    The transfer of dither-model predictions to RTN is empirically assessed, not proven; the whole practical selection procedure depends on this transfer.
  • domain assumption High-rate quantization law v ∝ 2^{-2b} for bit allocation (Section 9.1)
    The closed-form optimal bit split in Theorem 9.1 uses this high-rate law.
  • domain assumption Variance-normalized sub-Gaussian errors for the concentration certificate (Appendix B)
    The single-run certificate in Corollary B.2 requires this tail assumption; stochastic rounding needs a separate step-size-based bound.
  • standard math Standard tools: geometric programming, Hanson–Wright, Schur-convexity, Ailon–Chazelle random Hadamard incoherence (various sections)
    These background results are cited and used without derivation.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Contraction-Gauge Preconditioning for Quantized Matrix Multiplication." pith.science (2026). https://pith.science/paper/RNUDDPVO

@misc{pith2026260718745,
  author       = {Pith},
  title        = {Pith review of: Contraction-Gauge Preconditioning for Quantized Matrix Multiplication},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RNUDDPVO}},
  note         = {Machine review of arXiv:2607.18745}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We study low-precision computation of C=AB with both factors quantized. We derive an exact finite-dimensional identity for the expected squared product error under independent, zero-mean entrywise errors with known variance fields; it holds exactly for non-overloading subtractive dither and for independent stochastic rounding, and we empirically assess deterministic round-to-nearest (RTN). Using the product-preserving equivalence AB=(AT)(T^{-1}B), we formulate contraction-gauge preconditioning: jointly choosing a factor representation and its sharing pattern before quantization. Preconditioning can reduce product error but may require extra transformed, quantized copies of the opposite operand: a shared transform needs one copy, a block-specific transform up to one per block. Within the bounded family of positive diagonal gauges (folds), a geometric program computes a globally optimal shared fold and a linear program decides whether the identity fold is already optimal. For other families we derive computable selection statistics -- tail index for scaling, profile spread for partitioning, coherence and weighted-Gram energy for rotations, slice-energy covariance for hierarchy depth -- with upper bounds for ranking heuristic candidates. Across twelve linear products from a trained three-block image classifier, median within-product rank correlations between dither-model predictions and deterministic-RTN errors are 0.937 at 8 bits and 0.918 at 4 bits. The GP fold cuts held-out product error over the identity fold by 18.0% (8-bit) and 20.5% (4-bit) in geometric mean, beats a SmoothQuant-style grid baseline at both precisions and on ten of twelve products, and lowers composed logit MSE by 15.4% and 26.4%. We thus provide exact stochastic product-error accounting, certified selection within the diagonal family, and a common objective for evaluating reusable transform candidates under RTN.

Figures

Figures reproduced from arXiv: 2607.18745 by Keita Teranishi, Narasinga Miniskar, Pedro Valero-Lara, Piyush Sao, Sudip Seal.

Figure 1
Figure 1. Figure 1: Opposite-factor weighting of a single entry error. An error (EA)ik = e propagates through row Bk,: and perturbs the entire output row Ci,: by eBk,: . Its output norm is |e|∥Bk,:∥2, so its squared contribution is e 2∥Bk,:∥ 2 2 . Symmetrically, an error in Bkj perturbs an output column and is weighted by ∥A:,k∥ 2 2 . Equation (3) concerns the mean; a single computation uses one noise realization. Appendix B … view at source ↗
Figure 2
Figure 2. Figure 2: Gauge sharing controls opposite-factor quantized-copy count. With four distinct block￾specific gauges, the row blocks pair with four transformed and quantized representations of B (left). Sharing one gauge across all row blocks permits one reusable representation (right). The displayed equalities nopp = ngauge ∈ {4, 1} assume common quantizer settings, distinct quantized outputs for distinct gauges, and no… view at source ↗
Figure 3
Figure 3. Figure 3: Gauge design and taxonomy. The top chain maps a product-preserving gauge through group ranges to the variance field in eq. (3). Below it, diagonal, orthogonal, and block-diagonal gauges give folds, rotations, and hierarchies, while sorting and splitting define a block-constant gauge-domain pattern. The domain pattern controls copy count; the within-domain structure controls achievable error. With a common … view at source ↗
Figure 4
Figure 4. Figure 4: Heuristic folding gains over per-vector scaling. (a) Gain by input family for [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Slice-energy alignment controls the single-level hierarchy decision. The normalized over￾lap g P r prqr equals the ratio of the hierarchical and flat leading-error surrogates in Theorem 8.1. Comple￾mentary (anti-correlated) slice-energy profiles produce a ratio below one and favor refinement (left); aligned profiles produce a ratio above one and favor the flat candidate (right). Bars and displayed ratios a… view at source ↗
Figure 6
Figure 6. Figure 6: Scaling behavior across tail regimes. (a) Refinement ordering from Section [PITH_FULL_IMAGE:figures/full_fig_p026_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Optimization predictions for domain-shared folds and bit allocation. (a) Modeled error relative [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Controlled tests of data-driven transform-selection criteria. (a) Block-spread ratios relative to [PITH_FULL_IMAGE:figures/full_fig_p028_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Orthogonal preconditioning on constructed coordinate-outlier families. (a) Paired unrotated-to [PITH_FULL_IMAGE:figures/full_fig_p028_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Shared-fold selection across the SmoothQuant-style α sweep. Each blue point is the geometric-mean held-out RTN error ratio for one common α across twelve products; blue shading shows the interquartile range. The red line and band show the geometric mean and interquartile range of the product-specific leading-objective GP folds. The horizontal value one is the identity-fold reference. Both factors are quan… view at source ↗
Figure 11
Figure 11. Figure 11: Quantizer design and deterministic-rounding model assessment. (a) On Gaussian data, max scal [PITH_FULL_IMAGE:figures/full_fig_p032_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Paired deterministic-RTN error ratios relative to a shared rotation on heavy-tailed operands. The [PITH_FULL_IMAGE:figures/full_fig_p033_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Product-level and composed deterministic-RTN error. (a) Held-out GP-fold error versus the best test-selected point on the eleven-value SmoothQuant-style α grid, each normalized by the identity fold. The GP is lower on ten of twelve products at both precisions. (b) Relative logit MSE after quantizing all twelve products simultaneously. At 8 bits, the ratios for α = 0.5, calibration-selected α, and the GP a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    Scikit-learn: Machine Learning in

    Pedregosa, Fabian and Varoquaux, Ga. Scikit-learn: Machine Learning in. Journal of Machine Learning Research , volume =. 2011 , url =

    2011

  2. [2]

    1998 , doi =

    Optical Recognition of Handwritten Digits , author =. 1998 , doi =

    1998

  3. [3]

    Physical Review B , volume=

    Gauge Fixing, Canonical Forms, and Optimal Truncations in Tensor Networks with Closed Loops , author=. Physical Review B , volume=. 2018 , doi=

    2018

  4. [4]

    SciPost Physics , volume=

    Gauging Tensor Networks with Belief Propagation , author=. SciPost Physics , volume=. 2023 , doi=

    2023

  5. [5]

    SIAM Journal on Optimization , volume=

    Gauge Optimization and Duality , author=. SIAM Journal on Optimization , volume=. 2014 , doi=

    2014

  6. [6]

    IEEE Transactions on Information Theory , volume=

    Optimal Quantization for Matrix Multiplication , author=. IEEE Transactions on Information Theory , volume=. 2026 , doi=

    2026

  7. [7]

    Proceedings of the 42nd International Conference on Machine Learning , series=

    NestQuant: Nested Lattice Quantization for Matrix Products and LLMs , author=. Proceedings of the 42nd International Conference on Machine Learning , series=. 2025 , note=

    2025

  8. [8]

    2025 IEEE International Symposium on Information Theory (ISIT) , pages=

    High-Rate Nested-Lattice Quantized Matrix Multiplication with Small Lookup Tables , author=. 2025 IEEE International Symposium on Information Theory (ISIT) , pages=. 2025 , doi=

    2025

  9. [9]

    Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke , booktitle=

  10. [10]

    Xiao, Guangxuan and Lin, Ji and Seznec, Mickael and Wu, Hao and Demouth, Julien and Han, Song , booktitle=

  11. [11]

    Chee, Jerry and Cai, Yaohui and Kuleshov, Volodymyr and De Sa, Christopher , booktitle=

  12. [12]

    Ashkboos, Saleh and Mohtashami, Amirkeivan and Croci, Maximilian L and Li, Bo and Cameron, Pashmina and Jaggi, Martin and Alistarh, Dan and Hoefler, Torsten and Hensman, James , booktitle=

  13. [13]

    2025 , note=

    Liu, Zechun and Zhao, Changsheng and Fedorov, Igor and Soran, Bilge and Choudhary, Dhruv and Krishnamoorthi, Raghuraman and Chandra, Vikas and Tian, Yuandong and Blankevoort, Tijmen , booktitle=. 2025 , note=

    2025

  14. [14]

    IEEE Transactions on Information Theory , volume=

    Quantization , author=. IEEE Transactions on Information Theory , volume=

  15. [15]

    Theoretical Computer Science , volume=

    Clustering to minimize the maximum intercluster distance , author=. Theoretical Computer Science , volume=

  16. [16]

    Approximate nearest neighbors and the fast

    Ailon, Nir and Chazelle, Bernard , booktitle=. Approximate nearest neighbors and the fast

  17. [17]

    2024 , url=

    Tseng, Albert and Chee, Jerry and Sun, Qingyao and Kuleshov, Volodymyr and De Sa, Christopher , booktitle=. 2024 , url=

    2024

  18. [18]

    arXiv preprint arXiv:2603.19559 , year=

    Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition , author=. arXiv preprint arXiv:2603.19559 , year=

    arXiv

  19. [19]

    Electronic Communications in Probability , volume=

    Hanson--Wright inequality and sub-Gaussian concentration , author=. Electronic Communications in Probability , volume=

  20. [20]

    2023 , url=

    Yuan, Zhihang and Niu, Lin and Liu, Jiawei and Liu, Wenyu and Wang, Xinggang and Shang, Yuzhang and Sun, Guangyu and Wu, Qiang and Wu, Jiaxiang and Wu, Bingzhe , journal=. 2023 , url=

    2023

  21. [21]

    High-Rate Quantized Matrix Multiplication I

    Ordentlich, Or and Polyanskiy, Yury , journal=. High-Rate Quantized Matrix Multiplication. 2026 , note=. doi:10.48550/arXiv.2601.17187 , url=

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2601.17187 2026

  22. [22]

    High-Rate Quantized Matrix Multiplication

    Ordentlich, Or and Polyanskiy, Yury , journal=. High-Rate Quantized Matrix Multiplication. 2026 , url=

    2026

  23. [23]

    2025 , url=

    Hu, Xing and Cheng, Yuan and Yang, Dawei and Chen, Zhixuan and Xu, Zukang and Yu, Jiangyong and Xu, Chen and Yuan, Zhihang and Jiang, Zhe and Zhou, Sifan , booktitle=. 2025 , url=

    2025

  24. [24]

    2025 , url=

    Sun, Yuxuan and Liu, Ruikang and Bai, Haoli and Bao, Han and Zhao, Kang and Li, Yuening and Hu, Jiaxin and Yu, Xianzhi and Hou, Lu and Yuan, Chun and Jiang, Xin and Liu, Wulong and Yao, Jun , booktitle=. 2025 , url=

    2025

  25. [25]

    arXiv preprint arXiv:2601.22347 , year=

    Pushing the Limits of Block Rotations in Post-Training Quantization , author=. arXiv preprint arXiv:2601.22347 , year=. doi:10.48550/arXiv.2601.22347 , url=

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.22347

  26. [26]

    arXiv preprint arXiv:2605.13810 , year=

    Provable Quantization with Randomized Hadamard Transform , author=. arXiv preprint arXiv:2605.13810 , year=

    Pith/arXiv arXiv

  27. [27]

    Proceedings of the 36th International Conference on Machine Learning , series=

    Same, Same But Different---Recovering Neural Network Quantization Error Through Weight Factorization , author=. Proceedings of the 36th International Conference on Machine Learning , series=. 2019 , url=

    2019

  28. [28]

    IEEE International Conference on Computer Vision (ICCV) , year=

    Data-Free Quantization Through Weight Equalization and Bias Correction , author=. IEEE International Conference on Computer Vision (ICCV) , year=

  29. [29]

    2021 , url=

    Kovaleva, Olga and Kulshreshtha, Saurabh and Rogers, Anna and Rumshisky, Anna , booktitle=. 2021 , url=

    2021

  30. [30]

    IEEE Transactions on Communication Technology , volume=

    Dither Signals and Their Effect on Quantization Noise , author=. IEEE Transactions on Communication Technology , volume=. 1964 , doi=

    1964

  31. [31]

    IEEE Transactions on Acoustics, Speech, and Signal Processing , volume=

    A Necessary and Sufficient Condition for Quantization Errors to Be Uniform and White , author=. IEEE Transactions on Acoustics, Speech, and Signal Processing , volume=. 1977 , doi=

    1977

  32. [32]

    arXiv preprint arXiv:2603.04359 , year=

    Dissecting Quantization Error: A Concentration--Alignment Perspective , author=. arXiv preprint arXiv:2603.04359 , year=. doi:10.48550/arXiv.2603.04359 , url=

    work page doi:10.48550/arxiv.2603.04359

  33. [33]

    IEEE Transactions on Communications Systems , volume=

    Block Quantization of Correlated Gaussian Random Variables , author=. IEEE Transactions on Communications Systems , volume=. 1963 , doi=

    1963

  34. [34]

    Advances in Neural Information Processing Systems , volume=

    Post Training 4-Bit Quantization of Convolutional Networks for Rapid-Deployment , author=. Advances in Neural Information Processing Systems , volume=. 2019 , url=

    2019

  35. [35]

    2024 , doi=

    Zhang, Aozhong and Wang, Naigang and Deng, Yanxia and Li, Xin and Yang, Zi and Yin, Penghang , booktitle=. 2024 , doi=

    2024

  36. [36]

    Foundations of Large Language Model Compression -- Part 1: Weight Quantization

    Foundations of Large Language Model Compression---Part 1: Weight Quantization , author=. arXiv preprint arXiv:2409.02026 , year=. doi:10.48550/arXiv.2409.02026 , url=

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2409.02026

  37. [37]

    Transactions of the American Institute of Electrical Engineers, Part II: Applications and Industry , volume=

    Statistical Analysis of Amplitude-Quantized Sampled-Data Systems , author=. Transactions of the American Institute of Electrical Engineers, Part II: Applications and Industry , volume=

  38. [38]

    Bell System Technical Journal , volume=

    Spectra of Quantized Signals , author=. Bell System Technical Journal , volume=. 1948 , doi=

    1948

  39. [39]

    , journal=

    Lloyd, Stuart P. , journal=. Least Squares Quantization in. 1982 , doi=

    1982

  40. [40]

    IEEE Transactions on Information Theory , volume=

    Quantizing for Minimum Distortion , author=. IEEE Transactions on Information Theory , volume=. 1960 , doi=

    1960

  41. [41]

    Inequalities , author=

  42. [42]

    2022 , url=

    Kuzmin, Andrey and van Baalen, Mart and Ren, Yuwei and Nagel, Markus and Peters, Jorn and Blankevoort, Tijmen , booktitle=. 2022 , url=

    2022

  43. [43]

    2024 , url=

    Lin, Ji and Tang, Jiaming and Tang, Haotian and Yang, Shang and Chen, Wei-Ming and Wang, Wei-Chen and Xiao, Guangxuan and Dang, Xingyu and Gan, Chuang and Han, Song , booktitle=. 2024 , url=

    2024

  44. [44]

    International Conference on Learning Representations (ICLR) , year=

    OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models , author=. International Conference on Learning Representations (ICLR) , year=

  45. [45]

    International Conference on Learning Representations (ICLR) , year=

    AffineQuant: Affine Transformation Quantization for Large Language Models , author=. International Conference on Learning Representations (ICLR) , year=

  46. [46]

    Empirical Methods in Natural Language Processing (EMNLP) , year=

    Outlier Suppression+: Accurate Quantization of Large Language Models by Equivalent and Optimal Shifting and Scaling , author=. Empirical Methods in Natural Language Processing (EMNLP) , year=

  47. [47]

    Proceedings of the 34th International Conference on Machine Learning , series=

    Distributed Mean Estimation with Limited Communication , author=. Proceedings of the 34th International Conference on Machine Learning , series=. 2017 , url=

    2017

  48. [48]

    2022 , url=

    Vargaftik, Shay and Ben Basat, Ran and Portnoy, Amit and Mendelson, Gal and Ben-Itzhak, Yaniv and Mitzenmacher, Michael , booktitle=. 2022 , url=

    2022

  49. [49]

    2017 , url=

    Alistarh, Dan and Grubic, Demjan and Li, Jerry and Tomioka, Ryota and Vojnovic, Milan , booktitle=. 2017 , url=

    2017

  50. [50]

    2023 , url=

    Frantar, Elias and Ashkboos, Saleh and Hoefler, Torsten and Alistarh, Dan , booktitle=. 2023 , url=

    2023

  51. [51]

    and Keutzer, Kurt , booktitle=

    Dong, Zhen and Yao, Zhewei and Gholami, Amir and Mahoney, Michael W. and Keutzer, Kurt , booktitle=. 2019 , url=

    2019

  52. [52]

    2024 , doi=

    Su, Jianlin and Ahmed, Murtadha and Lu, Yu and Pan, Shengfeng and Wen, Bo and Liu, Yunfeng , journal=. 2024 , doi=

    2024

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.