Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

WUSH, a non-orthogonal blockwise transform built from a Hadamard matrix and second-moment statistics, is the closed-form optimal linear transform for FP block quantizers and nearly optimal for INT block quantizers under round-to-nearest Abs

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

WUSH is a closed-form, data-aware, near-optimal block transform for 4-bit weight-activation quantization that improves LLM accuracy over Hadamard baselines.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection WUSH is a genuinely new closed-form transform with real empirical gains on MXFP4, but the headline optimality theorem is proven for a pseudo-noise surrogate of RTN, not the exact quantizer — worth a careful revision and a serious referee. the 3 major comments →

arxiv 2512.00956 v3 pith:WS4AZZ2D submitted 2025-11-30 cs.LG cs.CL

WUSH: Near-Optimal Adaptive Transforms for LLM Quantization

Jiale Chen , Vage Egiazarian , Roberto L. Castro , Torsten Hoefler , Dan Alistarh This is my paper
classification cs.LG cs.CL
keywords LLM quantizationblockwise transformHadamard transformweight-activation quantizationMXFP4round-to-nearestpost-training quantizationWUSH
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper addresses a standing question in LLM quantization: why fixed Hadamard rotations help, and whether a better transform is possible. It derives, rather than learns, the closed-form optimal blockwise linear transform for joint weight-and-activation quantization under standard round-to-nearest AbsMax block quantizers. The construction, called WUSH, combines a normalized Hadamard matrix with a data-dependent rescaling and rotation derived from the SVD of the calibration cross-covariance, and it is non-orthogonal. Under a pseudo-noise model of quantization error, WUSH is exactly optimal for floating-point formats and optimal up to a small factor for integer formats with tail-bounded data. Empirically it improves W4A4 accuracy over Hadamard baselines by up to 2.8 average points on Llama-3.1-8B-Instruct MXFP4 RTN while remaining cheap enough for fused GPU inference kernels.

Core claim

The central claim is that, for each quantization block, the transform pair T_wush = H S^{-1/2} U^T W'^T and T_xvsh = T_wush^{-T} minimizes the expected squared error of the quantized product of weights and activations. Here W' and X' are Cholesky factors of the block's second-moment matrices, and U S V^T is the SVD of W'^T X'. The construction is exactly optimal for FP quantizers under multiplicative pseudo-noise and asymptotically optimal for INT quantizers under scale-proportional pseudo-noise, with the S^{1/2} rescaling and the Hadamard factor both necessary; dropping either reduces it to the same loss as any orthogonal transform.

What carries the argument

The transform T_wush = H S^{-1/2} U^T W'^T (mnemonic WUSH) is the central object. It acts as: a whitening-like rescaling by S^{-1/2} that balances the quantization-error budget across principal directions; a rotation by U^T that aligns with the cross-covariance of the calibration activation and weights; an isotropic Hadamard factor H that spreads remaining variance; and a Cholesky factor W' that accounts for within-block correlation. The identity T_xvsh = T_wush^{-T} couples the weight and activation transforms, converting the two-sided quantization loss into a one-sided expectation E_{y,ε}||T^{-1}ε(Ty)||^2 that can be minimized in closed form.

Load-bearing premise

The proof's optimality rests on modeling the exact RTN AbsMax quantizer as zero-mean independent pseudo-noise—multiplicative for FP and scale-proportional for INT—and on dropping the interaction term between the weight and activation quantization errors; if real round-to-nearest errors correlate within a block or depend on the data beyond this noise model, WUSH is not mathematically guaranteed to be the optimal transform for exact RTN.

What would settle it

Brute-force search over all block transforms (e.g., d=2 or d=4) on synthetic weights/activations with exact RTN AbsMax quantization: any transform with lower blockwise loss than WUSH for an FP format would falsify the FP optimality claim. Alternatively, craft an FP quantizer with deterministic, within-block-correlated errors (all errors sharing sign) and check whether the predicted lower bound d^{-1}(tr S)^2 is violated.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • For FP block quantizers, under the paper's noise model, any orthogonal transform—including Hadamard alone—leaves quantization loss unchanged; only the non-orthogonal WUSH construction reduces it, by up to a factor d in the extreme-outlier regime.
  • For INT block quantizers, the gap between Hadamard and the optimal bound grows with outlier strength; WUSH closes that gap up to a o(1) factor for tail-bounded data.
  • Because the transform is calibration-derived and closed-form, it can be recomputed per model/layer without training or fine-tuning, and it integrates with round-to-nearest and second-order weight-update quantization schemes.
  • The per-block specialization survives GPU efficiency: fused kernels match Hadamard throughput within about 1.3% while the FP4 matmul gives up to ~6.6x per-layer speedup over BF16.
  • On Llama-3.1-8B-Instruct MXFP4, WUSH moves average accuracy from 70.45 to 73.21 with RTN, and from 73.65 to 74.35 with second-order updates, nearly closing the gap to higher-precision formats.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same derivation suggests a recipe for other quantizers: model the error as value-proportional (FP-like) or scale-proportional (INT-like) noise, then the optimal transform is a whitened-Hadamard rotation in the SVD basis; closed forms for non-uniform codebooks or different group sizes could be obtained by changing the noise model.
  • The paper notes MXFP4 behaves like a hybrid of FP and INT; a noise model mixing multiplicative and scale-proportional terms might yield an even better transform for MXFP4 specifically, and the layer-wise loss tables suggest room remains.
  • If a diagonal approximation of the data-aware component preserves most of the gain (as the authors propose for future work), the online cost per token could drop, making the method viable for smaller block sizes where per-block matrices dominate memory.
  • A testable prediction of the mechanism: WUSH's advantage over Hadamard should grow with the anisotropy (singular-value spread) of the calibration block; measuring layerwise gains against trace(S)^2/tr(S^2) would confirm this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes WUSH, a blockwise, data-dependent linear transform for joint weight-activation quantization under RTN AbsMax block quantizers. The transform has the form T_wush = H S^{-1/2} U^T W'^T (Eq. 8) and is derived by minimizing an L2 output-loss surrogate under a stochastic pseudo-noise model of the quantizer. The authors prove a closed-form optimality result for FP quantization and a near-optimality result for INT quantization under tail-bounded distributions (Theorem 1), and they integrate WUSH with RTN and GPTQ in a practical GPU kernel. Experiments on Llama-3 and Qwen-3 models show consistent accuracy gains over Hadamard-based baselines, especially for MXFP4, and kernel throughput close to optimized Hadamard baselines.

Significance. If the theoretical claim is taken at face value, this is a significant contribution: it gives the first closed-form optimal transform for FP block quantizers and a principled explanation for the empirical success of Hadamard rotations, while also providing a practical, efficient kernel and solid end-to-end accuracy gains. The paper is also well positioned in the current literature: the comparison against MR-GPTQ, QuaRot, SpinQuant, and SmoothQuant is appropriate, and the open-source release is a strength. However, the central optimality proof is for a stochastic surrogate of the exact RTN quantizer, not for the deterministic quantizer used in deployment; this gap must be addressed before the 'provably near-optimal' claim can be accepted as stated.

major comments (3)
  1. [Section 4.1, Theorem 1, Eq. (18), Eq. (23)] The optimality theorem is proved for a stochastic pseudo-noise model of RTN, not for the exact deterministic quantizer. For FP, Eq. (18) assumes epsilon(alpha)=diag(eta)alpha with i.i.d. zero-mean eta; for INT, Eq. (23) assumes epsilon(alpha)=||alpha||_inf eta. In reality, RTN AbsMax errors are deterministic, share a common scale within a block, and are not independent across coordinates. The deployed method (Algorithm 1, line 14; Eq. (10)) uses exact RTN. Thus Theorem 1 supports optimality of WUSH only for the surrogate objective, not for the exact quantizer. This is load-bearing for the abstract's 'provably near-optimal' claim. I request either a rephrasing of the theorem to explicitly claim optimality under the stated pseudo-noise model, or a synthetic exact-RTN check (e.g., brute-force or coordinate-descent minimization of the exact blockwise loss on small blocks) to demonstrate that
  2. [Eq. (13) and surrounding derivation] The split of the loss into two non-negative terms relies on a first-order approximation that drops the cross-term epsilon(T_W w)^T epsilon(T_X x) and its square. No bound or empirical estimate is given for the magnitude of this term. At 4-bit precision the quantization errors are not necessarily small, so the omitted term can be of the same order as the retained terms. This affects the equality conditions and hence the exact optimality claim for FP and the near-optimality factor for INT. I recommend adding a quantitative bound, or at least a numerical experiment measuring the dropped term on real calibration data, to justify the approximation.
  3. [Section 3, after Eq. (5)] The paper states 'We will approximate the layerwise loss as ell approximately sum_i ell(i)' and then minimizes the blockwise losses independently. This ignores cross-block error correlations. Since the objective is the norm of the sum of blockwise errors, correlations between blocks can in principle change the optimal transform. This is an approximation that is not discussed or tested. I would like to see at least a brief justification (or an experiment comparing the independently optimized blockwise loss to the true layerwise loss) to support the claimed optimality of the per-block construction.
minor comments (6)
  1. [Abstract vs. Section 5] The abstract reports 'up to 5.8x per-layer throughput' while Section 5 and the full-text abstract report 'up to 6.63x'. Please reconcile these numbers.
  2. [Author list] Typesetting errors in author names: 'V age Egiazarian' and 'T orsten Hoefler' have stray spaces.
  3. [Eq. (26) and Eq. (28)] The notation 'do(1)-1' is ambiguous; it should be typeset as d^{o(1)-1} to avoid confusion with a product d * o(1).
  4. [Section 4.3.2, Eq. (30)] For general distributions the bound is only within a factor d, which is weak for the block sizes used (d=32). The paper should state more prominently that 'near-optimal' for INT applies only to tail-bounded distributions, and that the general-distribution guarantee is only a d-factor bound.
  5. [Table 4 (Qwen3-32B)] For Qwen3-32B MXFP4 RTN, WUSH average accuracy (82.01) is slightly below Hadamard (82.15). The text says 'similar trends' across models; please acknowledge this exception explicitly.
  6. [Algorithm 1] The damping ratio lambda is an input parameter but its value/selection is not reported. The theoretical derivation does not use damping; please state how lambda is chosen in the experiments and whether the results are sensitive to it.

Circularity Check

0 steps flagged

No circular derivation; the optimality theorem is self-contained for the stated pseudo-noise quantizer models, with an acknowledged modeling gap rather than a circular reduction.

full rationale

The derivation chain is a conventional minimization: start from blockwise loss Eq. (5), impose an unbiased quantizer and T_W = T_X^{-⊤}, split the loss into two non-negative terms Eq. (13), reduce the two-sided problem to the one-sided loss Eq. (14), parameterize all invertible transforms via SVD Eq. (16), and then derive lower bounds (Eqs. (20), (25)-(29)) that are attained or nearly attained by U'=H, S'=S^{1/2}, R=I. This is a self-contained optimization problem. The WUSH transform in Eq. (8) is the closed-form solution of that optimization, not a fitted constant renamed as a prediction. The only real caveat is model adequacy: Theorem 1 proves optimality for the zero-mean pseudo-noise surrogates Eqs. (18) and (23), not for the exact deterministic RTN AbsMax quantizer, and the paper itself labels the dropped ε^⊤ε term a first-order approximation in Eq. (13) and describes its error models as modeling choices ('we can model', 'We provide more rigorous justifications'). That is an assumption-to-conclusion gap about whether the surrogate captures RTN, not circularity: the assumptions, objective, and theorem are all stated independently, and no parameter is fitted to the target layer-loss table. Self-citations to Egiazarian et al. (2025) and Chen et al. (2025) are used for MR-GPTQ baselines, kernel support, and an alternative inter-block update formulation, not for the load-bearing optimality argument. The empirical evaluations on held-out benchmarks provide independent support for the usefulness of WUSH. Overall, no circular step is exhibited; the central derivation is mathematically self-contained given its stated assumptions.

Axiom & Free-Parameter Ledger

1 free parameters · 8 axioms · 0 invented entities

The central theoretical claim rests on a surrogate noise model for RTN and a first-order approximation, plus distributional assumptions for the INT near-optimality gap. The only explicit tunable is the damping ratio λ in Algorithm 1. No new physical or mathematical entities are postulated.

free parameters (1)
  • damping ratio λ = not reported
    Added to the calibration Hessian in Algorithm 1 before Cholesky decomposition; handles rank-deficient second moments and affects the computed transforms. Its value is a free calibration choice.
axioms (8)
  • ad hoc to paper RTN AbsMax quantization error is a zero-mean independent random vector: ε(α)=diag(η)α for FP (Eq. 18) and ε(α)=∥α∥∞ η for INT (Eq. 23), with η i.i.d. and Eη=0.
    Load-bearing surrogate for deterministic RTN; exact RTN optimality is not proven under this model.
  • domain assumption The quantizer is unbiased and the ε(T_W w)^T ε(T_X x) cross term is negligible (first-order approximation) in Eq. (13).
    Allows splitting the two-sided loss into two independent one-sided expectations; this is an approximation, not an exact property of deterministic RTN.
  • domain assumption Columns of W and X within a block are i.i.d. samples from independent distributions w∼D_w and x∼D_x (Section 4.1).
    Probabilistic reformulation of blockwise loss; real weight and activation columns are not strictly i.i.d.
  • domain assumption The layerwise loss is approximated as the sum of per-block losses, with block-diagonal transforms and quantization groups aligned to block size d (Eqs. 3-5).
    Blockwise independence is a design constraint; the approximation ignores cross-block error terms.
  • domain assumption For INT near-optimality up to d^{o(1)}, D_y is zero-mean multivariate Gaussian or Laplacian (Lemma 3; Section 4.3.2).
    Without tail-boundedness, the proved gap degrades to a d-factor.
  • standard math W'(i), X'(i) are full-rank Cholesky factors of second moments; damping λ is applied otherwise (Section 3, Appendix A.1).
    Needed for invertibility and SVD; standard linear algebra.
  • ad hoc to paper Smooth FP model approximates 1+x≈2^x so casting error is multiplicative (Appendix A.4.2).
    Justifies FP error model; approximation error is bounded (~0.086) but not exact.
  • standard math Hadamard matrix exists for block size d a power of two (Section 3).
    H is the data-agnostic backbone of WUSH.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of WUSH: Near-Optimal Adaptive Transforms for LLM Quantization." pith.science (2026). https://pith.science/paper/WS4AZZ2D

@misc{pith2026251200956,
  author       = {Pith},
  title        = {Pith review of: WUSH: Near-Optimal Adaptive Transforms for LLM Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WS4AZZ2D}},
  note         = {Machine review of arXiv:2512.00956}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors. Prior transform-based mitigations (e.g., Hadamard rotations) are fixed and data-agnostic, and their optimality for quantization has remained unclear. We derive closed-form optimal linear blockwise transforms for joint weight-activation quantization under standard RTN AbsMax-scaled block quantizers, covering both integer and floating-point formats. The resulting construction, WUSH, combines a Hadamard backbone with a data-dependent second-moment component to form a non-orthogonal transform that is provably near-optimal for FP and INT quantizers under mild assumptions while admitting an efficient fused GPU implementation. Empirically, WUSH improves W4A4 accuracy over the strongest Hadamard-based baselines (e.g., on Llama-3.1-8B-Instruct in MXFP4, it gains +2.8 average points with RTN and +0.7 with GPTQ) while delivering up to 5.8$\times$ per-layer throughput over BF16 via FP4 MatMul. Source code is available at https://github.com/IST-DASLab/WUSH.

Figures

Figures reproduced from arXiv: 2512.00956 by Dan Alistarh, Jiale Chen, Roberto L. Castro, Torsten Hoefler, Vage Egiazarian.

Figure 1
Figure 1. Figure 1: 2D illustration of how transforms shape the one-sided quantization error ∥T −1 ε (T y) ∥ 2 in Eq. (14) under FP and INT block quantizers. Each row corresponds to a different transform T (identity; Hadamard: 45° rotation; calibrated Hadamard: equally spreading energy into each dimension; whitening; WUS: WUSH without Hadamard; WUSH), and columns show FP and INT quantization in both original and transformed s… view at source ↗
Figure 2
Figure 2. Figure 2: Per-layer MXFP4 (group size G=32) GEMM speedups relative to BF16 at batch size M=1024 for Qwen3-8B, Qwen3-14B, and Llama-3.1-70B. We compare kernels without transform or quantization (None), with Hadamard and quantization (H + Quant), and with WUSH and quantization (WUSH + Quant). For block size G and a layer with K=C×G input and N output channels, the Hadamard matrix size is (G,G), while the WUSH size is … view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of different transforms on Qwen3-8B. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Rotation can equalize marginals without alleviating multivariate outliers. Top: an anisotropic 2D distribution with unequal coordinate marginals. Bottom: after an orthogonal rotation, the two marginals become nearly identical, and their average appears more Gaussian-like. Nevertheless, the joint distribution retains the same eigenvalues (spec￾trum) under rotation, so extreme points remain extreme in the mu… view at source ↗
Figure 5
Figure 5. Figure 5: The two curves y = 1 + x and y = 2x are close for 0 ≤ x ≤ 1. Using the approximation 1 + (·) ≈ 2 (·) for 2 −bm ∈ [0, 1] (visualized in [PITH_FULL_IMAGE:figures/full_fig_p029_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of different transforms on Qwen3-14B for both NVFP4 and MXFP4 [PITH_FULL_IMAGE:figures/full_fig_p036_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of different transforms on Qwen3-32B for both NVFP4 and MXFP4 [PITH_FULL_IMAGE:figures/full_fig_p037_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of different transforms on Llama-3.2-3B-Instruct for both NVFP4 and [PITH_FULL_IMAGE:figures/full_fig_p038_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison of different transforms on Llama-3.1-8B-Instruct for both NVFP4 and [PITH_FULL_IMAGE:figures/full_fig_p039_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Price of metric universality in vector quantization is at most 0.11 bit

    cs.IT 2026-02 conditional novelty 6.0

    A universal vector-quantization codebook exists that is within 0.11 bit/coordinate of covariance-adaptive waterfilling simultaneously for all input covariances, for Gaussian weights.

  2. High-Rate Quantized Matrix Multiplication I

    cs.IT 2026-01 unverdicted novelty 5.0

    High-rate quantization theory yields accurate approximations for the distortion of absmax INT and FP schemes in generic weight-plus-activation matrix multiplication.

Reference graph

Works this paper leans on

2 extracted references · cited by 2 Pith papers

  1. [2023]

    Gaussian-like,

    URL https://www.opencompute.org/documents/ocp-microscaling-formats-m x-v1-0-spec-final-pdf. 3 NVIDIA. Nvidia blackwell architecture technical brief. Technical report, NVIDIA, 2024. URLhttps://resources.nvidia.com/en-us-blackwell-architecture. 3 Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von ...

    arXiv 2024

  2. [2024]

    1, 3 Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort

    URL https://proceedings.mlsys.org/paper_files/paper/2024/file/42a452 cbafa9dd64e9ba4aa95cc1ef21-Paper-Conference.pdf. 1, 3 Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. Spinquant: LLM quantization with learned rotations. InThe Thirteenth Internation...

    2024

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.