REVIEW 6 major objections 4 minor 12 references
This paper traces post-training quantization's disparate impact to a forward-pass cascade: reduced bit precision changes and zeros weights, which shifts and narrows the logits, softens the softmax probabilities, and raises loss most for min
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 22:36 UTC pith:XYCWO4OV
load-bearing objection A clear empirical demonstration that PTQ worsens group accuracy gaps, with a plausible but unproven causal chain and an in-sample mitigation that needs more rigor. the 6 major comments →
Explaining How Quantization Disparately Skews a Model
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper claims that post-training quantization's disparate impact is explained by a chain of four forward-pass effects. Quantization changes the weights and pushes a fraction of them to absolute zero, increasing sparsity; the altered weights change the logits, both rotating them in angle and reducing their variance; the logit changes soften the softmax distribution, shrinking the distance to the decision boundary for minority samples; the softened probabilities raise cross-entropy loss and lower accuracy, with the minority group affected most. On the backward pass, the paper shows that per-group gradient norms and the largest Hessian eigenvalues grow for the minority class, indicating that
What carries the argument
The impact flow: precision reduction changes weights and zeros a fraction of them; those changes rotate and narrow the logits (measured by cosine distance and logit variance); narrower logits soften softmax outputs and reduce the distance to the decision boundary; the softened probabilities raise loss and widen the accuracy gap. The paper formalizes the gap with FVO, the largest pairwise group-accuracy difference, and reads the optimization state from per-group gradient norms and the largest Hessian eigenvalue.
Load-bearing premise
The causal order of the cascade is assumed: the paper never changes one factor (precision, sparsity, logit variance) while holding the others fixed, and it assumes quantization-induced sparsity acts like pruning-induced sparsity, so the pruning disparity result carries over.
What would settle it
Quantize the same trained model to int4 under standard rounding and under rounding that assigns zeroed weights to the smallest nonzero grid point; if minority accuracy loss and logit-variance collapse are identical in both, sparsity is not the driver. A null result on the broader claim would be a quantized model whose per-group accuracy losses are uniform across groups despite visible weight and sparsity changes.
If this is right
- At low precisions like int2 and int4, quantization-induced sparsity makes the model behave like a magnitude-pruned one, so the disparate accuracy effects documented for pruning carry over to quantized models.
- Lower precision reduces logit variance and shifts softmax distributions left for minority samples, meaning quantized models make low-confidence, boundary-adjacent predictions for those groups.
- Per-group gradient norms and top Hessian eigenvalues increase for the minority class, indicating that quantization leaves the model farther from a good optimum for that group than for the majority.
- Combining under/over-sampling, weighted cross-entropy, and mixed-precision quantization-aware training achieves the highest overall accuracy and lowest fairness violation at int4, so fairness at low precision does not have to cost global accuracy.
- Activation quantization, not studied directly, would likely resemble or worsen the same disparity, since it adds another source of representational error to the same cascade.
Where Pith is reading between the lines
- A testable extension the paper leaves open: post-training logit rescaling per group. If variance collapse is the mechanism, re-expanding minority logit variance should recover accuracy without retraining.
- The same chain should generalize beyond vision: weight-quantized language models should show larger perplexity or capability drops on underrepresented languages or tokens, a prediction the paper does not test.
- Because the paper's factors co-vary, an intervention study—quantizing with and without zeroing, or with artificially variance-preserving rounding—would separate genuine causes from markers.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies how post-training quantization (PTQ) affects per-class accuracy on UTKFace with ResNet18. It observes that lower precision increases the disparity in accuracy between majority and minority groups, with the smallest class (Others) suffering the most. The authors propose a causal chain: quantization changes weights and induces sparsity, which reduces logit variance and softmax probability variance, raises loss, lowers accuracy disproportionately for minority classes, and degrades the optimization state as measured by group gradient norms and Hessian eigenvalues. They then propose a mitigation combining under/over-sampling, a weighted cross-entropy loss with fixed weights, and mixed-precision QAT, reporting reduced FVO and improved overall accuracy. The paper frames its contribution as identifying the factors of PTQ that cause disparity.
Significance. If the proposed causal chain were established, the paper would go beyond prior observational studies of compression-induced unfairness by offering a mechanistic account and a practical mitigation recipe. The empirical observations are plausible and the use of multiple diagnostics (weight distance, sparsity, cosine distance, logit variance, softmax variance, gradient norm, Hessian eigenvalues) is a useful descriptive contribution. FVO is an interpretable multi-class disparity measure. However, the central claim is supported only by correlations across precision levels, the sparsity mechanism is confounded with overall weight perturbation, the fairness interpretation of FVO is not justified, and the mitigation is evaluated in-sample with hand-selected weights. No code, repeated-seed results, or the promised appendix are provided. The paper is a reasonable preliminary empirical study, but as written its central explanatory claim is overclaimed and several load-bearing points require substantial additional work.
major comments (6)
- [Sec. 4.1, Eqs. (5)-(6), Contribution 2] The claim that 'increased sparsity' is a causal factor in quantization's disparate impact is unsupported. In PTQ, reducing precision simultaneously rounds every weight and sends some weights to the zero bin, so sparsity and overall weight perturbation are perfectly confounded. The argument that a quantized network is 'more like a pruned network' and the citation to Tran et al. (2022) do not separate these mechanisms: pruning zeroes selected weights and leaves the rest at their original values, while quantization perturbs all weights. No experiment holds weight perturbation fixed while varying sparsity, or vice versa. The observed co-occurrence of higher sparsity, lower logit variance, and larger disparity does not establish sparsity as the driver; it may be an epiphenomenon of global weight perturbation.
- [Secs. 4.2-4.3, Figs. 5-7] The central explanatory chain—weight change → reduced logit variance → softened softmax → higher loss → lower accuracy for minority groups—is inferred from correlations across precision levels, not from interventions or mediation analysis. The paper measures each quantity at several precisions and observes that the minority group is worst on each, which is consistent with a single confounder (reduced precision) affecting all these quantities. To support the causal arrow, the paper would need, for example, an experiment that perturbs logit variance directly (e.g., temperature scaling) while holding weights fixed, or a mediation-style analysis controlling for the other factors. As written, Contribution 2 is a set of co-occurring measurements rather than an established mechanism.
- [Sec. 3.1, Eq. (2)] The statement that minimizing FVO 'inherently captures equalized odds' is not correct as stated. FVO is the maximum absolute accuracy gap between groups. Accuracy depends on group base rates: equalized odds (equal TPR and FPR across groups) does not imply equal accuracy when the proportions of labels differ, and equal accuracy does not imply equal TPR/FPR. Thus FVO is not a valid surrogate for equalized odds without additional assumptions about base rates. This matters because FVO is the paper's primary fairness metric and the target of the mitigation.
- [Secs. 5.1-5.3, Discussion, Figs. 11-12] The mitigation evaluation is in-sample and the WCR weights are hand-selected on the same test data. The weights [0.1,0.1,0.1,0.1,0.6] are justified by the observation that Others has the lowest test accuracy and highest loss, so the reported FVO reduction is partly constructed by design. There is no held-out set or sensitivity analysis. The Discussion also concedes that QAT alone achieves lower FVO than Fair QAT for many precisions, which limits the claimed benefit of the additional sampling and WCR components. Finally, Section 5 promises an appendix with results on other datasets, but no appendix is present in the manuscript, so the generalization claim is unsupported.
- [Sec. 4.6] The attribution of the Others group's test-set performance to 'relatively complex facial features' is asserted without evidence. Equal training accuracy at the end of balanced training and lower test accuracy is equally consistent with distribution shift, label noise, or sampling artifacts in the test set. This assertion is load-bearing because it motivates the WCR weights. It needs to be supported by additional analyses (e.g., per-sample difficulty measures, or validation on held-out folds) or removed.
- [Figs. 1, 5d, 7, 8, 9] All empirical trends are reported without error bars, confidence intervals, or repeated seeds. Given that these observations are the basis for the paper, at least multiple seeds and standard deviations should be reported for the core accuracy and loss results. Without them, it is difficult to assess whether the reported disparities and their changes with precision are statistically reliable.
minor comments (4)
- [Eqs. (8) and (9)] These equations use |G| (the number of groups) as the normalization factor for a sum over samples. This should be the number of samples in the group, e.g., |D_g| or N_g, or the notation needs to be clarified.
- [Sec. 4.2] The term 'Distance To the Decision Boundary (DTDB)' is defined as the softmax probability. This is not a distance to the decision boundary in the usual geometric sense; consider renaming it (e.g., 'softmax confidence') or providing a formal definition that justifies the name.
- [Sec. 4.2, Fig. 5] The text says 'norm based metrics fail to capture it,' but Figures 5b and 5c show clear differences in L1 and L2 distances across groups and precisions. The claim is stronger than the evidence; soften it or justify what 'capture' means.
- [Sec. 5.3 and Discussion] The Discussion says that for many precisions QAT alone has lower FVO than Fair QAT, which seems to contradict the abstract's claim that integrating mixed precision QAT with sampling and weighted loss provides fair deployment. Please clarify for which precisions FairQAT is recommended and why.
Circularity Check
The core disparity observation is independently measured, but the sparsity factor is imported from the authors' own prior pruning work, and the mitigation weights are fit to the observed test-set disparity, making part of the reported fairness improvement circular.
specific steps
-
self citation load bearing
[Sec 4.1 (Changes in the Weights)]
"Clearly, reducing the precision increases the sparsity of the network, therefore, making it more like a pruned network (by weight magnitude). As shown in Tran et al. (2022), increasing the pruning ratio, i.e., increasing sparsity, has a disparate impact on the accuracy of the model. With higher sparsity in a quantized model, disparity worsens, analogous to the impact of sparsity in a pruned model."
Contribution 2 lists 'increased sparsity' as a causal factor in the forward-pass chain, but the only support offered is an analogy to pruning and a citation to Tran et al. (2022), a prior work co-authored by Jung-Eun Kim, an author of this paper. The paper's own experiments show that sparsity increases with lower precision, but they never manipulate sparsity independently of the global weight perturbation caused by quantization. Thus 'sparsity causes disparity' is imported from the authors' own prior work via analogy, rather than demonstrated here. Without that citation, sparsity is merely a correlate, not an established causal factor, and the claimed explanation rests on a load-bearing self-citation.
-
fitted input called prediction
[Sec 5.1 and Sec 6]
"Next, we train a fairer model θ_f using both U-O and WCR. ... We choose WCR with weights([0.1,0.1,0.1,0.1,0.6]) such that Others has the highest weight and the rest of the classes are given equal weight. ... The aforementioned WCR weights were chosen as a result of the example difficulty observed for the Others class."
The WCR class weights are selected after the authors observed, on the same evaluation data (Sec 4.3 and 4.6), that Others has the lowest accuracy and highest loss. Assigning Others a weight of 0.6 and then reporting that Fair QAT maximizes OA and minimizes FVO (Fig. 12) is partly a consequence of this in-sample fit: upweighting the known worst-performing group on the evaluation distribution mechanically reduces its loss and tends to raise its accuracy. The reported fairness improvement is therefore not an independent out-of-sample prediction; it is a post-hoc tuning of the loss function to the observed disparity.
full rationale
The paper's central observation—that post-training quantization exacerbates accuracy disparity across groups—is independently measured and not circular. The forward-pass chain (weight change → logit change/variance reduction → softmax probability shift → higher loss and lower accuracy) is largely descriptive and supported by direct measurements on the model. The Hessian-eigenvalue and gradient-norm analyses are also independent measurements of the optimization landscape. However, two elements introduce partial circularity. First, the claim that 'increased sparsity' is a causal factor is supported only by an analogy to pruning and by citing Tran et al. (2022), a paper co-authored by one of the present authors; the authors never isolate sparsity from the overall weight perturbation, so this factor is imported from their own prior work rather than derived here. Second, the mitigation's class weights are chosen based on the observed test-set difficulty of Others, and the fairness improvement is then reported on that same data—so the improvement is partly by construction, not an independent prediction. These issues do not invalidate the core disparity finding, but they make the explanatory and mitigation claims partially circular. Score 5 reflects the presence of a load-bearing self-citation and a fitted mitigation input, short of the fully forced circularity that would warrant 6+.
Axiom & Free-Parameter Ledger
free parameters (2)
- WCR class weights a_g =
[0.1, 0.1, 0.1, 0.1, 0.6]
- QAT hyperparameters (learning rate, etc.) =
int4 tuning used for all precisions
axioms (4)
- domain assumption Gradient norms and top Hessian eigenvalues of a quantized network indicate the quality of the optimization state and scope for improvement.
- ad hoc to paper Accuracy parity (FVO) captures equalized odds.
- domain assumption Quantization-induced sparsity is analogous to unstructured weight pruning, so pruning's disparity results carry over.
- ad hoc to paper For the Others class, lower test accuracy after balanced training is due to intrinsic example difficulty (complex facial features).
Cite this review
Pith. "Pith review of Explaining How Quantization Disparately Skews a Model." pith.science (2026). https://pith.science/paper/XYCWO4OV
@misc{pith2026250907222,
author = {Pith},
title = {Pith review of: Explaining How Quantization Disparately Skews a Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/XYCWO4OV}},
note = {Machine review of arXiv:2509.07222}
}
read the original abstract
Post Training Quantization (PTQ) is widely adopted due to its high compression capacity and speed with minimal impact on accuracy. However, we observed that disparate impacts are exacerbated by quantization, especially for minority groups. Our analysis explains that in the course of quantization there is a chain of factors attributed to a disparate impact across groups during forward and backward passes. We explore how the changes in weights and activations induced by quantization cause cascaded impacts in the network, resulting in logits with lower variance, increased loss, and compromised group accuracies. We extend our study to verify the influence of these impacts on group gradient norms and eigenvalues of the Hessian matrix, providing insights into the state of the network from an optimization point of view. To mitigate these effects, we propose integrating mixed precision Quantization Aware Training (QAT) with dataset sampling methods and weighted loss functions, therefore providing fair deployment of quantized neural networks.
Figures
Reference graph
Works this paper leans on
-
[1]
Un- covering and mitigating algorithmic bias through learned latent structure
Alexander Amini, Ava P Soleimany, Wilko Schwarting, Sangeeta N Bhatia, and Daniela Rus. Un- covering and mitigating algorithmic bias through learned latent structure. InProceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 289–295,
work page 2019
-
[3]
Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.arXiv preprint arXiv:1510.00149, 2015a. Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network.Advances in neural information processing systems, 2...
-
[6]
On large-batch training for deep learning: Generalization gap and sharp minima
Nitish Shirish Keskar, Jorge Nocedal, Ping Tak Peter Tang, Dheevatsa Mudigere, and Mikhail Smelyanskiy. On large-batch training for deep learning: Generalization gap and sharp minima. In5th International Conference on Learning Representations, ICLR 2017,
work page 2017
-
[9]
A white paper on neural network quantization.arXiv preprint arXiv:2106.08295,
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization.arXiv preprint arXiv:2106.08295,
-
[12]
Michael Zhu and Suyog Gupta. To prune, or not to prune: exploring the efficacy of pruning for model compression.arXiv preprint arXiv:1710.01878,
-
[1991]
Morgan Kaufmann Publishers Inc. ISBN 1558602224. Christina Wadsworth, Francesca Vera, and Chris Piech. Achieving fairness through adversarial learning: an application to recidivism prediction.arXiv preprint arXiv:1807.00199,
-
[2016]
What do com- pressed deep neural networks forget?arXiv preprint arXiv:1911.05248,
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome. What do com- pressed deep neural networks forget?arXiv preprint arXiv:1911.05248,
Pith/arXiv arXiv 1911
-
[2017]
Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization.arXiv preprint arXiv:2305.14152,
-
[2019]
Efficient and Robust Quantization-aware Training via Adaptive Coreset Selection
Xijie Huang, Zechun Liu, Shih-Yang Liu, and Kwang-Ting Cheng. Efficient quantization-aware training with adaptive coreset selection.arXiv preprint arXiv:2306.07215,
work page internal anchor Pith review Pith/arXiv arXiv
-
[2020]
URLhttps://proceedings. mlr.press/v119/wang20c.html. Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuandong Tian, Peter Vajda, and Kurt Keutzer. Mixed precision quantization of convnets via differentiable neural architecture search.arXiv preprint arXiv:1812.00090,
-
[2022]
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323,
-
[2023]
Prun- ing vs quantization: Which is better?arXiv preprint arXiv:2307.02973,
13 Andrey Kuzmin, Markus Nagel, Mart van Baalen, Arash Behboodi, and Tijmen Blankevoort. Prun- ing vs quantization: Which is better?arXiv preprint arXiv:2307.02973,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.