Pith. sign in

REVIEW 4 major objections 5 minor 15 references

Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Models can keep their accuracy after quantization while changing the reasons for their decisions.

desk verdict Useful empirical first pass on how PTQ changes CNN explanations, but the central accuracy-vs-interpretability claim is unsupported without an accuracy table and LIME reproducibility details. read the letter →

arxiv 2607.22872 v1 pith:SXTCHZXF submitted 2026-07-24 cs.LG cs.CV

Kazi Kamruzzaman Rabbi , Md. Zami Al Zunaed Farabe , M. Sohel Rahman This is my paper · ORCID
classification cs.LGcs.CV
keywords Post-trainingquantizationModelinterpretabilityExplainableAIGrad-CAMLIMECNNEdgedeploymentFeatureattributionstability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Post-training quantization (PTQ) is widely used to shrink neural networks for edge devices, and past work showed it largely preserves classification accuracy. This paper establishes that accuracy preservation does not imply that the model's internal reasoning is preserved. By comparing Grad-CAM spatial attention and LIME input-level attributions between full-precision and INT8/INT4 versions of five CNN architectures, the authors find that explanation stability is strongly architecture-dependent. DenseNet161's explanations stay nearly identical to the full-precision model at both precisions, while EfficientNet-B0's LIME attributions collapse to a Pearson correlation of 0.228 at INT8 and 0.109 at INT4 even though its accuracy remains competitive. The practical stakes: choosing a deployment architecture for interpretability-sensitive applications requires validation of explanation stability, not just accuracy.

What carries the argument

The machinery is a paired heatmap-comparison pipeline. Each architecture is fine-tuned, quantized with the GPFQ weight-only method to INT8 and INT4, and then asked to explain the same images through two complementary lenses: Grad-CAM, which produces spatial attention maps from gradient-weighted final-layer feature maps, and LIME, which fits a local surrogate model to estimate input-level feature attribution. The full-precision and quantized explanation maps are then compared with the Pearson correlation coefficient, the structural similarity index, and top-20% intersection-over-union, with deletion/insertion AUC acting as a faithfulness check on whether the explanations track the model's actual confidence. This turns the question 'did quantization change the model's reasoning?' into a numerical distance between explanation maps.

What would settle it

Measure LIME explanation similarity between two independently trained full-precision EfficientNet-B0 models on the same images; if that fp32-versus-fp32 PCC is already near 0.228, the low fp32-versus-INT8 score cannot be attributed to quantization.

Watch

Extended reading notes

Core claim

The paper's central claim is that static PTQ to INT8 and INT4 can substantially change which input evidence a model uses even when classification accuracy is stable, so accuracy is not a trustworthy proxy for interpretability stability. The supporting result is a head-to-head measurement: across Food-101 and Poribohon-BD subsets, DenseNet161 maintains LIME PCC of 0.987 at INT8 and 0.984 at INT4, while EfficientNet-B0 drops to 0.228 and 0.109 respectively; Grad-CAM stays more stable within each architecture, but perturbation-based attributions diverge sharply. The paper interprets this as evidence that architecture selection matters as much as quantization strategy for trustworthy deployment, and that quantized models that agree with full-precision predictions can still be reaching the same predictions for different reasons.

Load-bearing premise

The comparison treats the full-precision model's explanations as the reference that quantized explanations should match, so if fp32 explanations are themselves noisy or unfaithful, a low similarity score does not prove that quantization damaged the model's reasoning.

Editorial extensions

If this is right

  • Transparency-focused deployments should validate explanation stability per architecture after PTQ, since accuracy curves can look healthy while attribution maps drift.
  • DenseNet161 is the safer pick when interpretability is a hard requirement: its explanation similarity to fp32 stays above 0.96 for both bit-widths and both explainers.
  • EfficientNet-B0 should be kept at full precision in interpretability-critical settings; at INT8 its LIME similarity is already below 0.23.
  • INT8 explanations are statistically closer to fp32 than INT4 are, so teams that want the deployed model to reason like the original model should prefer INT8 despite INT4's larger compression.
  • When quantization does flip a prediction, the model tends to attend to different image regions, so the failure is not just a label change but an evidence shift.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the tested CNNs, the same measurement pipeline could be applied to vision transformers; their attention mechanisms may show either stronger or weaker explanation stability under PTQ, since the paper explicitly leaves that architecture class out.
  • A useful control the paper does not run is fp32-versus-fp32 variability: if two independent full-precision runs of EfficientNet-B0 already disagree in LIME attributions as much as fp32-versus-INT8 do, part of the reported degradation is innate explanation noise, not quantization damage.
  • The faithfulness results hint that a quantized model's explanations may stay faithful to the quantized model's own decisions even when they diverge from fp32 explanations; a direct test would be whether quantized explanations predict the quantized model's confidence better than fp32 explanations do.
  • A practical extension would be to make explanation stability a quantizer objective, per layer or per bit-width, rather than a post-hoc validation metric.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports an empirical study of how static post-training quantization (PTQ) at INT8 and INT4 precision affects the explanations produced by five CNN architectures (VGG19, ResNet18, EfficientNet-B0, DenseNet161, MobileNetV2) on two binary image classification datasets. The authors use Grad-CAM and LIME, compare quantized-model heatmaps against full-precision heatmaps via PCC, SSIM, and top-20% IoU, and supplement the similarity analysis with deletion/insertion faithfulness and paired statistical tests with effect sizes. The central claim is that classification accuracy is not a reliable indicator of interpretability stability under reduced precision, illustrated by the contrast between DenseNet161, whose LIME explanations remain highly correlated with the fp32 model, and EfficientNet-B0, whose LIME PCC drops to 0.228 at INT8 and 0.109 at INT4.

Significance. If the central empirical claim holds, the paper provides a useful and timely caution for deploying quantized models in interpretability-sensitive applications, and it makes a concrete architectural recommendation (DenseNet161 over EfficientNet-B0 when explanation stability matters). The study's strengths include the systematic coverage of five architectures, the use of two complementary explainability methods, the combination of similarity and faithfulness metrics, and the use of paired statistical tests with effect sizes rather than raw means only. The main limitation is that the headline contrast depends on the assumption that LIME's stochasticity is negligible and that the fp32 explanations are a stable reference, and the paper does not currently provide enough experimental detail to validate that assumption.

major comments (4)
  1. [Abstract and Section 3.1 ("Misclassifications after quantization")] The paper repeatedly claims that classification accuracy is preserved or only marginally reduced (e.g., "stable accuracy under quantization"), yet no classification accuracy numbers are reported anywhere for fp32, INT8, or INT4 for any model or dataset. Without this table, the headline claim that accuracy is not a reliable indicator of interpretability stability cannot be evaluated, and the size of the disagreement subset in the misclassification analysis is also unknown. Please report per-model, per-precision, per-dataset accuracy (and ideally the number of disagreement images) in a dedicated table.
  2. [Section 2.4 and Table 1 (LIME rows)] LIME is a stochastic, perturbation-based explainer, but the manuscript reports no seed, no number of perturbation samples, no segmentation parameters, no kernel width, and no repeated-run self-consistency measurement. The means and standard deviations in Table 1 are across images, not across repeated LIME runs. The EfficientNet-B0 LIME PCC values of 0.228 (INT8) and 0.109 (INT4) could therefore reflect LIME run-to-run instability rather than a quantization-induced change in model reasoning. To support the central contrast, the authors should report LIME's self-consistency (e.g., repeated-run PCC on the same fp32 model) and fix or randomize seeds in a documented way.
  3. [Section 2.5 and Table 3 caption] The faithfulness analysis measures the model's softmax confidence for the fp32-predicted class, not for each model's own predicted class. This makes the comparison asymmetric: a quantized model that changes its prediction may show artificially low deletion AUC or high insertion AUC simply because the tracked class is no longer the model's preferred output. Moreover, the caption's choice of reference class means Table 3 does not directly establish that the quantized model's explanations are faithful to its own decisions. Please either recompute faithfulness with each model's own predicted class or justify why tracking the fp32 class is the appropriate quantity for the claim.
  4. [Sections 2.2 and 2.3] The experimental setup is not reproducible from the text: the fine-tuning procedure (epochs, learning rate, optimizer, data split, augmentation) is unspecified, and the GPFQ quantization details (calibration set size, number of calibration batches, observer/scale settings, and whether activations are quantized) are not given. Since the paper's conclusions are empirical and architecture-specific, these choices could materially affect the reported similarity scores. Please provide full training and quantization configuration details, or release code and scripts.
minor comments (5)
  1. [Section 2.3] There are typos such as "targetted weigt-only quantization" (should be "targeted weight-only quantization").
  2. [Section 3.1] The phrase "Tables 1 report" is ungrammatical; it should be "Table 1 reports." Also, model names are inconsistent (e.g., "DenseNet-161" vs "DenseNet161", "MobileNet-V2" vs "MobileNetV2").
  3. [Section 3.1] The statement that EfficientNet-B0's correspondence with full precision is "under 23%" appears to conflate a PCC value of 0.228 with a percentage; it would be clearer to report the coefficient directly and reserve percentages for accuracy or IoU-like measures.
  4. [Section 3.1 and Table 2] Table 2 reports only effect sizes and a blanket "p < 0.001" claim. Please report the actual p-values (or a compact significance notation) and, because many paired tests are performed, clarify whether any multiple-comparison correction was applied.
  5. [Figure 3] The qualitative heatmaps would be easier to interpret if they included a color scale or explicit normalization details; currently it is difficult to compare spatial focus across models and precisions by eye.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical measurement study; all comparisons are explicit and no result is defined in terms of its own target.

full rationale

The paper's claimed contribution is an empirical evaluation of how post-training quantization affects Grad-CAM and LIME explanations. The implementation chain is explicit and self-contained: pre-trained fp32 models are fine-tuned, quantized with GPFQ to INT8/INT4, explanations are generated with Grad-CAM and LIME, and the quantized explanations are compared with the fp32 explanations using PCC, SSIM, and top-20% IoU, supplemented by deletion/insertion faithfulness analysis. None of these quantities is fitted to, or defined in terms of, the paper's headline conclusion that accuracy preservation does not imply interpretability preservation. The fp32 model is used as a reference baseline throughout, but this is an openly stated comparison choice rather than a circular reduction; no equation or claim presupposes the result it is used to establish. The paper does not invoke a self-citation chain or a uniqueness theorem to force its architectural ranking, and the DenseNet161-versus-EfficientNet-B0 contrast is an observed measurement outcome, not an artifact of construction. Potential concerns about LIME's stochasticity or about whether fp32 explanations are the correct ground truth are external-validity or correctness risks, not circularity, because the paper's quantitative claims remain well-defined measurements even if those concerns were valid. No circular step can be quoted or exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on domain assumptions about explanation methods and evaluation metrics, plus several hand-chosen evaluation constants. No new physical or mathematical entities are introduced, and no numbers are fitted to data.

free parameters (4)
  • Top-20% IoU threshold K = 20
    Hand-chosen mask threshold for the spatial overlap metric; IoU values change with K, although architecture ordering appears stable.
  • Deletion/insertion block size = 16x16 over 50 steps
    Hand-chosen perturbation protocol for faithfulness curves; different block sizes would change AUC values.
  • Gaussian blur sigma for insertion baseline = 15
    Hand-chosen baseline smoothness in the insertion faithfulness test.
  • LIME perturbation configuration = not reported
    Number of perturbations, kernel width, and segmentation parameters for LIME are not reported, so the exact attribution maps are underdetermined.
assumptions (4)
  • domain assumption Grad-CAM and LIME explanations are valid proxies for the model's internal reasoning.
    The paper infers changes in 'internal reasoning' from changes in explanation heatmaps (Section 2.4).
  • domain assumption The full-precision model's explanations serve as the reference ground truth for interpretability preservation.
    All similarity metrics compare quantized explanations to fp32 explanations (Section 2.5, Tables 1 and 2).
  • domain assumption Explanation similarity scores (PCC, SSIM, top-20% IoU) capture meaningful interpretability preservation.
    The paper treats these image-similarity metrics as interpretability stability metrics without validating them against human judgment or downstream tasks (Section 2.5).
  • domain assumption Softmax confidence in the fp32-predicted class is an appropriate basis for deletion/insertion faithfulness of both models.
    Table 3 caption: 'We record the model's softmax confidence for the fp32-predicted class along each curve'; this assumes the quantized model's decision aligns with the fp32 label.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations." pith.science (2026). https://pith.science/paper/SXTCHZXF

@misc{pith2026260722872,
  author       = {Pith},
  title        = {Pith review of: Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SXTCHZXF}},
  note         = {Machine review of arXiv:2607.22872}
}
read the original abstract

Post-training quantization (PTQ) has become a practical solution for deploying deep learning models on resource-constrained edge devices by compressing high-precision floating-point weights into low-precision representations without requiring retraining. Past research has demonstrated that quantization largely preserves classification accuracy; however, whether it also preserves the model's internal reasoning remains an open question. This study presents a systematic evaluation on how static PTQ affects the interpretability / explainability of five widely used CNN architectures: VGG19, ResNet18, EfficientNet-B0, DenseNet161, and MobileNetV2 at INT8 and INT4 precision. We employ a dual interpretability framework that combines Grad-CAM for spatial attention analysis with LIME for input-level feature attribution, and systematically compare full-precision and quantized models on two binary classification datasets. Interpretability is evaluated using three complementary metrics: the Pearson correlation coefficient, structural similarity index, and top-20% IoU to capture distributional and structural variations in model explanations, supplemented by deletion/insertion faithfulness analysis. The results show that classification accuracy is not a reliable indicator of interpretability stability under reduced precision. DenseNet161 maintains strong feature consistency across both precision levels, whereas EfficientNet-B0, despite achieving competitive spatial attention and classification accuracy at INT8 precision, exhibits a substantial degradation in input-level feature attribution. These findings have direct implications for the trustworthy deployment of quantized models in applications with high interpretability requirements, demonstrating that architecture selection is as important as the quantization strategy.

Figures

Figures reproduced from arXiv: 2607.22872 by the authors.

Figure 1
Figure 1. Post-training static quantization workflow. Pre-trained models (fp32) were fine-tuned on selected datasets and statically quantized to INT8/INT4. This reduces memory consumption and computation while maintaining accuracy, enabling efficient, retraining-free deployment. classes, namely, pizza and steak. The second dataset we have used is a subset of Poribohon-BD (Tabassum et al., 2020), containing 1,000 images of two… view at source ↗
Figure 2
Figure 2. Interpretability analysis pipeline for quantized and full-precision models. Input images are analyzed by both LIME and Grad-CAM. Heatmaps from both processess are compared using different evaluation metrics to assess how quantization affects attention and decision-making. 3. Results and Discussion Our empirical analysis reveals clear patterns in how different CNN architectures retain interpretable features under Pos… view at source ↗
Figure 3
Figure 3. Comparison of Grad-CAM and LIME explanation heatmaps for multiple CNN architectures under different precision levels (float32, INT8, INT4). Rows represent models, and columns correspond to precision levels with associated confidence scores. Overall, the visual evidence reinforces the paper’s quantitative findings: most models tolerate quantization reasonably well at the spatial attention level, but EfficientNet-B0 s… view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 10 canonical work pages

  1. [1]

    Qgen:Ontheability to generalize in quantization aware training

    AskariHemmat,M.,Jeddi,A.,Hemmat,R.A.,Lazarevich,I.,Hoffman,A.,Sah,S.,Saboori,E.,Savaria,Y.,David,J.P.,2024. Qgen:Ontheability to generalize in quantization aware training. arXiv preprint arXiv:2404.11769 . Bany Muhammad, M., Yeasin, M.,

    arXiv 2024

  2. [8]

    Q-HyViT: Post-Training Quantization of Hybrid Vision Transformers with Bridge Block Reconstruction for IoT Systems

    Q-hyvit: Post-training quantization of hybrid vision transformers with bridge block reconstruction for iot systems. arXiv preprint arXiv:2303.12557 . Li, Z., Xiao, J., Yang, L., Gu, Q.,

    work page Pith review arXiv

  3. [9]

    ICCV 2023 (Workshop / Proceedings)

    Repq-vit: Scale reparameterization for post-training quantization of vision transformers. ICCV 2023 (Workshop / Proceedings) . Nagel, M., Fournarakis, M., Amjad, R.A., Bondarenko, Y., Van Baalen, M., Blankevoort, T.,

    work page 2023

  4. [14]

    arXiv preprint arXiv:1409.1556

    Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 . Tabassum,S.,Ullah,S.,Al-Nur,N.H.,Shatabda,S.,2020. Poribohon-bd:Bangladeshilocalvehicleimagedatasetwithannotationforclassification. Data in brief 33, 106465. Tan, M., Le, Q.,

    arXiv 2020

  5. [1992]

    IEEE Transactions on Neural Networks 3, 334–338

    Analysis of the effects of quantization in multilayer neural networks using a statistical model. IEEE Transactions on Neural Networks 3, 334–338. Zhang,J.,Zhou,Y.,Saab,R.,2023. Post-trainingquantizationforneuralnetworkswithprovableguarantees. SIAMjournalonmathematicsofdata science 5, 373–399. Zhong, Y., Hu, J., Huang, Y., Zhang, Y., Ji, R.,

    work page 2023

  6. [1995]

    IEEE Transactions on Neural Networks 6, 1446–1451

    The effects of quantization on multilayer neural networks. IEEE Transactions on Neural Networks 6, 1446–1451. Gholami,A.,Kim,S.,Dong,Z.,Yao,Z.,Mahoney,M.W.,Keutzer,K.,2022. Asurveyofquantizationmethodsforefficientneuralnetworkinference, in: Low-power computer vision. Chapman and Hall/CRC, pp. 291–326. Gordon, O., Cohen, E., Habi, H.V., Netzer, A.,

    work page 2022

  7. [2009]

    Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee. pp. 248–255. Dundar, G., Rose, K.,

    work page 2009

  8. [2014]

    Chen,Y.,Zheng,B.,Zhang,Z.,Wang,Q.,Shen,C.,Zhang,Q.,2020

    Food-101 – mining discriminative components with random forests, in: European Conference on Computer Vision. Chen,Y.,Zheng,B.,Zhang,Z.,Wang,Q.,Shen,C.,Zhang,Q.,2020. Deeplearningonmobileandembeddeddevices:State-of-the-art,challenges, and future directions. ACM Computing Surveys (CSUR) 53, 1–37. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.,

    work page 2020

Show all 15 references
  1. [2016]

    why should i trust you?

    " why should i trust you?" explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135–1144. Rogers,H.,DeLaIglesia,B.,Zebin,T.,2023. Evaluatingtheuseofinterpretablequantizedconvo...

    2023

  2. [2018]

    arXiv preprint arXiv:1806.08342

    Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 . Krizhevsky,A.,Sutskever,I.,Hinton,G.E.,2012. Imagenetclassificationwithdeepconvolutionalneuralnetworks. Advancesinneuralinformation processing systems

    2012 arXiv

  3. [2021]

    arXiv preprint arXiv:2106.08295

    A white paper on neural network quantization. arXiv preprint arXiv:2106.08295 . Pearson, K.,

    arXiv

  4. [2023]

    arXiv preprint arXiv:2309.11531

    Eptq: Enhanced post-training quantization via hessian-guided network-wise optimization. arXiv preprint arXiv:2309.11531 . He,K.,Zhang,X.,Ren,S.,Sun,J.,2016. Deepresiduallearningforimagerecognition,in:ProceedingsoftheIEEEconferenceoncomputervision and pattern recognition, pp. 7...

    2016 arXiv

  5. [2024]

    Quantization effects on neural networks perception: How would quantization change the perceptual field of vision models?, in: 2024 IEEE Thirteenth International Conference on Image Processing Theory, Tools and Applications (IPTA), IEEE. pp. 01–06. Kim, N.J., Lee, J., Kim, H.,

    2024

  6. [2025]

    ArXiv preprint arXiv:2506.11093

    Efficientquant: An efficient post-training quantization for cnn-transformer hybrid models on edge devices. ArXiv preprint arXiv:2506.11093. Page 11 of 12 Sandler,M.,Howard,A.,Zhu,M.,Zhmoginov,A.,Chen,L.C.,2018. Mobilenetv2:Invertedresidualsandlinearbottlenecks,in:Proceedingsof...

    2018 arXiv

  7. [2026]

    Rezatofighi,H.,Tsoi,N.,Gwak,J.,Sadeghian,A.,Reid,I.,Savarese,S.,2019

    PyTorch Official Documentation, in: Online; accessed 2026-07-20. Rezatofighi,H.,Tsoi,N.,Gwak,J.,Sadeghian,A.,Reid,I.,Savarese,S.,2019. Generalizedintersectionoverunion:Ametricandalossforbounding box regression, in: Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2026

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.