{"id":"10bebdc5-22fd-4a82-b01e-5e10aa7627e3","arxiv_id":"2607.22872","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Post-training quantization to 8 or 4 bits preserves accuracy for many CNNs but can silently degrade the similarity of Grad-CAM and LIME explanations to the full-precision model, with DenseNet161 most stable and EfficientNet-B0 least stable.","lead":"This paper measures whether compressing a neural network to lower precision changes how it decides, not just whether it decides correctly. It finds that some architectures keep their explanations nearly unchanged while others, notably EfficientNet-B0, lose almost all agreement with the full-precision model, even when accuracy looks fine.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The EfficientNet-B0 LIME 'degradation' could be explainer noise: the paper never measures LIME run-to-run self-consistency, so fp32-vs-INT8 differences are not yet attributable to quantization.","rationale":"I agree with the reader's conditional assessment but locate the load-bearing risk more specifically than the broad 'fp32 ground truth' assumption. The paper's headline contrast relies entirely on explanation-similarity numbers produced by a stochastic explainer, yet no baseline establishes how stable LIME is on the same model. This is a correctness risk internal to the paper's measurement protocol, not a disagreement with consensus. The absence of accuracy tables is also real and should be a secondary condition: the abstract claims 'classification accuracy is not a reliable indicator' but no accuracy numbers are reported anywhere, so the 'same predictions' half of 'same predictions, different reasons' is unsupported. However, the self-consistency question is more directly threatening because, if it lands, it invalidates the main empirical evidence even after accuracy values are supplied. The proposed test is cheap and would settle the matter. Until then, conditional acceptance with this specific reproducibility requirement is appropriate; if the test fails, the EfficientNet-B0 claim should be downgraded. The critique is about a missing baseline, not about author credibility.","tokens_in":13438,"tokens_out":6847,"duration_ms":60750,"concrete_test":"Run the exact LIME recipe from Section 2.4 with, say, 10 fixed random seeds on the same fp32 and INT8 models for a fixed set of 100 images from each dataset. Compute (a) mean pairwise PCC/SSIM/top-20% IoU between two independent LIME runs on the same fp32 model (self-consistency), (b) the same for the INT8 model, and (c) the cross-precision values. If the self-consistency distributions overlap with the fp32-vs-INT8 values for EfficientNet-B0 (e.g., fp32-vs-fp32 PCC is not substantially higher than 0.228), the observed drop cannot be attributed to quantization; if self-consistency is high (e.g., above 0.9), the paper's interpretation survives. Report seeds and all LIME hyperparameters so the test is reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical contrast is EfficientNet-B0's LIME PCC falling to 0.228 (INT8) and 0.109 (INT4) against DenseNet161's 0.987/0.984 (Table 1). The paper interprets this as quantization disrupting EfficientNet's input-level reasoning. That interpretation requires the fp32-vs-INT8 difference to reflect a change in model behavior, not noise in the explainer. LIME (Section 2.4) is a stochastic perturbation-based method: it samples perturbations and fits a local surrogate, so two runs on the same model and image produce different heatmaps. The paper reports no LIME seed, no number of perturbation samples, no kernel width, no segmentation parameters, and no repeated-run baseline. The standard deviations reported in Table 1 are across-image spreads, not the explainer's run-to-run variance. The faithfulness analysis in Table 3 does not close this gap: it measures whether an explanation aligns with the model's decisions (deletion/insertion AUC), which is orthogonal to whether the explanation is reproducible. If LIME is sufficiently unstable on EfficientNet-B0, two fp32 runs could agree no better than 0.228, making the headline 'different reasons' an artifact of explainer stochasticity rather than evidence about quantized reasoning. The same issue applies to the SSIM and IoU columns. This is load-bearing because the abstract's central claim and the deployment recommendation against EfficientNet depend on this measured degradation being real.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an empirical study of how static post-training quantization (PTQ) at INT8 and INT4 precision affects the explanations produced by five CNN architectures (VGG19, ResNet18, EfficientNet-B0, DenseNet161, MobileNetV2) on two binary image classification datasets. The authors use Grad-CAM and LIME, compare quantized-model heatmaps against full-precision heatmaps via PCC, SSIM, and top-20% IoU, and supplement the similarity analysis with deletion/insertion faithfulness and paired statistical tests with effect sizes. The central claim is that classification accuracy is not a reliable indicator of interpretability stability under reduced precision, illustrated by the contrast between DenseNet161, whose LIME explanations remain highly correlated with the fp32 model, and EfficientNet-B0, whose LIME PCC drops to 0.228 at INT8 and 0.109 at INT4.","tokens_in":13717,"tokens_out":3685,"duration_ms":35437,"significance":"If the central empirical claim holds, the paper provides a useful and timely caution for deploying quantized models in interpretability-sensitive applications, and it makes a concrete architectural recommendation (DenseNet161 over EfficientNet-B0 when explanation stability matters). The study's strengths include the systematic coverage of five architectures, the use of two complementary explainability methods, the combination of similarity and faithfulness metrics, and the use of paired statistical tests with effect sizes rather than raw means only. The main limitation is that the headline contrast depends on the assumption that LIME's stochasticity is negligible and that the fp32 explanations are a stable reference, and the paper does not currently provide enough experimental detail to validate that assumption.","major_comments":[{"comment":"The paper repeatedly claims that classification accuracy is preserved or only marginally reduced (e.g., \"stable accuracy under quantization\"), yet no classification accuracy numbers are reported anywhere for fp32, INT8, or INT4 for any model or dataset. Without this table, the headline claim that accuracy is not a reliable indicator of interpretability stability cannot be evaluated, and the size of the disagreement subset in the misclassification analysis is also unknown. Please report per-model, per-precision, per-dataset accuracy (and ideally the number of disagreement images) in a dedicated table.","section":"Abstract and Section 3.1 (\"Misclassifications after quantization\")"},{"comment":"LIME is a stochastic, perturbation-based explainer, but the manuscript reports no seed, no number of perturbation samples, no segmentation parameters, no kernel width, and no repeated-run self-consistency measurement. The means and standard deviations in Table 1 are across images, not across repeated LIME runs. The EfficientNet-B0 LIME PCC values of 0.228 (INT8) and 0.109 (INT4) could therefore reflect LIME run-to-run instability rather than a quantization-induced change in model reasoning. To support the central contrast, the authors should report LIME's self-consistency (e.g., repeated-run PCC on the same fp32 model) and fix or randomize seeds in a documented way.","section":"Section 2.4 and Table 1 (LIME rows)"},{"comment":"The faithfulness analysis measures the model's softmax confidence for the fp32-predicted class, not for each model's own predicted class. This makes the comparison asymmetric: a quantized model that changes its prediction may show artificially low deletion AUC or high insertion AUC simply because the tracked class is no longer the model's preferred output. Moreover, the caption's choice of reference class means Table 3 does not directly establish that the quantized model's explanations are faithful to its own decisions. Please either recompute faithfulness with each model's own predicted class or justify why tracking the fp32 class is the appropriate quantity for the claim.","section":"Section 2.5 and Table 3 caption"},{"comment":"The experimental setup is not reproducible from the text: the fine-tuning procedure (epochs, learning rate, optimizer, data split, augmentation) is unspecified, and the GPFQ quantization details (calibration set size, number of calibration batches, observer/scale settings, and whether activations are quantized) are not given. Since the paper's conclusions are empirical and architecture-specific, these choices could materially affect the reported similarity scores. Please provide full training and quantization configuration details, or release code and scripts.","section":"Sections 2.2 and 2.3"}],"minor_comments":[{"comment":"There are typos such as \"targetted weigt-only quantization\" (should be \"targeted weight-only quantization\").","section":"Section 2.3"},{"comment":"The phrase \"Tables 1 report\" is ungrammatical; it should be \"Table 1 reports.\" Also, model names are inconsistent (e.g., \"DenseNet-161\" vs \"DenseNet161\", \"MobileNet-V2\" vs \"MobileNetV2\").","section":"Section 3.1"},{"comment":"The statement that EfficientNet-B0's correspondence with full precision is \"under 23%\" appears to conflate a PCC value of 0.228 with a percentage; it would be clearer to report the coefficient directly and reserve percentages for accuracy or IoU-like measures.","section":"Section 3.1"},{"comment":"Table 2 reports only effect sizes and a blanket \"p < 0.001\" claim. Please report the actual p-values (or a compact significance notation) and, because many paired tests are performed, clarify whether any multiple-comparison correction was applied.","section":"Section 3.1 and Table 2"},{"comment":"The qualitative heatmaps would be easier to interpret if they included a color scale or explicit normalization details; currently it is difficult to compare spatial focus across models and precisions by eye.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practically relevant question, but the absence of accuracy numbers and the lack of LIME reproducibility controls are load-bearing gaps that should be fixed before publication. If the authors can supply those, the study could become a solid empirical contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nTwo things you should know about this arXiv paper. First, it is the first systematic look at how post-training quantization changes LIME and Grad-CAM explanations across five CNN architectures, and it gives a concrete, potentially useful ranking: DenseNet161 stays nearly identical, EfficientNet-B0's LIME attributions fall apart. Second, the central claim—accuracy is not a reliable indicator of interpretability stability—is plausible but not actually demonstrated, because the paper never reports accuracy numbers. That's a fixable omission, not a fatal one.\n\nWhat is genuinely new: the dual-explainer protocol (Grad-CAM plus LIME) with PCC/SSIM/IoU plus deletion/insertion faithfulness, applied to PTQ rather than QAT. Prior work was CAM-only or QAT-only. The paired statistical analysis with Cohen's d is a step up from typical table-eye comparisons. The faithfulness check is a good addition, and the finding that EfficientNet-B0 collapses on LIME while keeping decent Grad-CAM is consistent with Kerkouri et al.'s QAT result, so there is convergent evidence.\n\nWhere it gets soft. The LIME stochasticity issue is real and not addressed. No seed, no perturbation count, no kernel width, no superpixel settings, no repeated-run baseline. On EfficientNet-B0, LIME PCC drops to 0.228 at INT8. You can't rule out that two fp32 runs on EfficientNet-B0 would agree that poorly. The across-image std in Table 1 does not capture run-to-run variance. So the headline 'different reasons' is not yet nailed down. That said, the faithfulness table shows LIME is producing meaningful explanations even for EfficientNet (insertion-deletion gap ~0.15), so the paper is not nonsense; it just needs a reproducibility appendix.\n\nMissing accuracy table is the bigger problem. The entire abstract rests on 'accuracy is preserved but explanations are not', yet no accuracy per model/precision appears anywhere. The qualitative confidences for one image don't cut it. Add a table. Also unspecified: fine-tuning details, calibration set size for GPFQ, LIME internals, and whether similarity comparisons use each model's own predicted class or the fp32-predicted class. That last point matters because the title says 'same predictions' but the tables pool all images, including disagreements. The disagreement-subset analysis exists, but the main tables should be restricted to images both models classify correctly for the title to hold.\n\nThe limitations paragraph is honest about dataset and architecture scope. Two small binary datasets and no transformers is a narrow foundation, but fine for a first measurement.\n\nBottom line: this is a worthwhile empirical contribution that deserves serious peer review, but it needs one round of heavy revision: report accuracy, pin down LIME, and clarify the class used in comparisons. I would not cite it in its current form. If it gets code and accuracy tables, it becomes a reasonable reference for deployment decisions.","headline":"Useful empirical first pass on how PTQ changes CNN explanations, but the central accuracy-vs-interpretability claim is unsupported without an accuracy table and LIME reproducibility details.","tokens_in":14282,"tokens_out":2914,"would_cite":false,"duration_ms":26392,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Models can keep their accuracy after quantization while changing the reasons for their decisions.","keywords":["Post-training quantization","Model interpretability","Explainable AI","Grad-CAM","LIME","CNN quantization","Edge deployment","Feature attribution stability"],"falsifier":"Measure LIME explanation similarity between two independently trained full-precision EfficientNet-B0 models on the same images; if that fp32-versus-fp32 PCC is already near 0.228, the low fp32-versus-INT8 score cannot be attributed to quantization.","tokens_in":13229,"feed_emoji":"🧠","tokens_out":7803,"duration_ms":63506,"temperature":0.7,"pith_summary":"Post-training quantization (PTQ) is widely used to shrink neural networks for edge devices, and past work showed it largely preserves classification accuracy. This paper establishes that accuracy preservation does not imply that the model's internal reasoning is preserved. By comparing Grad-CAM spatial attention and LIME input-level attributions between full-precision and INT8/INT4 versions of five CNN architectures, the authors find that explanation stability is strongly architecture-dependent. DenseNet161's explanations stay nearly identical to the full-precision model at both precisions, while EfficientNet-B0's LIME attributions collapse to a Pearson correlation of 0.228 at INT8 and 0.109 at INT4 even though its accuracy remains competitive. The practical stakes: choosing a deployment architecture for interpretability-sensitive applications requires validation of explanation stability, not just accuracy.","feed_headline":"Accuracy can survive quantization while model reasoning drifts","feed_subtitle":"EfficientNet-B0's LIME attributions drop to 0.11 correlation at INT4 while DenseNet161 stays near 0.98.","key_machinery":"The machinery is a paired heatmap-comparison pipeline. Each architecture is fine-tuned, quantized with the GPFQ weight-only method to INT8 and INT4, and then asked to explain the same images through two complementary lenses: Grad-CAM, which produces spatial attention maps from gradient-weighted final-layer feature maps, and LIME, which fits a local surrogate model to estimate input-level feature attribution. The full-precision and quantized explanation maps are then compared with the Pearson correlation coefficient, the structural similarity index, and top-20% intersection-over-union, with deletion/insertion AUC acting as a faithfulness check on whether the explanations track the model's actual confidence. This turns the question 'did quantization change the model's reasoning?' into a numerical distance between explanation maps.","core_discovery":"The paper's central claim is that static PTQ to INT8 and INT4 can substantially change which input evidence a model uses even when classification accuracy is stable, so accuracy is not a trustworthy proxy for interpretability stability. The supporting result is a head-to-head measurement: across Food-101 and Poribohon-BD subsets, DenseNet161 maintains LIME PCC of 0.987 at INT8 and 0.984 at INT4, while EfficientNet-B0 drops to 0.228 and 0.109 respectively; Grad-CAM stays more stable within each architecture, but perturbation-based attributions diverge sharply. The paper interprets this as evidence that architecture selection matters as much as quantization strategy for trustworthy deployment, and that quantized models that agree with full-precision predictions can still be reaching the same predictions for different reasons.","pith_inferences":["Beyond the tested CNNs, the same measurement pipeline could be applied to vision transformers; their attention mechanisms may show either stronger or weaker explanation stability under PTQ, since the paper explicitly leaves that architecture class out.","A useful control the paper does not run is fp32-versus-fp32 variability: if two independent full-precision runs of EfficientNet-B0 already disagree in LIME attributions as much as fp32-versus-INT8 do, part of the reported degradation is innate explanation noise, not quantization damage.","The faithfulness results hint that a quantized model's explanations may stay faithful to the quantized model's own decisions even when they diverge from fp32 explanations; a direct test would be whether quantized explanations predict the quantized model's confidence better than fp32 explanations do.","A practical extension would be to make explanation stability a quantizer objective, per layer or per bit-width, rather than a post-hoc validation metric."],"forward_implications":["Transparency-focused deployments should validate explanation stability per architecture after PTQ, since accuracy curves can look healthy while attribution maps drift.","DenseNet161 is the safer pick when interpretability is a hard requirement: its explanation similarity to fp32 stays above 0.96 for both bit-widths and both explainers.","EfficientNet-B0 should be kept at full precision in interpretability-critical settings; at INT8 its LIME similarity is already below 0.23.","INT8 explanations are statistically closer to fp32 than INT4 are, so teams that want the deployed model to reason like the original model should prefer INT8 despite INT4's larger compression.","When quantization does flip a prediction, the model tends to attend to different image regions, so the failure is not just a label change but an evidence shift."],"supporting_citations":[{"why":"Supplies the Grad-CAM method that produces the spatial attention heatmaps compared across precisions.","marker":"(Selvaraju et al., 2017)"},{"why":"Supplies LIME, the perturbation-based method whose coefficients form the input-level attributions that show the largest degradation.","marker":"(Ribeiro et al., 2016)"},{"why":"Defines GPFQ, the weight-only post-training quantization method used to create the INT8 and INT4 models.","marker":"(Zhang et al., 2023)"},{"why":"Earlier architecture-dependent quantization sensitivity results, including EfficientNet-B0's fragility, that this study extends to PTQ and LIME.","marker":"(Kerkouri et al., 2024)"},{"why":"Prior CAM-based comparison of quantized and full-precision CNNs that the paper extends with quantitative metrics and input-level attribution.","marker":"(Rogers et al., 2023)"},{"why":"Establishes the premise that PTQ often preserves near-full accuracy, the very premise the paper challenges on interpretability grounds.","marker":"(Krishnamoorthi, 2018)"},{"why":"Defines SSIM, used here as the structural similarity metric for explanation heatmaps.","marker":"(Wang et al., 2004)"},{"why":"Defines the Pearson correlation coefficient used to measure linear correspondence between explanation maps.","marker":"(Pearson, 1896)"}],"fun_headline_variants":["Quantization can silently switch a model's reasoning path","Accuracy isn't proof a quantized model still reasons","EfficientNet's logic degrades while accuracy holds at INT8","INT4 attribution collapse happens without accuracy loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison treats the full-precision model's explanations as the reference that quantized explanations should match, so if fp32 explanations are themselves noisy or unfaithful, a low similarity score does not prove that quantization damaged the model's reasoning.","fun_headline_variants_meta":{"raw":{"variants":["Quantization can silently switch a model's reasoning path","Accuracy isn't proof a quantized model still reasons","EfficientNet's logic degrades while accuracy holds at INT8","INT4 attribution collapse happens without accuracy loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00075,"raw_usage":{"total_tokens":3640,"prompt_tokens":1045,"completion_tokens":2595,"prompt_tokens_details":{"cached_tokens":896},"prompt_cache_hit_tokens":896,"prompt_cache_miss_tokens":149,"completion_tokens_details":{"reasoning_tokens":2531}},"tokens_in":149,"tokens_out":2595,"duration_ms":21338,"temperature":1.0,"reasoning_tokens":2531,"cache_read_input_tokens":896,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:27:42.440209+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure LIME explanation similarity between two independently trained full-precision EfficientNet-B0 models on the same images; if that fp32-versus-fp32 PCC is already near 0.228, the low fp32-versus-INT8 score cannot be attributed to quantization.","supporting_citations":[],"review_version":2}