REVIEW 4 major objections 5 minor 33 references
Explainable Image Classification with Reduced Overconfidence for Tissue Characterisation
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Repeated stochastic forward passes with dropout sharpen pixel-attribution maps and add a per-pixel risk score for tissue-characterisation images.
desk verdict A cheap, sensible wrapper that puts MC Dropout inside pixel attribution and adds a CV risk map; the idea is useful but the evaluation needs an accuracy check, an ablation, and table cleanup before the headline claims hold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a PA volume: T stochastic forward passes of the classification model with dropout enabled, each converted into a pixel-attribution map by the same PA method and stacked along a third dimension. From this volume the method derives two outputs: the enhanced PA map given by the per-pixel expectation over the T maps, and a risk map given by the coefficient of variation $\mathrm{CV}_{i,j} = \operatorname{std}(S_{i,j})/\operatorname{E}(S_{i,j})$. The coefficient of variation is the load-bearing choice: because pixels correspond to different semantic features with different attribution scales, raw variance would be misleading, while dividing by the pixel's mean makes the risk scores comparable. The mechanism converts dropout's cheap, repeated model averaging into both denoising (the expectation) and disagreement measurement (the CV).
What would settle it
Measure top-1 classification accuracy and calibration on the held-out pCLE and ImageNet test sets under the proposed protocol—dropout enabled, T=10 forward passes. If accuracy falls materially below the deterministic model's accuracy, then the higher ADCC scores would be describing a worse classifier, and the claim that the enhanced heatmap is a better explanation for clinical use would not stand.
Extended reading notes
Core claim
The paper's central claim is that a single deterministic pixel-attribution map inherits the overconfidence of the neural network it explains, and that this can be corrected by treating attribution as a stochastic measurement. With dropout enabled at test time, the classifier is run T times (T=10 in the experiments) and any pixel-attribution method—Grad-CAM, Grad-CAM++, SmoothGrad-CAM++, Score-CAM, or Recipro-CAM—is recomputed on each run, producing a volume of maps. The enhanced map is the pixel-wise expectation $E(S_{i,j}) = \frac{1}{T}\sum_{t=1}^{T} f_s(\hat{Y}_t)_{i,j}$, and the risk of that map is the per-pixel coefficient of variation $\mathrm{CV}_{i,j}=\sqrt{\mathrm{Var}(S_{i,j})}/E(S_{i,j})$, chosen because it allows comparison across pixels with different scales. On pCLE data and ImageNet this raises ADCC relative to the deterministic versions of the same methods and ties Score-CAM, while adding a risk surface that concentrates low risk where attribution is high. The paper concludes that this two-channel output is a more trustworthy explainability tool for intraoperative tissue characterisation.
Load-bearing premise
The load-bearing premise is that the network still classifies accurately, and that its prediction scores remain meaningful, when dropout is switched on at test time; the paper reports explanation metrics but never reports classification accuracy in that mode.
Editorial extensions
If this is right
- When surgeons see a bright but high-CV region, the method tells them not to treat that highlight as evidence, even though a single deterministic map would present it confidently.
- Because the procedure is agnostic to the base PA method, any existing heatmap technique can be upgraded to the stochastic-averaging version without retraining the classifier.
- The T-fold increase in latency, roughly ten attribution computations per image, is the main deployment cost, and the paper selects T=10 as the trade-off point between ADCC gain and speed.
- The ImageNet results suggest the mechanism transfers beyond medical microscopy, so the same heatmap-plus-risk output should apply to other image classification settings.
Reading between the lines
- A natural extension, not tested in the paper, would threshold the CV map and automatically grey out high-risk pixels, turning the risk estimate from a passive visualisation into an active safeguard against acting on untrustworthy highlights.
- We infer the same PA volume could double as an out-of-distribution detector: dropout disagreement tends to rise on inputs unlike the training data, so unusually high global CV could flag images the classifier was never competent to explain.
- A direct test of the trustworthiness claim would compare low-CV pixels against regions a clinician independently marks as diagnostically relevant, or against known spurious-correlation artefacts such as the crack visible in the meningioma videos shown in the paper's supplement.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a stochastic pixel-attribution framework: at test time a CNN trained with Dropout is run T times with Dropout enabled, a PA method computes a map per pass, and the T maps are averaged pixel-wise to form an enhanced PA map; the coefficient of variation across the T maps is presented as a pixel-wise risk estimate. The method is evaluated with Grad-CAM, Grad-CAM++, SmoothGrad-CAM++, Score-CAM, and Recipro-CAM on a pCLE brain-tumour dataset (ResNet18, MobileNetV2) and on ImageNet (ResNet50), using ADCC and latency. The authors claim that the framework improves on all tested SOTA PA methods' ADCC and that the CV map provides trustworthy risk information for intraoperative tissue characterisation.
Significance. The idea is simple, model-agnostic, and PA-method-agnostic, and the CV-based risk map is a genuinely useful addition for clinical explainability: it lets a surgeon see not only which pixels are salient but also how stable that saliency is across stochastic passes. The evaluation spans both medical and natural-image domains, and the qualitative figures support the plausibility of the risk visualisation. If the missing controls and consistency issues are resolved, this would be a solid contribution to intraoperative explainability. In its current form, however, the central quantitative claim is not yet established because the comparison changes two variables at once and the stochastic model's classification competence is never reported.
major comments (4)
- [§3.0.2, §3.1.1, Table 1] The central comparison changes two variables at once: test-time Dropout is enabled and T=10 PA maps are averaged, while the baselines use neither. No experiment isolates the averaging effect from the Dropout effect, and no classification accuracy, F1, or calibration is reported for the MC Dropout regime. If enabling Dropout degrades the classifier, then the improved ADCC values and the qualitative PA maps are produced by a less competent model, and the 'reduced overconfidence' claim is not supported. Please add an ablation with (i) T=1 with Dropout, (ii) T=10 without Dropout, and (iii) the original deterministic model, and report accuracy/calibration for all regimes.
- [§3.1, Table 5] T is selected on the same ADCC metric and the same benchmark used for the final evaluation, and the reported ADCC differences are often small (e.g., ResNet18 Grad-CAM 76.6 to 77.7; MobileNetV2 Grad-CAM 48.0 vs 47.9 at nearby T values in Table 5). No error bars, standard deviations, or patient-level repeated evaluations are provided. This selection procedure can inflate the claimed gain. Please use a held-out validation split to choose T and report variability over runs or patients.
- [Table 1 vs Supplementary Table 3; Table 2 vs Supplementary Table 4] There are internal inconsistencies that must be resolved. Recipro-CAM's original ADCC for ResNet18 is 66.5 in Table 1 but 72.8 in Supplementary Table 3; MobileNetV2 Recipro-CAM is 33.3 vs 35.8; and original Grad-CAM latency for ResNet18 is 75.6 ms in Table 1 but 5.6 ms in Supplementary Table 3. In addition, the 'Proposed' Score-CAM rows are numerically identical to the 'Original' rows on all three datasets, even though the latency scales roughly by a factor of 10. Please state whether ADCC is computed with averaged logits or with a single forward pass, explain the Score-CAM result, and correct the inconsistent values.
- [§3.0.3, Eq. (4)] The ADCC definition in Eq. (4) is written as the reciprocal of a sum of reciprocals, which omits the factor 3 required for a harmonic mean of three components as defined in the cited work [26]. Moreover, Eq. (6) defines Complexity as 100 times the L1 norm, and Eq. (4) then uses 1 - Complexity in a denominator, which is only meaningful if Complexity is normalised to [0,1]. Please verify the formula against [26] and clarify the normalisation so that the reported ADCC values are comparable with the literature.
minor comments (5)
- [§2] The text says 'Our proposed method improves ADDC' but the metric is ADCC; please fix the typo.
- [§2, Eq. (1)] The variance formula in Eq. (1) uses a transpose notation on what appear to be scalar pixel attribution values; please clarify whether this is a vectorised expression over pixels or a typographical error.
- [§3.0.2] The implementation paragraph first gives separate learning rates for ResNet18 (0.001) and MobileNetV2 (0.01), then says 'Both were trained using Adam-W ... with a learning rate of 0.001'. Please disambiguate which optimiser/learning-rate combination was used for each model.
- [Throughout] There are several typos and inconsistencies: 'MoblieNetV2' (Supplementary Table 3), 'spurrious', 'adverserial', 'pixe-wise', 'classfication', 'Dropout it not enabled', and 'assitive DL model'. A careful proofread is needed.
- [Abstract and Introduction] The claim of being 'the first approach which incorporates risk estimation into a pixel attribution method' is strong; earlier uncertainty-aware saliency works should be discussed, or the claim should be tempered to 'first in the context of intraoperative tissue characterisation'.
Circularity Check
T is selected on the same ADCC benchmark used to claim improvement, so the headline ADCC gain is partly a selected value rather than an independent prediction; the PA-averaging construction itself is not circular.
-
fitted input called prediction
[Section 3.1 'T Study' and Section 3.1.1 'Performance Evaluation'; Supplementary Table 5]
"A parameter search was performed to find the optimal value of T. As show in Supplementary C there is a positive correlation between ADCC and the value of T. With increase of T, there is an implicit increase in latency. We found the optimal tradeoff of ADCC against latency to be T = 10. ... In Table 1, we show that our method outperforms all the compared CNN-PA method combinations on ADCC apart from Score-CAM."
The free parameter T is chosen by searching ADCC on the same pCLE benchmark that is later used to report the method's ADCC improvement. Supplementary Table 5 lists ADCC for many T values on this benchmark, and T=10 is selected as the best ADCC-versus-latency tradeoff. The Section 3.1.1 'outperforms' claim is therefore not an out-of-sample prediction of the method; it is the outcome of an ADCC-directed selection. This is the fitted-input-called-prediction pattern: the reported improvement is in-sample with respect to the selection criterion, so it is statistically forced by the tuning procedure rather than independently demonstrated. It does not make the expectation estimator in Eq. (1) or the CV map in Eq. (2) definitionally circular, but it weakens the headline benchmark claim.
full rationale
The core construction is not self-definitional: the enhanced PA map is the empirical expectation of T stochastic PA maps (Eq. 1), and the risk map is the coefficient of variation of those values (Eq. 2). Neither quantity is defined in terms of ADCC, and the averaging could in principle have hurt ADCC; the paper reports that it helped. No load-bearing self-citation chain or imported uniqueness theorem is used, and the MC Dropout assumption is taken from external literature rather than from the authors' own prior results. The main circularity concern is the T-study: T is selected by maximizing ADCC on the same pCLE evaluation benchmark, and the reported 'improved ADCC' is then presented as the method's performance. This is a real but partial circularity because only the hyperparameter, not the PA map itself, was fitted to the target metric. Additional weaknesses, such as the internal inconsistency in Recipro-CAM's original ADCC (66.5 in Table 1 versus 72.8 in Supplementary Table 3) and the identical Score-CAM proposed and original results, are not themselves circularity but reduce confidence in the evaluation. Overall, the derivation is mostly self-contained; the score reflects the in-sample selection of T for the central ADCC claim.
Assumptions & free parameters
free parameters (2)
- T (number of stochastic forward passes) =
10
- Dropout probability p =
0.2
assumptions (3)
- domain assumption MC Dropout at test time provides useful stochastic perturbations for estimating attribution risk even though its Bayesian validity has been questioned.
- domain assumption Pixel attribution values are ratio-scaled and non-negative, making the coefficient of variation a meaningful per-pixel risk statistic.
- ad hoc to paper ADCC is a sufficient proxy for explainability quality and overconfidence reduction.
Cite this review
Pith. "Pith review of Explainable Image Classification with Reduced Overconfidence for Tissue Characterisation." pith.science (2026). https://pith.science/paper/5TBKNQ5I
@misc{pith2026250723709,
author = {Pith},
title = {Pith review of: Explainable Image Classification with Reduced Overconfidence for Tissue Characterisation},
year = {2026},
howpublished = {\url{https://pith.science/paper/5TBKNQ5I}},
note = {Machine review of arXiv:2507.23709}
}
read the original abstract
The deployment of Machine Learning models intraoperatively for tissue characterisation can assist decision making and guide safe tumour resections. For image classification models, pixel attribution methods are popular to infer explainability. However, overconfidence in deep learning model's predictions translates to overconfidence in pixel attribution. In this paper, we propose the first approach which incorporates risk estimation into a pixel attribution method for improved image classification explainability. The proposed method iteratively applies a classification model with a pixel attribution method to create a volume of PA maps. This volume is used for the first time, to generate a pixel-wise distribution of PA values. We introduce a method to generate an enhanced PA map by estimating the expectation values of the pixel-wise distributions. In addition, the coefficient of variation (CV) is used to estimate pixel-wise risk of this enhanced PA map. Hence, the proposed method not only provides an improved PA map but also produces an estimation of risk on the output PA values. Performance evaluation on probe-based Confocal Laser Endomicroscopy (pCLE) data and ImageNet verifies that our improved explainability method outperforms the state-of-the-art.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[26]
Samuele Poppi, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Exper- imental Analysis. 4 2021
work page 2021
-
[1]
Sanity Checks for Saliency Maps
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim, and Google Brain. Sanity Checks for Saliency Maps. 10
-
[2]
Julia Amann, Alessandro Blasimme, Effy Vayena, Dietmar Frey, and Vince I. Madai. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC, 20(1), 12 2020
work page 2020
-
[3]
Towards better un- derstanding of gradient-based attribution methods for Deep Neural Networks
Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better un- derstanding of gradient-based attribution methods for Deep Neural Networks. 11 2017
work page 2017
-
[4]
Weight Uncertainty in Neural Networks
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight Uncertainty in Neural Networks. 5 2015
work page 2015
-
[5]
Recipro-CAM:Gradient-freereciprocalclassactivation map
Seok-YongByunandWonjuLee. Recipro-CAM:Gradient-freereciprocalclassactivation map. 9 2022
work page 2022
-
[6]
Grad-CAM++: Improved Visual Explanations for Deep Convolutional Net- works
Aditya Chattopadhyay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubra- manian. Grad-CAM++: Improved Visual Explanations for Deep Convolutional Net- works. 10 2017
work page 2017
-
[7]
Andreas C. Damianou and Neil D. Lawrence. Deep Gaussian Processes. 11 2012
work page 2012
Show all 33 references
-
[8]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE CVPR . IEEE, 2009
2009
-
[9]
Diprose, Nicholas Buist, Ning Hua, Quentin Thurier, George Shand, and Reece Robinson
William K. Diprose, Nicholas Buist, Ning Hua, Quentin Thurier, George Shand, and Reece Robinson. Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator.Journal of the American Medical Informatics Associ- ation, 27(4), 4 2020
2020
-
[10]
TorchCAM: class activation explorer, 2020
François-Guillaume Fernandez. TorchCAM: class activation explorer, 2020
2020
-
[11]
Martinez Manzanera, Arjun Nair, Huaqi Qiu, Julia Schnabel, and Ben Glocker
Loic Le Folgoc, Vasileios Baltatzis, Sujal Desai, Anand Devaraj, Sam Ellis, Octavio E. Martinez Manzanera, Arjun Nair, Huaqi Qiu, Julia Schnabel, and Ben Glocker. Is MC Dropout Bayesian? 10 2021
2021
-
[12]
Dropout as a Bayesian Approximation: Appendix
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Appendix. 6 2015
2015
-
[13]
Dropout as a Bayesian Approximation: Represent- ing Model Uncertainty in Deep Learning
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Represent- ing Model Uncertainty in Deep Learning. 6 2015. 11
2015
-
[14]
Goodfellow, Jonathon Shlens, and Christian Szegedy
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. 12 2014
2014
-
[15]
Practical Variational Inference for Neural Networks
Alex Graves. Practical Variational Inference for Neural Networks
-
[16]
IdentifyingSpurious Correlations and Correcting them with an Explanation-based Learning
MisginaTsigheHagos, KathleenM.Curran, andBrianMacNamee. IdentifyingSpurious Correlations and Correcting them with an Explanation-based Learning. 11 2022
2022
-
[17]
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. 12 2015
2015
-
[18]
Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature de- tectors. 7 2012
2012
-
[19]
Hinton and Drew van Camp
Geoffrey E. Hinton and Drew van Camp. Keeping neural networks simple by minimizing the description length of the weights. pages 5–13, 1993
1993
-
[20]
Network In Network
Min Lin, Qiang Chen, and Shuicheng Yan. Network In Network. 12 2013
2013
-
[21]
Decoupled Weight Decay Regularization
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. 11 2017
2017
-
[22]
Radford M. Neal. Bayesian Learning for Neural Networks. 118, 1996
1996
-
[23]
Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Con- volutional Neural Network Models
Daniel Omeiza, Skyler Speakman, Celia Cintas, and Komminist Weldermariam. Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Con- volutional Neural Network Models. 8 2019
2019
-
[24]
Py- Torch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, L...
2019
-
[25]
RISE: Randomized Input Sampling for Explanation of Black-box Models
Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: Randomized Input Sampling for Explanation of Black-box Models. 6 2018. 12
2018
-
[27]
Why Should I Trust You?Explaining the Predictions of Any Classifier
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why Should I Trust You?Explaining the Predictions of Any Classifier. 2 2016
2016
-
[28]
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. 1 2018
2018
-
[29]
Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization.Proceedings of the IEEE International Conference on Computer Vision, 2017-October:618–...
2017
-
[30]
Sutskever, J
I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the importance of initialization and momentum in deep learning.30th International Conference on Machine Learning, ICML 2013, pages 1139–1147, 01 2013
2013
-
[31]
Score-CAM: Score-Weighted Visual Explanations for Convolu- tional Neural Networks
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-CAM: Score-Weighted Visual Explanations for Convolu- tional Neural Networks. 10 2019
2019
-
[32]
Visualizing and Understanding Convolutional Net- works
Matthew D Zeiler and Rob Fergus. Visualizing and Understanding Convolutional Net- works. 11 2013
2013
-
[33]
Learn- ing Deep Features for Discriminative Localization
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learn- ing Deep Features for Discriminative Localization. 13 Supplementary Material A Table 3: Performance evaluation study on pCLE data based on the ADCC and time metrics. Coh is Coherence, Comp is ...
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.