Pith. sign in

REVIEW 2 major objections 1 minor 37 references

Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifiers

T0 review · 2 major / 1 minor · reviewed 2026-05-23 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗

Pith's one-line read Seven explainability methods applied to a skin lesion classifier reveal biases yet fall short of the comprehensiveness needed for clinical transparency.

desk verdict This paper compares seven standard XAI methods on one ISIC-trained skin lesion model and notes that they surface biases but fall short on comprehensiveness. read the letter →

arxiv 2412.03166 v1 submitted 2024-12-04 cs.CV

Rosa Y. G. Paccotacya-Yanque , Alceu Bissoto , Sandra Avila This is my paper
classification cs.CV
keywords skinlesionclassificationexplainabilitymethodsGrad-CAMLIMESHAPmodelbiasdetectionmedicalAItransparencyISICdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper compares four pixel-attribution techniques and three concept-based techniques on a deep network trained to classify skin lesions from the ISIC archive. It identifies specific desiderata that explanations must meet for medical use and tests whether the selected methods satisfy them. The analysis shows that the methods can surface model biases but leave gaps in explanatory coverage. A reader would care because high accuracy alone does not suffice when a model influences skin-cancer decisions that affect patients.

What carries the argument

Comparative evaluation of pixel-attribution methods (Grad-CAM, Score-CAM, LIME, SHAP) and high-level concept methods (ACE, ICE, CME) on a single deep neural network trained on the ISIC skin-lesion archive.

What would settle it

A follow-up study that applies the same seven methods to multiple independent skin-lesion models and finds every method meets all identified desiderata without further development would falsify the claim that room remains for improvement.

Watch

Extended reading notes

Core claim

When seven established explainability methods—Grad-CAM, Score-CAM, LIME, SHAP, ACE, ICE, and CME—are applied to the same ISIC-trained network, they surface biases in the model’s decision process, yet none supplies explanations that fully meet the desiderata required for transparency in skin-lesion classification.

Load-bearing premise

The seven chosen methods adequately represent the broader space of explainability techniques and the desiderata observed on this one model extend to other skin-lesion classifiers and clinical settings.

Editorial extensions

If this is right

  • Biases present in skin-lesion models become detectable once pixel-attribution and concept-based explanations are generated.
  • Current methods leave explanatory gaps that prevent full transparency for clinical deployment.
  • Additional work on explanation comprehensiveness is required before skin-lesion models can support reliable medical decisions.
  • The identified desiderata provide concrete targets for developing improved explanation techniques tailored to dermatology images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same comparative protocol could be run on other medical imaging tasks to test whether the same explanatory shortcomings appear outside dermatology.
  • Human-subject studies measuring whether clinicians actually change decisions after seeing these explanations would test practical utility beyond the paper’s technical analysis.
  • Hybrid methods that combine pixel and concept information might close the comprehensiveness gap the paper identifies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript presents an empirical comparison of seven explainability methods—four pixel-attribution techniques (Grad-CAM, Score-CAM, LIME, SHAP) and three concept-based methods (ACE, ICE, CME)—applied to a single deep neural network trained on the ISIC skin lesion archive. It identifies desiderata for explanations in this domain and concludes that the tested techniques can reveal biases but are insufficiently comprehensive for achieving full transparency in skin-lesion classifiers.

Significance. If the empirical observations hold, the work offers a useful case study on the practical shortcomings of existing XAI tools in a clinically relevant computer-vision task. The balanced selection of pixel- and concept-level methods is a constructive choice, and the use of a public dataset supports reproducibility. The significance remains modest, however, because the findings are tied to a single model and dataset without demonstrated transfer.

major comments (2)
  1. [Abstract] Abstract and overall experimental design: the claim that the seven methods reveal biases yet leave room for more comprehensive explanations is derived entirely from one DNN trained on ISIC; no ablation across architectures, no second dataset, and no clinical-context transfer test are reported, rendering the generalization to “skin-lesion models” in general unsupported.
  2. [Abstract] Abstract: the high-level conclusion is stated without any quantitative metrics, error rates, statistical tests, or details on how biases were identified or measured, so the data-to-claim link cannot be verified from the provided information.
minor comments (1)
  1. The title asks whether explanations are “helpful,” yet the abstract and reported findings focus on identifying desiderata rather than directly quantifying helpfulness or clinical utility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We agree that the abstract should more precisely reflect the study's scope and provide additional context on the evaluation approach. We address each major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract and overall experimental design: the claim that the seven methods reveal biases yet leave room for more comprehensive explanations is derived entirely from one DNN trained on ISIC; no ablation across architectures, no second dataset, and no clinical-context transfer test are reported, rendering the generalization to “skin-lesion models” in general unsupported.

    Authors: We agree that the study is limited to a single DNN architecture and the ISIC dataset, with no ablations or additional datasets reported. The abstract's reference to 'skin-lesion models' in general can be revised to more accurately state that the findings apply to the specific classifier and setting examined. We will update the abstract wording accordingly in the revised manuscript. revision: yes

  2. Referee: [Abstract] Abstract: the high-level conclusion is stated without any quantitative metrics, error rates, statistical tests, or details on how biases were identified or measured, so the data-to-claim link cannot be verified from the provided information.

    Authors: The conclusions are drawn from qualitative visual analysis of the generated explanations and their comparison to the desiderata identified in the paper. To strengthen the link, we will revise the abstract to include a short description of the evaluation method (e.g., 'via qualitative inspection of explanation maps and concept activations'). The work does not include quantitative bias metrics, as the focus is on comparative visualization; we can expand the methods section to detail the identification criteria used. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical comparison with no derivation or self-referential steps

full rationale

The paper conducts an empirical analysis of seven standard explainability methods (Grad-CAM, Score-CAM, LIME, SHAP, ACE, ICE, CME) applied to one DNN trained on the public ISIC archive. No equations, fitted parameters, predictions derived from inputs, uniqueness theorems, or self-citations appear in the reported setup. The central findings are observational comparisons on this single instance; they do not reduce to any input by construction. This matches the default case of a self-contained empirical study.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim rests on the domain assumption that explanations are required for clinical reliability and that the chosen methods plus the ISIC archive suffice to diagnose explanation shortcomings.

assumptions (1)
  • domain assumption Good accuracy is not enough for skin cancer predictions; understanding the model's behavior is crucial for clinical application.
    Stated directly in the abstract as motivation for the study.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifiers." pith.science (2026). https://pith.science/paper/2412.03166

@misc{pith2026241203166,
  author       = {Pith},
  title        = {Pith review of: Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2412.03166}},
  note         = {Machine review of arXiv:2412.03166}
}
read the original abstract

Deep Learning has shown outstanding results in computer vision tasks; healthcare is no exception. However, there is no straightforward way to expose the decision-making process of DL models. Good accuracy is not enough for skin cancer predictions. Understanding the model's behavior is crucial for clinical application and reliable outcomes. In this work, we identify desiderata for explanations in skin-lesion models. We analyzed seven methods, four based on pixel-attribution (Grad-CAM, Score-CAM, LIME, SHAP) and three on high-level concepts (ACE, ICE, CME), for a deep neural network trained on the International Skin Imaging Collaboration Archive. Our findings indicate that while these techniques reveal biases, there is room for improving the comprehensiveness of explanations to achieve transparency in skin-lesion models.

Figures

Figures reproduced from arXiv: 2412.03166 by the authors.

Figure 1
Figure 1. shows four examples from the pixel-attribution meth￾ods for each model (each one presents a skin-lesion image with dermoscopic attributes, and the superimposed images) according to the correctness of the prediction: true positive, true negative, false positive, and false negative. The superim￾posed images are known as local explanations, as they explain only one prediction. Globules Milia-like cyst Negative Network … view at source ↗
Figure 2
Figure 2. Concept-based explanations results. without localizing them. These three methods provide global explanations (explain a class), but only ICE provides local ones (explain the prediction for a single input). For ACE (Fig. 2d), we found it hard to interpret the obtained concepts. At first sight, the most salient concept seems to be related to the pigmented network attribute. However, this attribute is also present in o… view at source ↗
Figure 3
Figure 3. Saliency results for predictions with high confidence to [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages

  1. [1]

    https: //isic-archive.com

    International Skin Imaging Collaboration: Melanoma Project. https: //isic-archive.com

    work page

  2. [2]

    Sulthana A, V

    R. Sulthana A, V . Chamola, Z. Hussain, F. Albalwy, and A. Hussain. A novel end-to-end deep convolutional neural network based skin lesion classification framework. Expert Systems with Applications, 246:123056, 2024

    work page 2024

  3. [3]

    A. B. Arrieta, N. D ´ıaz-Rodr´ıguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins, R. Chatila, and F. Herrera. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82–115, 2020

    work page 2020

  4. [4]

    Avila and M

    S. Avila and M. Marini. The Day I Discovered I Was Collaborating on a Eugenics Project: On Imponderables in Collaborative Research. https://blog.castac.org/2023/09/the-day-i-discovered-i-was-collaborati ng-on-a-eugenics-project-on-imponderables-in-collaborative-research/, 2023

    work page 2023

  5. [5]

    Barros, L

    L. Barros, L. Chaves, and S. Avila. Assessing the generalizability of deep neural networks-based models for black skin lesions. In Iberoamerican Congress on Pattern Recognition , 2023

    work page 2023

  6. [6]

    Bissoto, C

    A. Bissoto, C. Barata, E. Valle, and S. Avila. Artifact-based domain generalization of skin lesion models. In Computer Vision–ECCV 2022 Workshops, pages 133–149, 2022

    work page 2022

  7. [7]

    Bissoto, C

    A. Bissoto, C. Barata, E. Valle, and S. Avila. Test-time selection for robust skin lesion analysis. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 37–46, 2023

    work page 2023

  8. [8]

    Bissoto, C

    A. Bissoto, C. Barata, E. Valle, and S. Avila. Even small correlation and diversity shifts pose dataset-bias issues. Pattern Recognition Letters, 179:87–93, 2024

    work page 2024

Show all 37 references
  1. [9]

    Bissoto, M

    A. Bissoto, M. Fornaciali, E. Valle, and S. Avila. (De)constructing bias on skin lesion datasets. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019

    2019

  2. [10]

    Bissoto, E

    A. Bissoto, E. Valle, and S. Avila. Debiasing skin lesion datasets and models? Not so fast. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020

    2020

  3. [11]

    D. V . Carvalho, E. M. Pereira, and J. S. Cardoso. Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8), 2019

    2019

  4. [12]

    Chaves, A

    L. Chaves, A. Bissoto, E. Valle, and S. Avila. An evaluation of self- supervised pre-training for skin-lesion analysis. In European Conference on Computer Vision , pages 150–166, 2022

    2022

  5. [13]

    Codella, V

    N. Codella, V . Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, M. Marchetti, H. Kittler, and A. Halpern. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic), 2019

    2018

  6. [14]

    Ghorbani, J

    A. Ghorbani, J. Wexler, J. Y . Zou, and B. Kim. Towards automatic concept-based explanations. In Advances in Neural Information Pro- cessing Systems, pages 9277–9286, 2019

    2019

  7. [15]

    Haggenm ¨uller, R

    S. Haggenm ¨uller, R. C. Maron, A. Hekler, J. S. Utikal, C. Barata, R. L. Barnhill, H. Beltraminelli, C. Berking, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. European Journal of Cancer, 156:202– 216, 2021

    2021

  8. [16]

    Kawahara, S

    J. Kawahara, S. Daneshvar, G. Argenziano, and G. Hamarneh. Seven- point checklist and skin lesion classification using multitask multimodal neural nets. IEEE Journal of Biomedical and Health Informatics , 23(2):538–546, 2019

    2019

  9. [17]

    Kazhdan, B

    D. Kazhdan, B. Dimanov, M. Jamnik, P. Li `o, and A. Weller. Now you see me (CME): Concept-based model extraction. In CEUR Proceedings of the CIKM Workshops , volume 2699, 2020

    2020

  10. [18]

    G. Lima, N. Grgi ´c-Hlaˇca, J. Keun Jeong, and M. Cha. The conflict between explainable and accountable decision-making algorithms. In ACM Conference on Fairness, Accountability, and Transparency , page 2103–2113, 2022

    2022

  11. [19]

    A. N. Mir, I. Nissar, D. R. Rizvi, and A. Kumar. Lesnet: An automated skin lesion deep convolutional neural network classifier through augmen- tation and transfer learning. Procedia Computer Science , 235:112–121, 2024

    2024

  12. [20]

    Mir ´o-Nicolau, A

    M. Mir ´o-Nicolau, A. Jaume i Cap ´o, and G. Moy `a-Alcover. Assessing fidelity in xai post-hoc techniques: A comparative study with ground truth explanations datasets. Artificial Intelligence, 335:104179, 2024

    2024

  13. [21]

    Moher, A

    D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, and PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4):264–269, 2009

    2009

  14. [22]

    C. Molnar. Interpretable Machine Learning. 2019. https://christophm.g ithub.io/interpretable-ml-book/

    2019

  15. [23]

    Pouyanfar, S

    S. Pouyanfar, S. Sadiq, Y . Yan, H. Tian, Y . Tao, M. P. Reyes, M.-L. Shyu, S.-C. Chen, and S. S. Iyengar. A survey on deep learning: Algorithms, techniques, and applications. ACM Comput. Surv., 51(5), 2018

    2018

  16. [24]

    M. A. R. Lara, M. V . R. Kowalczuk, M. L. Eliceche, M. G. Ferraresso, D. R. Luna, S. E. Benitez, and L. D. Mazzuoccolo. A dataset of skin lesion images collected in argentina for the evaluation of ai tools in this population. Scientific Data, 10(1):712, 2023

    2023

  17. [25]

    M. T. Ribeiro, S. Singh, and C. Guestrin. Why Should I Trust You?: Explaining the Predictions of Any Classifier. In ACM International Con- ference on Knowledge Discovery and Data Mining , page 1135–1144, 2016

    2016

  18. [26]

    Robnik- ˇSikonja and M

    M. Robnik- ˇSikonja and M. Bohanec. Perturbation-based explanations of prediction models. Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent, pages 159–175, 2018

    2018

  19. [27]

    Samek, G

    W. Samek, G. Montavon, A. Vedaldi, L. Hansen, and K. M ¨uller. Explainable AI: interpreting, explaining and visualizing deep learning , volume 11700. Springer Nature, 2019

    2019

  20. [28]

    Schwalbe and B

    G. Schwalbe and B. Finzel. A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Mining and Knowledge Discovery , pages 1–59, 2023

    2023

  21. [29]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision, pages 618–626, 2017

    2017

  22. [30]

    L. S. Shapley. A value for n-person games. Contributions to the Theory of Games, 2(28):307–317, 1953

    1953

  23. [31]

    Shinde, P

    S. Shinde, P. Tupe-Waghmare, T. Chougule, J. Saini, and M. Ingalha- likar. Predictive and discriminative localization of pathology using high resolution class activation maps with CNNs. PeerJ Computer Science , 7:1–14, 2021

    2021

  24. [32]

    Alison Noble

    Jia Sun, Tapabrata Chakraborti, and J. Alison Noble. A comparative study of explainer modules applied to automated skin lesion classifica- tion. CEUR Workshop Proceedings, 2796, 2020

    2020

  25. [33]

    Valle, M

    E. Valle, M. Fornaciali, A. Menegola, J. Tavares, F. V . Bittencourt, L. T. Li, and S. Avila. Data, depth, and design: Learning reliable models for skin lesion analysis. Neurocomputing, 2020

    2020

  26. [34]

    H. Wang, Z. Wang, M. Du, F. Yang, Z. Zhang, S. Ding, P. Mardziel, and X. Hu. Score-CAM: Score-weighted visual explanations for convo- lutional neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 24–25, 2020

    2020

  27. [35]

    Guide to cancer early diagnosis

    World Health Organization. Guide to cancer early diagnosis . World Health Organization, 2017

    2017

  28. [36]

    Yuksekgonul, M

    M. Yuksekgonul, M. Wang, and J. Zou. Post-hoc concept bottleneck models. ICLR 2022 Workshop, 2022

    2022

  29. [37]

    Zhang, P

    R. Zhang, P. Madumal, T. Miller, K. A. Ehinger, and B. I. P. Rubinstein. Invertible concept-based explanations for cnn models with non-negative concept activation vectors. In AAAI Conference on Artificial Intelligence, volume 35, pages 11682–11690, 2021

    2021

Pith tools

Reviewed May 23, 2026 · model on record in the stance chip above.