Pith. sign in

REVIEW 3 major objections 55 references

Concept-level AI explanations look human-readable but still need independent perturbation audits before they can be trusted.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A perturbation-and-surrogate audit shows MedSAM and VLM retinal concept explanations have pathway- and concept-specific reliability, not automatic trustworthiness.

T0 review reviewed 2026-07-13 challenge →

load-bearing objection Useful SMILE-style concept audit with a clear multi-metric protocol; the MedSAM-vs-VLM findings are real but rest on a small, black-mask local test that the paper itself flags as imperfect. the 3 major comments →

arxiv 2607.09649 v1 pith:AZ6Z3HJM submitted 2026-07-10 cs.AI

ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI

Mohadeseh Mollapour , Koorosh Aslansefat , Zeinab Dehghani , Bhupesh Kumar Mishra , Tejal Shah , Zhibao Mian This is my paper
classification cs.AI
keywords concept-based XAIexplanation auditingperturbation-based interpretabilitysurrogate fidelityfaithfulnessretinal fundus imagingMedSAMvision-language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Concept-based explainable AI promises to turn opaque model decisions into human-readable ideas such as lesions, vessels, or optic-disc structures. This paper argues that readability is not the same as reliability: a concept can sound clinically sensible while still being driven by shortcuts, unstable evidence, or incomplete representations. ConceptSMILE is offered as a model-agnostic audit layer that systematically perturbs image regions, tracks how concept responses shift, weights nearby samples, and fits a local XGBoost surrogate. Reliability is scored along five axes: attribution accuracy, surrogate fidelity, faithfulness, stability under artefacts, and consistency across repeated runs. On retinal fundus images, segmentation-guided visual concepts and vision-language semantic concepts show different strengths rather than a single winner, so concept explanations should be treated as claims to be tested, not as self-certifying evidence.

Core claim

Human-understandable concept explanations are not automatically trustworthy. ConceptSMILE shows that their reliability is pathway- and concept-dependent: on retinal images, MedSAM-based visual concepts deliver stronger spatial attribution and the highest local surrogate fidelity (R² = 0.8503, weighted R² = 0.8465), while the vision-language pathway is stronger on vessel faithfulness and on stability under selected non-clinical artefacts. Trustworthy concept XAI therefore requires multi-metric, concept-specific auditing rather than acceptance of plausible concept labels.

What carries the argument

ConceptSMILE: a perturbation audit that masks superpixel regions, measures concept-response shifts, applies cosine or Wasserstein locality weights, fits a local XGBoost surrogate to those shifts, and scores reliability on attribution accuracy, fidelity, faithfulness, stability, and consistency.

Load-bearing premise

The method assumes that blacking out a few superpixel patches is a fair local test of whether a concept is truly tied to the right image evidence, rather than an artificial change that models react to for non-clinical reasons.

What would settle it

On a larger multi-centre retinal set with clinician-marked concept regions, check whether ConceptSMILE’s faithfulness and fidelity rankings reverse when superpixel black-masking is replaced by anatomy-aware or inpainted counterfactuals that keep retinal appearance realistic.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes ConceptSMILE, a model-agnostic perturbation audit for concept-based XAI. It treats existing concept pathways as fixed black boxes, generates superpixel-masked local neighbourhoods, measures concept-response shifts, applies cosine/Wasserstein locality weights, and fits a local XGBoost surrogate. Reliability is scored with five metrics: attribution accuracy, surrogate fidelity, faithfulness, stability under non-clinical artefacts, and consistency across repeated runs. On a 40-image retinal subset (HRF, APTOS, ODIR, IDRiD), MedSAM visual concepts and VLM semantic concepts for lesion, vessels, and optic disc are compared. The main empirical claim is that reliability is pathway- and concept-dependent: MedSAM is stronger on spatial attribution and surrogate fidelity (R²=0.8503, R_w²=0.8465), while the VLM pathway is stronger on vessel faithfulness and selected artefact stability.

Significance. If the audit is valid, the paper supplies a useful independent reliability layer for C-XAI rather than another concept extractor. The multi-metric protocol, explicit surrogate-selection comparison (Table 2), and side-by-side visual vs semantic pathway evaluation are concrete contributions, and the paper is appropriately cautious that human-readable concepts are not automatically trustworthy. The retinal case study is a reasonable high-stakes demonstration domain. The main significance is methodological: a transferable audit recipe that can be applied to other concept pathways once the local test is better validated.

major comments (3)
  1. §4.2, Eqs. (3)–(6), and Table 1: the central comparative claim rests on binary SLIC masking with target segments = 7 and masked regions set to black. With only ~7 large patches, blacking out a region is a strong non-clinical intervention that can alter global image statistics, MedSAM confidence, and VLM grounding for reasons unrelated to the named concept. The Limitations section itself notes that such masks may not be plausible retinal counterfactuals. Without a control or alternative perturbation (e.g., inpainting, anatomy-aware masking, or random black patches outside concept regions), it is not established that the reported faithfulness correlations, fidelity R² values, and pathway rankings measure concept-evidence linkage rather than black-patch artefact sensitivity. This is load-bearing for the MedSAM-vs-VLM reliability profile.
  2. §4.5 / Table 1 and §5: the evaluation uses only 40 images (10 per dataset) and three coarse concepts. Several faithfulness results in Table 4 are non-significant (p ≥ 0.05), and attribution accuracy for lesions is often weak. With this sample size and concept vocabulary, the paper’s strong pathway-dependent conclusions are underpowered as a general reliability claim. Either enlarge the evaluation, report uncertainty/confidence intervals over images, or substantially soften claims that currently read as established comparative properties of MedSAM vs VLM concept pathways.
  3. §4.4, Eq. (11) and Table 3: attribution accuracy converts surrogate importance into binary labels with a threshold τ against clinical reference labels, but the manuscript does not specify how the reference masks/labels are obtained, how τ is chosen, or whether it is fixed across concepts/datasets. Without that protocol, ATT ACC/F1/AUROC cannot be independently interpreted, and the claim that MedSAM has stronger spatial attribution is only partially supported by the reported numbers.

Circularity Check

1 steps flagged

No load-bearing circular derivation: ConceptSMILE audits fixed external concept pathways under perturbation; reported reliability metrics are measured outcomes, not quantities forced by definition or self-fit.

specific steps
  1. self citation load bearing [§2.2 / Introduction contributions; SMILE lineage [17,45–47]]
    "Rather than replacing SMILE, ConceptSMILE extends its perturbation-based logic from feature or region level attribution to the auditing of human-understandable concept explanations. ... SMILE extends the logic of local model-agnostic explanation by using perturbation-based sampling, distance-based locality weighting, and surrogate modelling [17]."

    Minor only: the framework is positioned as an extension of prior SMILE work by overlapping authors. This is methodological self-citation of the perturbation/locality/surrogate template, not a uniqueness theorem or fitted premise that forces the MedSAM-vs-VLM reliability results. The audit metrics are computed on external concept pathways and do not reduce to the SMILE citations by construction.

full rationale

The paper's central claim is empirical and comparative: concept-level explanations from fixed MedSAM and VLM pathways are not automatically trustworthy, and multi-metric perturbation auditing shows pathway- and concept-dependent reliability (MedSAM stronger on spatial attribution and surrogate fidelity; VLM stronger on vessel faithfulness and selected artefact stability). Concept pathways are treated as black boxes; ConceptSMILE does not train or redefine them. Perturbation vectors z^(k), response shifts Δy, locality weights, and the XGBoost surrogate are tools for measuring external behaviour; fidelity (R², R_w²) is explicitly an approximation score for the surrogate, not a circular proof that concepts are faithful. Attribution accuracy uses external clinical reference labels; faithfulness is a Pearson correlation between perturbation strength and |Δy|; stability and consistency are Jaccard/variance measurements under artefacts and repeated runs. Self-citations to SMILE and related SMILE variants are methodological lineage for the perturbation-surrogate idea, not uniqueness theorems that force the reliability conclusions. The superpixel-masking validity concern is a methodological/correctness risk (acknowledged in Limitations), not circularity: the paper does not claim the black-mask neighbourhood is derived from the trustworthiness result it reports. No step reduces a claimed prediction to a fitted input by construction. Score 1 reflects only ordinary non-load-bearing self-citation of the SMILE family.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

The central claim depends less on new physics-like entities than on methodological choices: that local black superpixel neighborhoods, embedding distances, exponential locality weights, and an XGBoost surrogate are adequate probes of concept reliability; that the five chosen metrics jointly operationalize trustworthiness; and that MedSAM/VLM outputs are valid concept responses to audit. Free parameters are mostly experimental knobs (segment count, perturbation count, kernel width, thresholds, surrogate family). Invented entities are framework constructs rather than ontological discoveries.

free parameters (6)
  • SLIC target superpixel count M = 7
    Fixed at 7 segments per image (Table 1); controls perturbation granularity and can change which evidence is co-masked.
  • Number of binary perturbations K = 50 per image
    50 unique superpixel masks per image (Table 1); affects surrogate fit stability and sampling coverage.
  • Locality kernel width σ
    Exponential weight w=exp(-(d/σ)^2) in Eq. (9); not numerically reported but required for all weighted metrics.
  • Attribution threshold τ
    Converts continuous attribution scores to binary labels for ACC/F1 (Eq. 11); threshold choice affects attribution accuracy.
  • XGBoost surrogate hyperparameters = XGBoost regressor (details underspecified)
    Final surrogate class chosen after Table 2 comparison; tree depth/learning rate/regularization not fully specified but drive reported R².
  • Evaluation image subset size = 40 images
    40 images total, 10 from each of HRF/APTOS/ODIR/IDRiD; selection criteria for the controlled subset are only lightly described.
axioms (5)
  • ad hoc to paper Local concept reliability can be audited by measuring concept-response shifts under region masking and fitting a local surrogate to those shifts.
    Core ConceptSMILE premise in §§3–4; extends SMILE/LIME logic to concepts without independent proof that this exhausts trustworthiness.
  • domain assumption Perturbations closer in embedding space (cosine or Wasserstein on DINOv2 embeddings) should receive higher locality weight.
    Eqs. (7)–(9); standard local-explanation assumption, but embedding choice can redefine locality.
  • domain assumption MedSAM masks and fixed-prompt VLM JSON scores are valid concept responses for lesion, vessels, and optic disc.
    §4.1 pathways; audit quality inherits pathway concept quality, as the limitations section notes.
  • ad hoc to paper Attribution accuracy, fidelity, faithfulness, stability, and consistency jointly indicate concept-explanation trustworthiness.
    §4.4 metric suite; useful but incomplete relative to clinical usefulness or causal concept responsibility.
  • domain assumption Pearson correlation between concept-relevant perturbation strength and |Δy| measures faithfulness.
    Eq. (16); common XAI proxy, not a causal guarantee (acknowledged in limitations).
invented entities (2)
  • ConceptSMILE audit pipeline no independent evidence
    purpose: Model-agnostic protocol that turns concept outputs into multi-metric reliability scores via perturbation, locality weighting, and XGBoost surrogates.
    Primary contribution; independent evidence is the empirical tables, not external formal verification.
  • Concept-response shift Δy / Δc no independent evidence
    purpose: Scalar change in concept confidence after masking, used as the supervised target for local surrogates and faithfulness.
    Defined in Eqs. (2) and (6); operational construct rather than a new physical quantity.

reviewed 2026-07-13 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI." pith.science (2026). https://pith.science/paper/AZ6Z3HJM

@misc{pith2026260709649,
  author       = {Pith},
  title        = {Pith review of: ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AZ6Z3HJM}},
  note         = {Machine review of arXiv:2607.09649}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Concept-based explainable artificial intelligence (AI) can make model reasoning more human-understandable, but concept-level outputs are not automatically trustworthy. We introduce ConceptSMILE, a model-agnostic perturbation-based auditing framework for evaluating the reliability of concept-based explanations. Rather than replacing SMILE, ConceptSMILE extends its perturbation-based logic from feature- or region-level attribution to the auditing of human-understandable concept explanations. The framework perturbs input regions, measures concept-response shifts, applies locality weighting, and fits an XGBoost surrogate to approximate local concept behaviour. Reliability is assessed through attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency. We evaluate ConceptSMILE on retinal fundus images by comparing MedSAM-derived visual concepts with VLM-based semantic concepts. Results show that reliability varies across concepts and pathways: MedSAM achieves stronger spatial attribution and the highest surrogate fidelity ($R^2 = 0.8503$, $R_w^2 = 0.8465$), while the VLM pathway shows stronger vessel faithfulness and stronger stability under selected artefact conditions. ConceptSMILE provides an independent audit layer for evaluating the trustworthiness of concept-based XAI.

Figures

Figures reproduced from arXiv: 2607.09649 by Bhupesh Kumar Mishra, Koorosh Aslansefat, Mohadeseh Mollapour, Tejal Shah, Zeinab Dehghani, Zhibao Mian.

Figure 1
Figure 1. Figure 1: Integrated concept-based explainability pipeline for retinal analysis. 2 Literature Review This section reviews C-XAI and its role in making model reasoning more transparent, interpretable, and trustworthy [1, 2, 9]. We first summarise existing post-hoc and explainable-by-design C-XAI methods, then examine reliability challenges such as spurious correlation, concept leakage, instability, and weak faithfuln… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of Concept-Based XAI for retinal imaging. The pipeline maps fundus images into human-understandable clinical concepts and uses them to support concept-level explanation and prediction. 2.1.1 Challenges in Concept-Based XAI Although Concept-Based Explainable AI provides a more human-understandable form of interpretation, concept-level explanations should not be assumed to be inherently reliable. A … view at source ↗
Figure 3
Figure 3. Figure 3: Key reliability challenges in Concept-Based XAI, including spatial misalignment, concept entanglement, spurious correlation, label leakage, and concept uncertainty. 5 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the ConceptSMILE framework. The pipeline extracts concept-level outputs, applies superpixel perturbations, computes locality weights, and fits a surrogate model for reliability auditing. The resulting explanations are then assessed using multiple reliability dimensions, including attribution accuracy, fidelity, faithfulness, stability, and consistency. In this study, ConceptSMILE is demonstrate… view at source ↗
Figure 5
Figure 5. Figure 5: Perturbation-based concept graph auditing in ConceptSMILE. Masking the optic-disc region changes the corresponding concept node and illustrates how concept confidence responds to evidence removal. Let the original input image be denoted as I (0). The image is first divided into M non-overlapping superpixel regions: S = {S1, S2, . . . , SM}, (3) where each Sm represents one local image region. The superpixe… view at source ↗
Figure 6
Figure 6. Figure 6: ConceptSMILE evaluation framework for auditing concept-level explanations through attribution accuracy, fidelity, faithfulness, stability, and consistency. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: illustrates the fidelity evaluation pipeline, where observed concept-response shifts are compared with surrogate￾predicted shifts to assess local explanation fidelity [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative ConceptSMILE output illustrating traceable alignment between retinal concept evidence, the concept graph, and the generated concept-level report. Colour-matched highlights link the clear optic disc, normal cup-to-disc ratio, and hard-exudate concepts to their corresponding textual evidence. These qualitative outputs show that ConceptSMILE can represent retinal evidence at the concept level rath… view at source ↗
Figure 9
Figure 9. Figure 9: XGBoost surrogate fidelity comparison across HRF, APTOS, ODIR-5K, and IDRiD. The left column shows MedSAM-based visual node fidelity, and the right column shows VLM-based semantic concept fidelity. Points closer to the dashed diagonal indicate stronger agreement between actual and predicted concept-confidence values. As reported in [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: VLM-based ConceptSMILE robustness under contrast variation and simulated blink-related occlusion. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: MedSAM-based ConceptSMILE robustness under contrast variation and simulated blink-related occlusion. In Figures 10 and 11, individual points show repeated XGBoost train–test evaluations, solid lines show the mean, diamond markers indicate the median, and shaded regions represent mean ± standard deviation. The results show that acquisition artefacts affect retinal concepts differently. Blood-vessel and les… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 11 linked inside Pith

  1. [1]

    Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai.Information fusion, 58:82–115, 2020

    Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai.Information fusion, 58:82–115, 2020

    2020

  2. [2]

    A survey of methods for explaining black box models.ACM computing surveys (CSUR), 51(5):1–42, 2018

    Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models.ACM computing surveys (CSUR), 51(5):1–42, 2018

    2018

  3. [3]

    why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016

    2016

  4. [4]

    A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    2017

  5. [5]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. InProceedings of the IEEE international conference on computer vision, pages 618–626, 2017

    2017

  6. [6]

    Sanity checks for saliency maps.Advances in neural information processing systems, 31, 2018

    Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps.Advances in neural information processing systems, 31, 2018

    2018

  7. [7]

    Towards automatic concept-based explanations

    Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explanations. Advances in neural information processing systems, 32, 2019

    2019

  8. [8]

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 1(5):206–215, 2019

    Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 1(5):206–215, 2019

    2019

  9. [9]

    Concept-based explainable artificial intelligence: A survey.ACM Computing Surveys, 2023

    Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. Concept-based explainable artificial intelligence: A survey.ACM Computing Surveys, 2023

    2023

  10. [10]

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). InInternational conference on machine learning, pages 2668–2677. PMLR, 2018

    2018

  11. [11]

    Concept bottleneck models

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. InInternational conference on machine learning, pages 5338–5348. PMLR, 2020

    2020

  12. [12]

    This looks like that: deep learning for interpretable image recognition.Advances in neural information processing systems, 32, 2019

    Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition.Advances in neural information processing systems, 32, 2019

    2019

  13. [13]

    Language in a bottle: Language model guided concept bottlenecks for interpretable image classification

    Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187–19197, 2023

    2023

  14. [14]

    Do concept bottleneck models learn as intended? InarXiv preprint arXiv:2105.04289, 2021

    Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn as intended? InarXiv preprint arXiv:2105.04289, 2021

    Pith/arXiv arXiv 2021

  15. [15]

    Addressing leakage in concept bottleneck models

    Marton Havasi, Sonali Parbhoo, and Finale Doshi-Velez. Addressing leakage in concept bottleneck models. In Advances in Neural Information Processing Systems, volume 35, pages 23386–23397, 2022

    2022

  16. [16]

    Glancenets: Interpretable, leak-proof concept-based models.Advances in Neural Information Processing Systems, 35:21212–21227, 2022

    Emanuele Marconato, Andrea Passerini, and Stefano Teso. Glancenets: Interpretable, leak-proof concept-based models.Advances in Neural Information Processing Systems, 35:21212–21227, 2022

    2022

  17. [17]

    Explaining black boxes with a smile: Statistical model-agnostic interpretability with local explanations.arXiv preprint arXiv:2311.07286, 2023

    Koorosh Aslansefat, Mojgan Hashemian, Martin Walker, Mohammed Naveed Akram, Ioannis Sorokos, and Yiannis Papadopoulos. Explaining black boxes with a smile: Statistical model-agnostic interpretability with local explanations.arXiv preprint arXiv:2311.07286, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Network dissection: Quantifying interpretability of deep visual representations

    David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 6541–6549, 2017

    2017

  19. [19]

    How does a deep network make decisions? decomposing deep neural networks into functional parts.arXiv preprint arXiv:2402.13210, 2024

    Thomas Zarlenga, Hedi Ben-Younes, Kirk Ridgeway, Jonathan Grizou, Enguerrand Lachaud, Fernando de la Torre, and Pierre-Luc St-Charles. How does a deep network make decisions? decomposing deep neural networks into functional parts.arXiv preprint arXiv:2402.13210, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability

    Vikram V Ramaswamy, Sunnie SY Kim, Ruth Fong, and Olga Russakovsky. Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10932–10941, 2023. 29 APREPRINT- JULY13, 2026

    2023

  21. [21]

    Making corgis important for honeycomb classification: Adversarial attacks on concept-based explainability tools

    Davis Brown and Henry Kvinge. Making corgis important for honeycomb classification: Adversarial attacks on concept-based explainability tools. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 620–627, 2023

    2023

  22. [22]

    Promises and pitfalls of black-box concept learning models.arXiv preprint arXiv:2106.13314, 2021

    Anita Mahinpei, Justin Clark, Isaac Lage, Finale Doshi-Velez, and Weiwei Pan. Promises and pitfalls of black-box concept learning models.arXiv preprint arXiv:2106.13314, 2021

    Pith/arXiv arXiv 2021

  23. [23]

    Interpretation of neural networks is fragile

    Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. InProceedings of the AAAI conference on artificial intelligence, volume 33, pages 3681–3688, 2019

    2019

  24. [24]

    Probabilistic concept bottleneck models

    Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bottleneck models. arXiv preprint arXiv:2306.01574, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    On completeness- aware concept-based explanations in deep neural networks.Advances in neural information processing systems, 33:20554–20565, 2020

    Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness- aware concept-based explanations in deep neural networks.Advances in neural information processing systems, 33:20554–20565, 2020

    2020

  26. [26]

    Anchors: High-precision model-agnostic explanations

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model-agnostic explanations. InProceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  27. [27]

    S-lime: Stabilized-lime for model explanation

    Zhengze Zhou, Giles Hooker, and Fei Wang. S-lime: Stabilized-lime for model explanation. InProceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2429–2438, 2021

    2021

  28. [28]

    Us-lime: Increasing fidelity in lime using uncertainty sampling on tabular data.Neurocomputing, 597:127969, 2024

    Hamid Saadatfar, Zeinab Kiani-Zadegan, and Benyamin Ghahremani-Nezhad. Us-lime: Increasing fidelity in lime using uncertainty sampling on tabular data.Neurocomputing, 597:127969, 2024

    2024

  29. [29]

    Guided-lime: Structured sampling based hybrid approach towards explaining blackbox machine learning models

    Amit Sangroya, Mouli Rastogi, C Anantaram, and Lovekesh Vig. Guided-lime: Structured sampling based hybrid approach towards explaining blackbox machine learning models. InCIKM (Workshops), volume 17, 2020

    2020

  30. [30]

    Deterministic local interpretable model-agnostic explanations for stable explainability.Machine Learning and Knowledge Extraction, 3(3):525–541, 2021

    Muhammad Rehman Zafar and Naimul Khan. Deterministic local interpretable model-agnostic explanations for stable explainability.Machine Learning and Knowledge Extraction, 3(3):525–541, 2021

    2021

  31. [31]

    Defining locality for surrogates in post-hoc interpretablity.arXiv preprint arXiv:1806.07498, 2018

    Thibault Laugel, Xavier Renard, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. Defining locality for surrogates in post-hoc interpretablity.arXiv preprint arXiv:1806.07498, 2018

    Pith/arXiv arXiv 2018

  32. [32]

    Qlime-a quadratic local interpretable model- agnostic explanation approach.SMU Data Science Review, 3(1):4, 2020

    Steven Bramhall, Hayley Horn, Michael Tieu, and Nibhrat Lohia. Qlime-a quadratic local interpretable model- agnostic explanation approach.SMU Data Science Review, 3(1):4, 2020

    2020

  33. [33]

    Baylime: Bayesian local interpretable model-agnostic explanations

    Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, and David Flynn. Baylime: Bayesian local interpretable model-agnostic explanations. InUncertainty in artificial intelligence, pages 887–896. PMLR, 2021

    2021

  34. [34]

    Alime: Autoencoder based approach for local interpretability

    Sharath M Shankaranarayana and Davor Runje. Alime: Autoencoder based approach for local interpretability. In International conference on intelligent data engineering and automated learning, pages 454–463. Springer, 2019

    2019

  35. [35]

    Optilime: Optimized lime explanations for diagnostic computer algorithms.arXiv preprint arXiv:2006.05714, 2020

    Giorgio Visani, Enrico Bagli, and Federico Chesani. Optilime: Optimized lime explanations for diagnostic computer algorithms.arXiv preprint arXiv:2006.05714, 2020

    Pith/arXiv arXiv 2006

  36. [36]

    Glime: general, stable and local lime explanation.Advances in neural information processing systems, 36:36250–36277, 2023

    Zeren Tan, Yang Tian, and Jian Li. Glime: general, stable and local lime explanation.Advances in neural information processing systems, 36:36250–36277, 2023

    2023

  37. [37]

    Graphlime: Local interpretable model explanations for graph neural networks.IEEE Transactions on Knowledge and Data Engineering, 35(7):6968– 6972, 2022

    Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, and Yi Chang. Graphlime: Local interpretable model explanations for graph neural networks.IEEE Transactions on Knowledge and Data Engineering, 35(7):6968– 6972, 2022

    2022

  38. [38]

    Ts-mule: Local interpretable model-agnostic explanations for time series forecast models

    Udo Schlegel, Duy Lam V o, Daniel A Keim, and Daniel Seebacher. Ts-mule: Local interpretable model-agnostic explanations for time series forecast models. InJoint european conference on machine learning and knowledge discovery in databases, pages 5–14. Springer, 2021

    2021

  39. [39]

    Local interpretable model-agnostic explanations for music content analysis

    Saumitra Mishra, Bob L Sturm, and Simon Dixon. Local interpretable model-agnostic explanations for music content analysis. InISMIR, volume 53, pages 537–543, 2017

    2017

  40. [40]

    B-lime: An improvement of lime for interpretable deep learning classification of cardiac arrhythmia from ecg signals.Processes, 11(2):595, 2023

    Talal AA Abdullah, Mohd Soperi Mohd Zahid, Waleed Ali, and Shahab Ul Hassan. B-lime: An improvement of lime for interpretable deep learning classification of cardiac arrhythmia from ecg signals.Processes, 11(2):595, 2023

    2023

  41. [41]

    Which lime should i trust? concepts, challenges, and solutions

    Patrick Knab, Sascha Marton, Udo Schlegel, and Christian Bartelt. Which lime should i trust? concepts, challenges, and solutions. InWorld Conference on Explainable Artificial Intelligence, pages 28–52. Springer, 2025

    2025

  42. [42]

    Dseg-lime: Improving image explanation by hierarchical data-driven segmentation.arXiv preprint arXiv:2403.07733, 2024

    Pascal Knab, Stefan Marton, and Christian Bartelt. Dseg-lime: Improving image explanation by hierarchical data-driven segmentation.arXiv preprint arXiv:2403.07733, 2024

    arXiv 2024

  43. [43]

    Slice: Stabilized lime for consistent explanations for image classification

    Rohit Bora, Philipp Terhörst, Raymond Veldhuis, Raghavendra Ramachandra, and Kiran Raja. Slice: Stabilized lime for consistent explanations for image classification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10988–10996, 2024. 30 APREPRINT- JULY13, 2026

    2024

  44. [44]

    Which lime should i trust? concepts, challenges, and solutions.arXiv preprint arXiv:2503.24365, 2025

    Pascal Knab, Stefan Marton, Udo Schlegel, and Christian Bartelt. Which lime should i trust? concepts, challenges, and solutions.arXiv preprint arXiv:2503.24365, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    Mapping the mind of an instruction-based image editing using smile.arXiv preprint arXiv:2412.16277, 2024

    Zeinab Dehghani, Koorosh Aslansefat, Adil Khan, Adín Ramírez Rivera, Franky George, and Muhammad Khalid. Mapping the mind of an instruction-based image editing using smile.arXiv preprint arXiv:2412.16277, 2024

    Pith/arXiv arXiv 2024

  46. [46]

    Explainable knowledge graph retrieval-augmented generation (kg-rag) with kg-smile.arXiv preprint arXiv:2509.03626, 2025

    Zahra Zehtabi Sabeti Moghaddam, Zeinab Dehghani, Maneeha Rani, Koorosh Aslansefat, Bhupesh Kumar Mishra, Rameez Raja Kureshi, and Dhavalkumar Thakker. Explainable knowledge graph retrieval-augmented generation (kg-rag) with kg-smile.arXiv preprint arXiv:2509.03626, 2025

    Pith/arXiv arXiv 2025

  47. [47]

    Explainability of large language models using smile: Statistical model-agnostic interpretability with local explanations.arXiv preprint arXiv:2505.21657, 2025

    Zeinab Dehghani, Mohammed Naveed Akram, Koorosh Aslansefat, and Adil Khan. Explainability of large language models using smile: Statistical model-agnostic interpretability with local explanations.arXiv preprint arXiv:2505.21657, 2025

    arXiv 2025

  48. [48]

    Interpreting black-box large language models with sentence-level energy landscapes

    Maryam Rezaee Pooriya Safaei Maryam Asgarinezhad and S Fatemeh Seyyedsalehi. Interpreting black-box large language models with sentence-level energy landscapes

  49. [49]

    Computational optimal transport: With applications to data science.Founda- tions and Trends in Machine Learning, 11(5–6):355–607, 2019

    Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science.Founda- tions and Trends in Machine Learning, 11(5–6):355–607, 2019

    2019

  50. [50]

    Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, et al

    Varun Gulshan, Lily Peng, Marc Coram, Martin C. Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.JAMA, 316(22):2402–2410, 2016

    2016

  51. [51]

    Abràmoff, Philip T

    Michael D. Abràmoff, Philip T. Lavin, Michele Birch, Nilay Shah, and James C. Folk. Pivotal trial of an autonomous ai-based diagnostic system for detection of diabetic retinopathy in primary care offices.NPJ Digital Medicine, 1(1):39, 2018

    2018

  52. [52]

    Chia, Siegfried K

    Yukun Zhou, Mark A. Chia, Siegfried K. Wagner, Murat S. Ayhan, Dominic J. Williamson, Robbert R. Struyven, Ting Liu, Moucheng Xu, Mauricio G. Lozano, Peter Woodward-Court, et al. A foundation model for generalizable disease detection from retinal images.Nature, 622(7981):156–163, 2023

    2023

  53. [53]

    Kinahan, and Yu Qiao

    Zihan Li, Diping Song, Zefeng Yang, Deming Wang, Fei Li, Xiulan Zhang, Paul E. Kinahan, and Yu Qiao. Visionunite: A vision-language foundation model for ophthalmology enhanced with clinical knowledge.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  54. [54]

    Segment anything in medical images.Nature communications, 15(1):654, 2024

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images.Nature communications, 15(1):654, 2024

    2024

  55. [55]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025. 31

    Pith/arXiv arXiv 2025

This paper was first reviewed by grok-4.5 on July 13, 2026.