Pith. sign in

REVIEW 4 major objections 4 minor 87 references

Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single learned direction in latent space lets a medical AI pipeline detect, localize, and unlearn spurious artifacts with little expert labeling.

desk verdict A useful empirical integration of CAV-based bias annotation into Reveal2Revise, but the headline claim about ViT mitigation is not supported by the paper's own appendix. read the letter →

arxiv 2501.13818 v2 pith:DPN2RJ6E submitted 2025-01-23 cs.AI cs.CVcs.LG

Frederik Pahde , Thomas Wiegand , Sebastian Lapuschkin , Wojciech Samek This is my paper · ORCID
classification cs.AIcs.CVcs.LG
keywords spuriouscorrelationsshortcutlearningconceptactivationvectorsinterpretabilitybiasmitigationmedicalimagingECGtimeseriesexplainableAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep neural networks used for medical decisions can latch onto spurious artifacts—band-aids, rulers, pacemakers, brightness shifts, or inserted noise—that correlate with a diagnosis without causing it. This paper extends the Reveal2Revise bias-correction loop, which detects such shortcuts, models them, and unlearns them, with semi-automated annotation tools. A concept activation vector (CAV), a direction in the model's latent space that separates artifact-bearing from clean activations, ranks unlabeled samples by bias score to surface more artifact examples and produces spatial masks that localize the artifact. The authors demonstrate the extended loop on melanoma, gastrointestinal, chest radiograph, and electrocardiogram (ECG) datasets using VGG16, ResNet50, and Vision Transformer models, reporting that detected biases are identified, localized, and mitigated with reduced artifact reliance. The practical payoff is that bias correction becomes far less dependent on expensive expert labeling.

What carries the argument

The load-bearing object is the Concept Activation Vector (CAV), defined as a direction in a chosen layer's activation space that separates samples containing the artifact from samples without it. It works as a ranker, as an iteratively refinable bias model, and as a localizer: projecting activations onto the CAV yields a bias score, while using the element-wise product of activations and the CAV as the initialization for layer-wise relevance propagation turns the direction into input-space heatmaps and binary masks. The choice of layer is part of the machinery, and the paper treats it as a hyperparameter that strongly affects retrieval and localization quality.

What would settle it

Take a controlled artifact with ground-truth masks, such as the inserted timestamp in HyperKvasir, train CAVs on every layer of a fixed model, and test both ranking and localization on held-out samples; the central claim is falsified if some artifact admits no layer whose CAV separates artifact samples from clean samples above chance while its localization mask overlaps the ground truth no better than randomly.

Watch

Extended reading notes

Core claim

The paper's central claim is that the two least-automated steps of the Reveal2Revise life cycle—finding which samples carry a spurious artifact and marking where the artifact sits—can be driven by a single linear bias representation. Starting from a small seed set of artifact samples, a CAV trained at a chosen layer gives a bias score for every other sample; ranking by that score retrieves previously unlabeled artifact images, and iterating with brief human inspection refines the direction. The same CAV, when combined with layer-wise relevance propagation, localizes the artifact in input space as a soft or binary mask. The paper reports that this closes the loop across four medical datasets and two modalities: the identified biases are unlearned with gradient-penalty or activation-projection corrections, accuracy on biased test sets rises, and artifact relevance and concept-sensitivity (TCAV) scores fall while clean-test accuracy is largely preserved.

Load-bearing premise

The pipeline assumes a spurious artifact can be captured as one linear direction in some layer of the network's latent space, and that a suitable layer can be found.

Editorial extensions

If this is right

  • A small seed set of artifact samples can be expanded to a full dataset annotation by ranking unlabeled samples with CAV bias scores, reducing the expert labeling burden in the reveal and modeling stages.
  • Spatial masks produced by CAV localization can feed input-gradient-based correction methods such as RRR, removing the need for hand-drawn pixel-level artifact masks.
  • In the reported experiments, CAV-based mitigation methods such as RR-ClArC improve accuracy on biased test sets while keeping clean-test accuracy close to the baseline model's.
  • The framework transfers beyond images to time-series medical data, where ECG models trained with an inserted static-noise artifact show the same detect-and-mitigate pattern.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors leave implicit is automatic layer selection—for example, choosing the layer whose CAV maximizes retrieval or localization quality on a validation set—which would remove the extra hyperparameter they flag in their limitations.
  • The bias-score ranking could be deployed as a standalone dataset-auditing tool before any model correction, quantifying artifact prevalence and exposing labeling errors in medical training data.
  • If the linear-direction assumption fails for heavily entangled concepts, the same loop would need non-linear concept regions or sparse-autoencoder disentanglement; the paper itself points to sparse autoencoders as future work.
  • Outside medicine, the same CAV annotation loop should apply to any domain where device-specific artifacts correlate with labels, such as satellite imagery or industrial inspection, though non-localizable artifacts like color shifts would remain harder to mask.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reviews and extends the Reveal2Revise framework for detecting and mitigating spurious correlations in medical deep learning models. The extension adds semi-automated bias annotation: CAV-based biased-sample retrieval, iterative refinement of the bias representation, and spatial bias localization, followed by mitigation with RRR and ClArC variants. The authors evaluate on four medical datasets across two modalities (image and ECG) using VGG16, ResNet50, and ViT architectures, with both real-world and artificially controlled artifacts. The central claim is that the extended framework successfully identifies and mitigates these biases across all three vision architectures.

Significance. The paper has clear practical value if the claims are properly scoped. The strengths are real: controlled artifacts with ground-truth masks, externally anchored artifact labels for real-world biases, quantitative retrieval and localization evaluations on multiple layers and architectures, and released code and annotations. The bias identification and retrieval results are largely convincing and would be useful to practitioners. However, the headline claim of successful mitigation in ViTs is not supported by the reported numbers, and part of the mitigation evaluation is circular because the TCAV/artifact-relevance metrics are computed along the same CAV direction that the mitigation methods erase. These issues weaken the central claim as written, although they are addressable by revising the claims and adding independent evaluation.

major comments (4)
  1. [Abstract; §6.5; Appendix A.7.4 (Table 11)] The claim that the framework "successfully identify[ies] and mitigat[es] these biases in VGG16, ResNet50, and contemporary Vision Transformer models" is not supported for ViTs. In Table 11, the CAV-based methods RR-ClArC, P-ClArC, and rP-ClArC leave biased-test accuracy essentially unchanged: HyperKvasir stays at 0.42 for RR-ClArC, ISIC2019 moves only from 0.27 to 0.31, and CheXpert from 0.25 to 0.31. The only substantial ViT improvement comes from RRR with ground-truth masks (ISIC2019 0.27→0.59, HyperKvasir 0.42→0.54), which requires manual pixel-level annotations and is not the proposed semi-automated CAV annotation pipeline. The abstract and conclusions should be revised to claim ViT detection and localization, with mitigation demonstrated only when ground-truth masks are provided.
  2. [§6.5; Appendix A.7.4 (Eq. 12, Table 11)] The mitigation evaluation is partly circular: TCAV sensitivity and artifact relevance are computed along the same CAV direction that RR-ClArC and P-ClArC explicitly erase. Table 11 shows RR-ClArC driving ViT TCAVsens to 0.00 while biased accuracy remains flat, which indicates that the metric may be measuring removal of the direction rather than behavioral robustness. The paper should include an independent evaluation, such as a CAV computed from a held-out layer or annotation source, or behavioral insertion/deletion tests, before claiming successful ViT mitigation.
  3. [§6.5; Appendix A.7.4 (Tables 9 and 10)] The proposed semi-automated feature-level annotations are not actually connected to the main mitigation results. Table 1 in the main text uses RRR with ground-truth masks, and the automated heatmap and binarized masks appear only in the appendix with substantially smaller gains: for ResNet50 on ISIC2019, biased accuracy is 0.44 (hm) and 0.43 (bin) versus 0.51 with ground truth, and for HyperKvasir 0.77 versus 0.82. Automated masks are not reported for ViT at all. The claim that the feature-level annotation pipeline reduces labeling effort in the full mitigation loop therefore needs to be either demonstrated with automated masks in the main results or explicitly qualified.
  4. [§6.4; Appendix A.7.3 (Table 7)] The spatial localization results are weaker than the retrieval results: for the microscope artifact, IoU is below 0.16 for all layers and architectures because the models attend only to the border of the circle. The paper acknowledges this in the limitations section, but the low IoU has direct consequences for the automated-mask mitigation pipeline discussed in the previous comment. The authors should state more explicitly that RRR with automated masks is currently only practical for well-localized artifacts such as the timestamp, not for artifacts where the model's concept representation differs from the human-annotated region.
minor comments (4)
  1. [Throughout] There are several typos and small errors, including "n a medical context" and "sub-popluations" in §2, "overivew" in Appendix A.1, "miscroscope" in Table 7, "clearn dataset" in Appendix A.7.4, "perspetive" in Appendix A.7.3, and "in int terms" in Appendix A.7.4. These should be corrected.
  2. [§6.3; Figure 7] The retrieval plots would be easier to read if the best-performing layer per artifact were marked directly on the figure, since the text refers to layer choice as crucial and Figures 7 and 9 show that the optimal layer differs between retrieval and localization.
  3. [§3.3; Eq. (2)] Equation (2) writes R_l(x) = a_l(x) ∘ h_l without specifying whether a_l is the full spatial activation tensor or a pooled representation; because the paper uses max-pooled 1D activations for CAVs, the dimensions of the element-wise product should be stated explicitly.
  4. [§7; Bias Modeling] The limitation paragraph on layer selection is well placed, but since the paper identifies layer choice as an additional hyperparameter and shows that retrieval and localization prefer different layers, a short paragraph in the main text on how a practitioner would choose the layer in a real deployment would strengthen the practical contribution.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the TCAV-based mitigation metric is the square of the RR-ClArC training objective, and the proposed CAV methods do not improve ViT biased accuracy.

  1. fitted input called prediction [Sec. 6.5; Appendix A.4.1 Eq. (10) vs Appendix A.7.4 Eqs. (12)-(13); Appendix Table 11]
    "Given a bias direction hl, the latent right reason lossLRR-ClArC is defined as follows: LRR-ClArC(x, hl) = ( ∇a ˜f(al(x)· hl )2 . (10) ... The latter is based on the concept sensitivity measured via CAV h and the latent feature use, computed as gradient of the prediction w.r.t. latent activations a(x): TCAVsens(x) = ∇a ˜f(al(x))· h . (12) ... We report ∆TCAV = |TCAV−0.5|, where 0 indicates no sensitivity and higher values are interpreted as reliance on the artifact."

    The RR-ClArC training objective (Eq. 10) is the square of the exact sensitivity whose sign/fraction defines the TCAV score (Eqs. 12-13) used to evaluate mitigation success. Minimizing Eq. 10 therefore drives the reported TCAV/∆TCAV metric toward zero largely by construction, independent of whether the model actually becomes more robust on biased inputs. Appendix Table 11 exhibits this reduction: for ViT, RR-ClArC yields TCAVsens = 0.00 while biased-test accuracy remains at the Vanilla level (e.g., HyperKvasir 0.42 → 0.42).

full rationale

The framework's detection and annotation pipeline is anchored to external ground truth: controlled artifact insertions, manually provided artifact labels, and held-out retrieval AUROC/AP against those labels, so the main bias-modeling claims are not self-referential in their primary evaluation. The heavy self-citation to the authors' own Reveal2Revise, Pattern-CAV, and RR-ClArC works is normal and not load-bearing for the central empirical results, since the decisive accuracy numbers use independent biased test sets. The partial circularity is localized to the mitigation evaluation: for RR-ClArC, the reported TCAV/∆TCAV decrease is the squared quantity being minimized by the loss, and for P-ClArC/rP-ClArC the same CAV direction is both erased and used to measure sensitivity. This matters because the paper claims successful ViT mitigation, yet Appendix Table 11 shows the proposed CAV methods leave ViT biased accuracy near the Vanilla baseline while TCAVsens collapses to 0.00; the only ViT improvement comes from RRR with ground-truth masks, which is outside the proposed semi-automated annotation pipeline. The paper's Section 7 limitations about layer selection and concept entanglement are correctness risks, not circularity. Overall, because VGG/ResNet mitigation is independently supported by biased-accuracy gains and the retrieval/localization claims use external labels, the circularity is partial rather than total: score 4.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

All claimed capabilities rest on the linear-representation assumption for artifacts, on the fidelity of attribution methods, and on human validation of outliers. No new physical or model entities are introduced; the main tuned quantities are the CAV layer, mitigation weight, inspection counts, and neuron selection, with values chosen per dataset and model.

free parameters (5)
  • CAV layer l for bias representation = per artifact/model; e.g., ResNet50 Block 1-4, VGG16 Conv 4-13
    Layer choice is a hyperparameter selected per dataset and artifact by validation performance; the authors note it is crucial for retrieval and localization (Secs. 6.3-6.4, Tabs. 5, 7) and can differ between retrieval and localization objectives.
  • Bias mitigation weight lambda = values in Tab. 12, e.g., RRR 5e2-5e5, RR-ClArC 1e1-5e12
    Lambda balances the classification loss and the right-reason loss; it is selected on the validation set and reported in Tab. 12. Mitigation results depend strongly on it.
  • Number of inspected samples per iteration ninspect = not reported
    Algorithm 2 uses ninspect as the number of top-scoring samples manually inspected each CAV refinement round; no value is given, so the human-effort gain cannot be quantified.
  • Best neuron selection for neuron-based retrieval = chosen on validation set
    Single-neuron baselines pick the best neuron per artifact and layer on validation data before test reporting; this is an additional selection step not available in deployment.
  • Clustering and prototype parameters = not reported
    Outlier detection results depend on unspecified cluster counts for SpRAy, GMM components for PCX, and related prototype parameters, limiting reproducibility of the bias identification step.
assumptions (6)
  • domain assumption Spurious artifacts can be represented as linear directions in some latent layer of the network.
    Required for Eq. 1, Eq. 2, and all CAV-based annotation and mitigation; authors acknowledge in Sec. 7 that no guarantee exists for a perfectly separating layer.
  • domain assumption Neurons and latent directions in deep networks correspond to human-aligned concepts, especially near the head.
    Motivates concept-based interpretation in Sec. 3.1; cited works provide evidence, but there is no proof for medical models.
  • domain assumption Attribution methods such as LRP faithfully reflect the features the model uses.
    Used to compute SpRAy, latent relevances, artifact relevance, and localization masks in Secs. 3.3, 4.1, and 6.4.
  • domain assumption Outlier model behavior detected by clustering corresponds to spurious correlations rather than rare but valid clinical strategies.
    Acknowledged in Sec. 4: 'outlier model reasoning is not necessarily caused by spurious correlations'; human validation is required.
  • domain assumption Over-parameterization yields redundant neurons so that outlier concept detection is meaningful.
    Sec. 4.2 states 'this approach assumes over-parameterization resulting in redundant neurons'.
  • domain assumption The initial set of biased samples from bias identification methods is reliable enough to train a useful CAV.
    Algorithm 2 assumes a small seed set of biased samples from identification methods as accurate starting labels for the iterative refinement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data." pith.science (2026). https://pith.science/paper/DPN2RJ6E

@misc{pith2026250113818,
  author       = {Pith},
  title        = {Pith review of: Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DPN2RJ6E}},
  note         = {Machine review of arXiv:2501.13818}
}
read the original abstract

Deep neural networks are increasingly employed in high-stakes medical applications, despite their tendency for shortcut learning in the presence of spurious correlations, which can have potentially fatal consequences in practice. Whereas a multitude of works address either the detection or mitigation of such shortcut behavior in isolation, the Reveal2Revise approach provides a comprehensive bias mitigation framework combining these steps. However, effectively addressing these biases often requires substantial labeling efforts from domain experts. In this work, we review the steps of the Reveal2Revise framework and enhance it with semi-automated interpretability-based bias annotation capabilities. This includes methods for the sample- and feature-level bias annotation, providing valuable information for bias mitigation methods to unlearn the undesired shortcut behavior. We show the applicability of the framework using four medical datasets across two modalities, featuring controlled and real-world spurious correlations caused by data artifacts. We successfully identify and mitigate these biases in VGG16, ResNet50, and contemporary Vision Transformer models, ultimately increasing their robustness and applicability for real-world medical tasks. Our code is available at https://github.com/frederikpahde/medical-ai-safety.

Figures

Figures reproduced from arXiv: 2501.13818 by the authors.

Figure 1
Figure 1. Extending the (a) reveal and (b) bias modeling steps of the Reveal2Revise framework, we [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Left: Usage of a CAV encoding the reflection concept, trained on known artifact (•) and non-artifact (•) samples, for the annotation of unknown (•) samples. We rank samples by their bias score, computed by projecting their activation onto the CAV hl . Right: Localization of data artifacts by computing relevance heatmaps for the CAV hl for soft masking and their binarization. tical groupings without guidance, such th… view at source ↗
Figure 3
Figure 3. Iterative data annotation: ( I ) Given a small set of biased samples obtained via bias identification approaches, ( II) a first CAV can be fitted. Using this CAV, (III) samples with high bias scores are subject to manual inspection to improve label quality. In an iterative process ( IV ), CAVs are refitted, and the manual inspection is repeated, leading to an ( V ) improved set of annotated samples. [67] leverage Cl… view at source ↗
Figures from the paper (31 more)
Figure 4
Figure 4. Figure 4: Spurious correlations are identified by detecting outliers in model behavior. This involves [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Examples for considered artifacts (f.l.t.r.): We use the real-world artifacts [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Detected outlier behavior for the prediction of melanoma using a ResNet50 model trained on [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Quantitative data annotation results measuring the ranking capabilities via AP and AUROC [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Distribution of latent activations projected onto CAV direction for known artifacts [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Bias localization results measuring the artifact relevance and IoU for various layers of VGG16 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Artifact identification on the last conv layer of a XResNet50 model trained on ECG data [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: PCX visualizations using latent relevances after the last residual block of the XResNet50 [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Artifact identification on the last (13th) conv layer of a VGG16 model trained on HyperKvasir with samples from the “no-disease”-class using SpRAy on latent relevances for the data perspective (left) and pair-wise cosine similarities for the model perspective (right).…
Figure 13
Figure 13. Figure 13: Bias identification after the 3rd residual block of a ResNet50 model trained on HyperKvasir using the controlled timestamp artifact with samples from the “disease”-class using SpRAy on latent relevances for the data perspective (left) and pair-wise cosine similarities…
Figure 14
Figure 14. Figure 14: Bias identification after the 10th conv layer of a VGG16 model trained on CheXpert with samples from the “cardiomegaly”-class using SpRAy on latent relevances for the data perspective (left) and pair-wise cosine similarities for the model perspective (right). We ident…
Figure 15
Figure 15. Figure 15: Bias identification after the 12th conv layer of a VGG16 model trained on CheXpert using the controlled brightness artifact with samples from the “cardiomegaly”-class using SpRAy on latent relevances for the data perspective (left) and pair-wise cosine similarities fo…
Figure 16
Figure 16. Figure 16: Bias identification after the 3rd residual block of a ResNet50 model trained on ISIC2019 using the controlled microscope artifact with samples from the “melanoma”-class using SpRAy on latent relevances for the data perspective (left) and pair-wise cosine similarities …
Figure 17
Figure 17. Figure 17: Bias identification from the data perspective using SpRAy with relevances after the 3 [PITH_FULL_IMAGE:figures/full_fig_p031_17.png]
Figure 18
Figure 18. Figure 18: Bias identification from the data perspective using SpRAy with relevances after the 3 [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]
Figure 19
Figure 19. Figure 19: Bias identification from the model perspective using pair-wise cosine similarities between max [PITH_FULL_IMAGE:figures/full_fig_p033_19.png]
Figure 20
Figure 20. Figure 20: Bias identification from the model perspective using pair-wise cosine similarities between max [PITH_FULL_IMAGE:figures/full_fig_p034_20.png]
Figure 21
Figure 21. Figure 21: PCX visualizations using latent relevances after the last (13 [PITH_FULL_IMAGE:figures/full_fig_p034_21.png]
Figure 22
Figure 22. Figure 22: PCX visualizations using latent relevances after the 12 [PITH_FULL_IMAGE:figures/full_fig_p035_22.png]
Figure 23
Figure 23. Figure 23: PCX visualizations using latent relevances after the 3 [PITH_FULL_IMAGE:figures/full_fig_p036_23.png]
Figure 24
Figure 24. Figure 24: Artifact identification for a ViT model trained on CheXpert with the [PITH_FULL_IMAGE:figures/full_fig_p037_24.png]
Figure 25
Figure 25. Figure 25: PCX visualizations using latent relevances for the [PITH_FULL_IMAGE:figures/full_fig_p038_25.png]
Figure 26
Figure 26. Figure 26: Top: Distribution of bias scores for real artifacts band-aid (left) and skin-marker (right) in the ISIC2019, split into unlabeled samples (blue) and samples labeled (orange) as artifact. Bottom: Samples at the 1-, 50-, and 99-percentile of each sample set and the arti…
Figure 27
Figure 27. Figure 27: Top: Distribution of bias scores for controlled artifacts miscroscope in ISIC2019 (left) and timestamp in HyperKvasir (right), split into clean (blue) and manipulated (orange) samples. Bottom: Sam￾ples at the 1-, 50-, and 99-percentile of each sample set and the artif…
Figure 28
Figure 28. Figure 28: Top: Distribution of bias scores for the controlled brightness artifact in CheXpert for ResNet50 (left) and ViT (right), split into clean (blue) and manipulated (orange) samples. Bottom: Samples at the 1-, 50-, and 99-percentile of each sample set and the artifact loc…
Figure 29
Figure 29. Figure 29: Examples for bias localization for the controlled [PITH_FULL_IMAGE:figures/full_fig_p040_29.png]
Figure 30
Figure 30. Figure 30: Examples for bias localization for the controlled [PITH_FULL_IMAGE:figures/full_fig_p040_30.png]
Figure 31
Figure 31. Figure 31: Examples for bias localization for the real-world artifacts [PITH_FULL_IMAGE:figures/full_fig_p040_31.png]
Figure 32
Figure 32. Figure 32: Examples for bias localization for the real-world artifacts [PITH_FULL_IMAGE:figures/full_fig_p041_32.png]
Figure 33
Figure 33. Figure 33: Examples for bias localization for the controlled artifacts [PITH_FULL_IMAGE:figures/full_fig_p041_33.png]
Figure 34
Figure 34. Figure 34: Local explanations, computed via LRP and presented as relevance heatmaps, wihout ( [PITH_FULL_IMAGE:figures/full_fig_p042_34.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

87 extracted references · 74 canonical work pages

  1. [1]

    From attribution maps to human- understandable explanations through concept relevance propagation

    Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wie- gand, Wojciech Samek, and Sebastian La- puschkin. From attribution maps to human- understandable explanations through concept relevance propagation. Nat Mach Intell , 5(9): 1006–1019, 2023

    2023

  2. [2]

    Under- standing intermediate layers using linear classi- fier probes

    Guillaume Alain and Yoshua Bengio. Under- standing intermediate layers using linear classi- fier probes. ICLR, 2017

    2017

  3. [3]

    Finding and removing clever hans: Using explanation meth- ods to debug and improve deep models

    Christopher J Anders, Leander Weber, David Neumann, Wojciech Samek, Klaus-Robert M¨ uller, and Sebastian Lapuschkin. Finding and removing clever hans: Using explanation meth- ods to debug and improve deep models. Infor- mation Fusion, 77:261–295, 2022

    2022

  4. [4]

    On pixel-wise ex- planations for non-linear classifier decisions by layer-wise relevance propagation

    Sebastian Bach, Alexander Binder, Gr´ egoire Montavon, Frederick Klauschen, Klaus-Robert M¨ uller, and Wojciech Samek. On pixel-wise ex- planations for non-linear classifier decisions by layer-wise relevance propagation. PloS one , 10 (7):e0130140, 2015

    2015

  5. [5]

    Reactive model correction: Mitigating harm to task-relevant features via conditional bias suppression

    Dilyara Bareeva, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, and Sebastian La- puschkin. Reactive model correction: Mitigating harm to task-relevant features via conditional bias suppression. In CVPRW, pages 3532–3541, 2024

    2024

  6. [6]

    Understanding the role of individual units in a deep neural network

    David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Tor- ralba. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences , 117(48):30071– 30078, 2020

    2020

  7. [7]

    Explainability for fair machine learning

    Tom Begley, Tobias Schwedes, Christopher Frye, and Ilya Feige. Explainability for fair machine learning. arXiv preprint arXiv:2010.07389, 2020

    arXiv 2010

  8. [8]

    Probing classifiers: Promises, shortcomings, and advances

    Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219, 2022

    2022

Show all 87 references
  1. [9]

    Leace: Perfect linear concept erasure in closed form

    Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. NeurIPS, 36, 2024

    2024

  2. [10]

    Debiasing skin lesion datasets and models? not so fast

    Alceu Bissoto, Eduardo Valle, and Sandra Avila. Debiasing skin lesion datasets and models? not so fast. In CVPRW, pages 740–741, 2020

    2020

  3. [11]

    Hyper- kvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy.Sci- entific data, 7(1):283, 2020

    Hanna Borgli, Vajira Thambawita, Pia H Smed- srud, Steven Hicks, Debesh Jha, Sigrun L Es- keland, Kristin Ranheim Randel, et al. Hyper- kvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy.Sci- entific data, 7(1):283, 2020

    2020

  4. [12]

    Natural images are more informative for inter- preting cnn activations than state-of-the-art syn- thetic feature visualizations

    Judy Borowski, Roland Simon Zimmermann, Judith Schepers, Robert Geirhos, Thomas SA Wallis, Matthias Bethge, and Wieland Brendel. Natural images are more informative for inter- preting cnn activations than state-of-the-art syn- thetic feature visualizations. In NeurIPS 2020 Wor...

    2020

  5. [13]

    Lof: identifying density-based local outliers

    Markus M Breunig, Hans-Peter Kriegel, Ray- mond T Ng, and J¨ org Sander. Lof: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD , pages 93–104, 2000

    2000

  6. [14]

    Towards monosemanticity: Decomposing lan- guage models with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Bat- son, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, et al. Towards monosemanticity: Decomposing lan- guage models with dictionary learning. Trans- former Circuits Thread, 2, 2023

    2023

  7. [15]

    Deep learn- ing outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image clas- sification task

    Titus J Brinker, Achim Hekler, Alexander H Enk, Joachim Klode, Axel Hauschild, Carola Berking, Bastian Schilling, Sebastian Hafer- kamp, Dirk Schadendorf, et al. Deep learn- ing outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image clas- sificatio...

    2019

  8. [16]

    Detecting shortcut learning for fair medical ai using shortcut testing

    Alexander Brown, Nenad Tomasev, Jan Frey- berg, Yuan Liu, Alan Karthikesalingam, and Jessica Schrouff. Detecting shortcut learning for fair medical ai using shortcut testing. Nature communications, 14(1):4314, 2023

    2023

  9. [17]

    Dora: Exploring outlier representations in deep neural networks

    Kirill Bykov, Mayukh Deb, Dennis Grinwald, Klaus-Robert M¨ uller, and Marina MC H¨ ohne. Dora: Exploring outlier representations in deep neural networks. In ICLR Workshops, 2023

    2023

  10. [18]

    Labeling neural representations with inverse recognition

    Kirill Bykov, Laura Kopf, Shinichi Nakajima, Marius Kloft, and Marina H¨ ohne. Labeling neural representations with inverse recognition. NeurIPS, 36, 2024

    2024

  11. [19]

    Analysis of the isic image datasets: Usage, benchmarks and recommen- dations

    Bill Cassidy, Connah Kendrick, Andrzej Brodzicki, Joanna Jaworek-Korjakowska, and Moi Hoon Yap. Analysis of the isic image datasets: Usage, benchmarks and recommen- dations. Medical image analysis , 75:102305, 2022

    2022

  12. [20]

    Noel CF Codella, David Gutman, M Emre Celebi, Brian Helba, Michael A Marchetti, Stephen W Dusza, Aadi Kalloo, et al. Skin lesion analysis toward melanoma detection: A chal- lenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the interna- tional sk...

    2017

  13. [21]

    Bcn20000: Dermoscopic lesions in the wild, 2019

    Marc Combalia, Noel CF Codella, Veronica Rotemberg, Brian Helba, Veronica Vilaplana, Ofer Reiter, Cristina Carrera, et al. Bcn20000: Dermoscopic lesions in the wild, 2019

    2019

  14. [22]

    Concept activation regions: A generalized frame- work for concept-based explanations

    Jonathan Crabb´ e and Mihaela van der Schaar. Concept activation regions: A generalized frame- work for concept-based explanations. NeurIPS, 35:2590–2607, 2022

    2022

  15. [23]

    Visual-tcav: Concept-based attribution and saliency maps for post-hoc explainability in image classification

    Antonio De Santis, Riccardo Campi, Matteo Bianchi, and Marco Brambilla. Visual-tcav: Concept-based attribution and saliency maps for post-hoc explainability in image classification. arXiv preprint arXiv:2411.05698 , 2024

    2024 arXiv

  16. [24]

    Ai for radiographic covid-19 detection se- lects shortcuts over signal

    Alex J DeGrave, Joseph D Janizek, and Su-In Lee. Ai for radiographic covid-19 detection se- lects shortcuts over signal. Nat Mach Intell , 3 (7):610–619, 2021

    2021

  17. [25]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255. IEEE, 2009

    2009

  18. [26]

    Predicting parameters in deep learning

    Misha Denil, Babak Shakibi, Laurent Dinh, Marc’Aurelio Ranzato, and Nando De Fre- itas. Predicting parameters in deep learning. NeurIPS, 26, 2013

    2013

  19. [27]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, et al. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021

    2021

  20. [28]

    Understand- ing the (extra-) ordinary: Validating deep model decisions with prototypical concept-based expla- nations

    Maximilian Dreyer, Reduan Achtibat, Wojciech Samek, and Sebastian Lapuschkin. Understand- ing the (extra-) ordinary: Validating deep model decisions with prototypical concept-based expla- nations. In CVPRW, pages 3491–3501, 2024

    2024

  21. [29]

    From hope to safety: Unlearning bi- ases of deep models via gradient penalization in latent space

    Maximilian Dreyer, Frederik Pahde, Christo- pher J Anders, Wojciech Samek, and Sebastian Lapuschkin. From hope to safety: Unlearning bi- ases of deep models via gradient penalization in latent space. In AAAI, volume 38, pages 21046– 21054, 2024

    2024

  22. [30]

    Pure: Turning polysemantic neurons into pure features by identifying relevant cir- cuits

    Maximilian Dreyer, Erblina Purelku, Johanna Vielhaben, Wojciech Samek, and Sebastian La- puschkin. Pure: Turning polysemantic neurons into pure features by identifying relevant cir- cuits. In CVPRW, pages 8212–8217, 2024

    2024

  23. [31]

    Mech- anistic understanding and validation of large ai models with semanticlens

    Maximilian Dreyer, Jim Berend, Tobias Labarta, Johanna Vielhaben, Thomas Wiegand, Sebas- tian Lapuschkin, and Wojciech Samek. Mech- anistic understanding and validation of large ai models with semanticlens. arXiv preprint arXiv:2501.05398, 2025. 15

    2025 arXiv

  24. [32]

    Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022

    Nelson Elhage, Tristan Hume, Catherine Ols- son, Nicholas Schiefer, Tom Henighan, Shauna Kravec, et al. Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022

    2022 arXiv

  25. [33]

    Visualiz- ing higher-layer features of a deep network

    Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualiz- ing higher-layer features of a deep network. University of Montreal, 1341(3):1, 2009

    2009

  26. [34]

    Craft: Con- cept recursive activation factorization for ex- plainability

    Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, R´ emi Cad` ene, and Thomas Serre. Craft: Con- cept recursive activation factorization for ex- plainability. In CVPR, pages 2711–2721, 2023

    2023

  27. [35]

    Unlocking feature visu- alization for deep network with magnitude con- strained optimization

    Thomas Fel, Thibaut Boissin, Victor Boutin, Agustin Picard, Paul Novello, Julien Colin, Drew Linsley, Tom Rousseau, Remi Cadene, Lore Goetschalckx, et al. Unlocking feature visu- alization for deep network with magnitude con- strained optimization. NeurIPS, 36, 2024

    2024

  28. [36]

    A holis- tic approach to unifying automatic concept ex- traction and concept importance estimation

    Thomas Fel, Victor Boutin, Louis B´ ethune, R´ emi Cad` ene, Mazda Moayeri, L´ eo And´ eol, Mathieu Chalvidal, and Thomas Serre. A holis- tic approach to unifying automatic concept ex- traction and concept importance estimation. NeurIPS, 36, 2024

    2024

  29. [37]

    The use of multiple measure- ments in taxonomic problems

    Ronald A Fisher. The use of multiple measure- ments in taxonomic problems. Annals of eugen- ics, 7(2):179–188, 1936

    1936

  30. [38]

    Net2vec: Quan- tifying and explaining how concepts are encoded by filters in deep neural networks

    Ruth Fong and Andrea Vedaldi. Net2vec: Quan- tifying and explaining how concepts are encoded by filters in deep neural networks. In CVPR, pages 8730–8738, 2018

    2018

  31. [39]

    Shortcut learning in deep neural networks

    Robert Geirhos, J¨ orn-Henrik Jacobsen, Clau- dio Michaelis, Richard Zemel, Wieland Brendel, et al. Shortcut learning in deep neural networks. Nat Mach Intell , 2(11):665–673, 2020

    2020

  32. [40]

    Towards automatic concept- based explanations

    Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept- based explanations. NeurIPS, 32, 2019

    2019

  33. [41]

    Concept discovery and dataset exploration with singular value decomposition

    Mara Graziani, An-phi Nguyen, Laura O’Mahony, Henning M¨ uller, and Vincent Andrearczyk. Concept discovery and dataset exploration with singular value decomposition. In ICLR Workshops, 2023

    2023

  34. [42]

    Deep residual learning for image recog- nition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770–778, 2016

    2016

  35. [43]

    Bag of tricks for image classification with convolutional neural networks

    Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. Bag of tricks for image classification with convolutional neural networks. In CVPR, pages 558–567, 2019

    2019

  36. [44]

    Natural language descriptions of deep visual features

    Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Natural language descriptions of deep visual features. In ICLR, 2021

    2021

  37. [45]

    Sparse autoencoders find highly interpretable features in language models

    Robert Huben, Hoagy Cunningham, Lo- gan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In ICLR, 2023

    2023

  38. [46]

    Chexpert: A large chest radio- graph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, et al. Chexpert: A large chest radio- graph dataset with uncertainty labels and expert comparison. In AAAI, 2019

    2019

  39. [47]

    Interpretability beyond feature attribu- tion: Quantitative testing with concept activa- tion vectors (tcav)

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribu- tion: Quantitative testing with concept activa- tion vectors (tcav). In ICML, pages 2668–2677. PMLR, 2018

    2018

  40. [48]

    Unmasking clever hans predictors and assessing what ma- chines really learn

    Sebastian Lapuschkin, Stephan W¨ aldchen, Alexander Binder, Gr´ egoire Montavon, Wojciech Samek, and Klaus-Robert M¨ uller. Unmasking clever hans predictors and assessing what ma- chines really learn. Nature Communications, 10 (1):1096, 2019

    2019

  41. [49]

    Umap: Uniform man- ifold approximation and projection

    Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. Umap: Uniform man- ifold approximation and projection. Journal of Open Source Software, 3(29), 2018

    2018

  42. [50]

    Evaluating the stability of semantic concept representations in cnns for robust explainability

    Georgii Mikriukov, Gesina Schwalbe, Christian Hellert, and Korinna Bade. Evaluating the stability of semantic concept representations in cnns for robust explainability. In World Confer- ence on Explainable Artificial Intelligence, pages 499–524. Springer, 2023

    2023

  43. [51]

    Visualization of neu- ral networks using saliency maps

    Niels JS Morch, Ulrik Kjems, Lars Kai Hansen, Claus Svarer, Ian Law, Benny Lautrup, Steve Strother, and Kelly Rehm. Visualization of neu- ral networks using saliency maps. In ICNN, vol- ume 4, pages 2085–2090. IEEE, 1995

    1995

  44. [52]

    Spurious fea- tures everywhere-large-scale detection of harm- ful spurious features in imagenet

    Yannic Neuhaus, Maximilian Augustin, Valen- tyn Boreiko, and Matthias Hein. Spurious fea- tures everywhere-large-scale detection of harm- ful spurious features in imagenet. InICCV, 2023. 16

    2023

  45. [53]

    Clip- dissect: Automatic description of neuron repre- sentations in deep vision networks

    Tuomas Oikarinen and Tsui-Wei Weng. Clip- dissect: Automatic description of neuron repre- sentations in deep vision networks. In ICLR, 2023

    2023

  46. [54]

    Feature visualization

    Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11): e7, 2017

    2017

  47. [55]

    Zoom in: An introduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5 (3):e00024–001, 2020

    2020

  48. [56]

    A threshold selection method from gray-level histograms

    Nobuyuki Otsu et al. A threshold selection method from gray-level histograms. Automatica, 11(285-296):23–27, 1975

    1975

  49. [57]

    Reveal to re- vise: An explainable ai life cycle for iterative bias correction of deep models

    Frederik Pahde, Maximilian Dreyer, Wojciech Samek, and Sebastian Lapuschkin. Reveal to re- vise: An explainable ai life cycle for iterative bias correction of deep models. In MICCAI, 2023

    2023

  50. [58]

    Navigating neural space: Revisiting concept activation vectors to over- come directional divergence

    Frederik Pahde, Maximilian Dreyer, Leander Weber, Moritz Weckbecker, Christopher J An- ders, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. Navigating neural space: Revisiting concept activation vectors to over- come directional divergence. In International Conferen...

    2025

  51. [59]

    Py- torch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, et al. Py- torch: An imperative style, high-performance deep learning library. NeurIPS, 32, 2019

    2019

  52. [60]

    Interpretable data-based expla- nations for fairness debugging

    Romila Pradhan, Jiongli Zhu, Boris Glavic, and Babak Salimi. Interpretable data-based expla- nations for fairness debugging. In Proceedings of the 2022 international conference on manage- ment of data , pages 247–261, 2022

    2022

  53. [61]

    Learning to generate reviews and discovering sentiment

    Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444, 2017

    2017 arXiv

  54. [62]

    Interpretations are useful: penalizing explanations to align neural networks with prior knowledge

    Laura Rieger, Chandan Singh, William Mur- doch, and Bin Yu. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. In ICML, 2020

    2020

  55. [63]

    Right for the right reasons: training differentiable models by constraining their explanations

    Andrew Slavin Ross, Michael C Hughes, and Fi- nale Doshi-Velez. Right for the right reasons: training differentiable models by constraining their explanations. In IJCAI, 2017

    2017

  56. [64]

    Making deep neural networks right for the right scientific rea- sons by interacting with their explanations

    Patrick Schramowski, Wolfgang Stammer, Ste- fano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. Making deep neural networks right for the right scientific rea- sons by interacting with their explanations. Nat...

    2020

  57. [65]

    Grad-cam: Visual explanations from deep networks via gradient- based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. In ICCV, pages 618–626, 2017

    2017

  58. [66]

    Very deep convolutional networks for large-scale im- age recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. In Yoshua Bengio and Yann LeCun, editors, ICLR 2015, 2015

    2015

  59. [67]

    Salient imagenet: How to discover spurious features in deep learning

    S Singla and S Feizi. Salient imagenet: How to discover spurious features in deep learning. In ICLR, 2022

    2022

  60. [68]

    Explaining ma- chine learning models for clinical gait analysis

    Djordje Slijepcevic, Fabian Horst, Sebastian La- puschkin, Brian Horsak, Anna-Maria Raberger, Andreas Kranzl, Wojciech Samek, Christian Breiteneder, Wolfgang Immanuel Sch¨ ollhorn, and Matthias Zeppelzauer. Explaining ma- chine learning models for clinical gait analysis. ACM T...

    2021

  61. [69]

    Deep learning for ecg analysis: Benchmarks and insights from ptb- xl

    Nils Strodthoff, Patrick Wagner, Tobias Scha- effter, and Wojciech Samek. Deep learning for ecg analysis: Benchmarks and insights from ptb- xl. IEEE journal of biomedical and health infor- matics, 25(5):1519–1528, 2020

    2020

  62. [70]

    Intriguing prop- erties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing prop- erties of neural networks. In ICLR, 2014

    2014

  63. [71]

    The ham10000 dataset, a large collection of multi-source dermatoscopic images of com- mon pigmented skin lesions

    Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multi-source dermatoscopic images of com- mon pigmented skin lesions. Scientific data , 5 (1):1–9, 2018

    2018

  64. [72]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 9(11), 2008

    2008

  65. [73]

    Multi-dimensional concept discovery (mcd): A unifying framework with completeness guarantees

    Johanna Vielhaben, Stefan Bluecher, and Nils Strodthoff. Multi-dimensional concept discovery (mcd): A unifying framework with completeness guarantees. TMLR, 2023. 17

    2023

  66. [74]

    Ptb- xl, a large publicly available electrocardiography dataset

    Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima I Lunze, Wojciech Samek, and Tobias Schaeffter. Ptb- xl, a large publicly available electrocardiography dataset. Scientific data, 7(1):1–15, 2020

    2020

  67. [75]

    Explaining deep learning for ecg analysis: Building blocks for auditing and knowledge discovery

    Patrick Wagner, Temesgen Mehari, Wilhelm Haverkamp, and Nils Strodthoff. Explaining deep learning for ecg analysis: Building blocks for auditing and knowledge discovery. Comput- ers in Biology and Medicine , 176:108525, 2024

    2024

  68. [76]

    Fast diffusion-based counterfactuals for shortcut re- moval and generation

    Nina Weng, Paraskevas Pegios, Eike Petersen, Aasa Feragen, and Siavash Bigdeli. Fast diffusion-based counterfactuals for shortcut re- moval and generation. In ECCV, 2025

    2025

  69. [77]

    Pytorch image mod- els

    Ross Wightman. Pytorch image mod- els. https://github.com/rwightman/ pytorch-image-models, 2019

    2019

  70. [78]

    Discover and cure: Concept- aware mitigation of spurious correlation

    Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: Concept- aware mitigation of spurious correlation. In ICML, 2023

    2023

  71. [79]

    Variable generalization performance of a deep learning model to de- tect pneumonia in chest radiographs: a cross- sectional study

    John R Zech, Marcus A Badgeley, Manway Liu, Anthony B Costa, Joseph J Titano, and Eric Karl Oermann. Variable generalization performance of a deep learning model to de- tect pneumonia in chest radiographs: a cross- sectional study. PLoS medicine, 15(11), 2018

    2018

  72. [80]

    Invertible concept-based explanations for cnn models with non-negative concept activa- tion vectors

    Ruihan Zhang, Prashan Madumal, Tim Miller, Krista A Ehinger, and Benjamin IP Rubin- stein. Invertible concept-based explanations for cnn models with non-negative concept activa- tion vectors. In AAAI, volume 35, pages 11682– 11690, 2021

    2021

  73. [81]

    right-reason

    Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, pages 2921–2929, 2016. 18 A Appendix In the following, we will provide an algorithmic overview of the extended Reveal2Revise framework...

    2016

  74. [82]

    Slic superpixels compared to state- of-the-art superpixel methods

    Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine S¨ usstrunk. Slic superpixels compared to state- of-the-art superpixel methods. IEEE TPAMI , 34(11):2274–2282, 2012

    2012

  75. [83]

    Support- vector networks

    Corinna Cortes and Vladimir Vapnik. Support- vector networks. Machine learning, 20:273–297, 1995

    1995

  76. [84]

    A uni- fied approach to interpreting model predictions

    Scott M Lundberg and Su-In Lee. A uni- fied approach to interpreting model predictions. NeurIPS, 30, 2017

    2017

  77. [85]

    Beyond word importance: Contextual decompo- sition to extract interactions from lstms

    W James Murdoch, Peter J Liu, and Bin Yu. Beyond word importance: Contextual decompo- sition to extract interactions from lstms. ICLR, 2018

    2018

  78. [86]

    Null it out: Guarding protected attributes by iterative nullspace projection

    Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, ed- itors, ACL, pages 7237–7256, July 2020

    2020

  79. [87]

    Editing a classifier by rewriting its prediction rules

    Shibani Santurkar, Dimitris Tsipras, Mahalaxmi Elango, David Bau, Antonio Torralba, and Alek- sander Madry. Editing a classifier by rewriting its prediction rules. NeurIPS, 34:23359–23373, 2021. 44

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.