Pith. sign in

REVIEW 2 major objections 5 minor 49 references

XAI-Units: Benchmarking Explainability Methods with Unit Tests

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A benchmark of handcrafted synthetic models with known mechanisms provides ground-truth attributions that let feature attribution methods be compared objectively, and it catches a DeepLIFT implementation bug in Captum.

desk verdict Useful benchmark with a real Captum find, but the Pertinent Negatives ground truth contradicts its own stated goal and needs fixing. read the letter →

arxiv 2506.01059 v1 pith:523LYYI7 submitted 2025-06-01 cs.LG cs.AI

Jun Rui Lee , Sadegh Emami , Michael David Hollins , Timothy C. H. Wong , Carlos Ignacio Villalobos Sánchez , Francesca Toni , Dekai Zhang , Adam Dejl This is my paper
classification cs.LGcs.AI
keywords featureattributionexplainableAIbenchmarkground-truthattributionssyntheticdatasetsunittestingDeepLIFTCaptum
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that feature attribution methods can be compared objectively when the model being explained is handcrafted, deterministic, and perfectly aligned with a synthetic dataset. It introduces XAI-Units, an open-source benchmark where each dataset-model pair acts as a unit test for one atomic behaviour—weighted inputs, feature conflicts, cancellations, abrupt discontinuities, categorical interactions, irrelevant features, Boolean logic, image foreground/background structure, and trigger tokens in text—with ground-truth attribution scores defined in advance by ablating inputs to a baseline reference. Running common FA methods through the benchmark shows where each class of method fails, and it exposes a concrete implementation discrepancy in the popular Captum library's version of DeepLIFT. The value of the work is that it turns the disagreement problem among FA methods into testable claims about specific model behaviours rather than an unresolved philosophical debate.

What carries the argument

The load-bearing mechanism is the dataset-model unit test: each synthetic data generator is paired with a handcrafted neural network whose internal computation is given as an explicit formula and diagram in Appendix A. Ground truth is produced by a single rule—ablate an input to its baseline reference (almost always 0) and measure the change in output, with per-model formalizations for continuous, categorical, and Boolean features—and where exact attributions are undefined, the benchmark substitutes masks (image foreground, text trigger tokens, relevant features). Built-in evaluation metrics translate this ground truth into a score, which is what lets the benchmark attribute a poor score to the FA method rather than to unknown model behaviour.

What would settle it

For a Pertinent Negatives sample with a pertinent-negative feature at $x_i = 0$, the benchmark's ground truth is $FA_{x_i} = M(x) - M(x^{-i}) = 0$ by ablation to 0, so any FA method that returns a non-zero attribution for that feature—exactly what Section 3.1 says a good method should do—is scored as wrong; a single run showing no method can be both perfect and non-zero on that feature would settle whether the benchmark tests its own stated property.

Watch

Extended reading notes

Core claim

The central claim is that with fully known model mechanisms, ground truth for explanations exists and can be scored. For each synthetic dataset the authors handcraft a neural network whose forward logic is specified exactly; ground-truth attribution for a feature is defined as the change in output when that feature is ablated to a baseline reference value, usually 0, and the benchmark's default metrics (MSE, Mask Error, Mask Proportion, SensitivityMax, Infidelity) compare FA output against those expectations. On the seven tabular unit tests, gradient-based methods fail on zero-gradient conflicts and shattered gradients, linear-surrogate methods (LIME, KernelSHAP) fail on nonlinearity, and the Uncertainty model reveals that Captum's DeepLIFT normalizes at the wrong stage, yielding a high Mask Error that drops to exactly 0 when the normalization is applied to logits as the DeepLIFT paper prescribes.

Load-bearing premise

The ground-truth scores assume that a feature's importance equals how much the model output changes when that feature is replaced by a baseline value (usually 0), an assumption that in the Pertinent Negatives dataset assigns zero importance to a zero-valued feature even though the paper says such features should ideally receive non-zero attribution.

Editorial extensions

If this is right

  • The benchmark can verify whether an FA method meets its design specification, as demonstrated by the Captum DeepLIFT normalization bug.
  • Practitioners can identify, for any FA method, the atomic behaviour that causes it to fail, such as zero gradients in Conflicting Features or discontinuity in Shattered Gradients.
  • Handcrafted models give cleaner ground truth than trained models, since imperfect training can misalign an otherwise correct explanation with the data distribution.
  • The benchmark is extensible to custom FA methods, metrics, and new unit tests, covering tabular, image, and text/LLM modalities.
  • Relative rankings of FA methods on two LLM variants were consistent, suggesting the unit-test signal is stable across models of the same task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ablation-to-baseline ground truth encodes one specific semantics of importance—change in output—so methods built on Shapley values, counterfactuals, or necessary-and-sufficient conditions may score poorly without being 'wrong'; the benchmark ranks methods against this single semantics.
  • The Pertinent Negatives test appears internally inconsistent: zero-valued features expected to receive non-zero attributions are assigned zero by the benchmark's own ground truth, so perfect scores on that test may not reward the intended behaviour.
  • The DeepLIFT/Captum finding implies that published FA comparisons can be confounded by implementation details rather than method design, and the same unit-test procedure could audit other libraries for similar discrepancies.
  • Extending the unit-test idea to semi-synthetic data, where some real inputs enter an otherwise handcrafted model, could keep partial ground truth while gaining realism, though exact attribution would have to be relaxed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces XAI-Units, an open-source Python benchmark for evaluating feature attribution (FA) methods. It provides synthetic datasets paired with handcrafted neural network models whose internal mechanisms are known, defines ground-truth attributions (mostly by ablating inputs to a zero baseline), and evaluates several popular FA methods from Captum on tabular, image, and text datasets. The main claims are that the benchmark enables objective FA evaluation and that it uncovered a DeepLIFT normalization discrepancy in Captum. The paper also includes trained models for comparison and discusses limitations of synthetic evaluation.

Significance. If the ground-truth definitions were internally consistent, the benchmark would be a useful contribution to the XAI community: it is open-source, extensible, and provides atomic 'unit tests' for specific model behaviours. The identification of a possible DeepLIFT normalization bug in Captum is a concrete, reproducible finding that does not depend on the contested ground-truth choices. The authors are transparent about the benchmark's scope (synthetic, technical evaluation) and explicitly disclaim claims about real-world performance or human interpretability. However, the benchmark's reliability as a ground-truth standard is compromised by the Pertinent Negatives self-contradiction described below.

major comments (2)
  1. [Section 3.1 and Appendix A.4, Table 3] The Pertinent Negatives ground truth contradicts the stated intent. Section 3.1 says that for a pertinent negative feature with a meaningful zero value, FA methods 'should ideally return non-zero attributions.' However, Appendix A.4 defines the ground truth as ablation to the baseline x_ref=0, F_A_xi(x)=M(x)-M(x_-i). For a pertinent negative feature with value 0, x_-i equals x, so the attribution is identically zero. Consequently, methods that return zero for that feature receive a perfect MSE score, and methods that return the non-zero score the text says is ideal are penalized. Table 3 confirms this: DeepLIFT, IntegratedGradients, KernelSHAP, ShapleyValueSampling, and LIME(Linear) all report 0.000 MSE on the handcrafted Pertinent Negatives model, which is an artifact of the contradictory ground truth rather than evidence of satisfying the intended property. The dataset must be redesigned (e.g., using a non-zero baseline for pertinent negatives, or a mask-based expectation) before it can serve as a ground-truth standard.
  2. [Appendix A.6 (and A.3)] The ground truth for the Conflicting Features and Categorical Feature Interaction datasets uses a sequential ablation order without justification. For example, in Appendix A.6, F_A_xi is computed while holding the categorical feature at its observed value, while F_A_ci is computed after the continuous feature has been ablated to baseline. Reversing the order would produce different attributions. Since the paper presents these as objective ground truths, it should either justify the chosen ordering (e.g., as reflecting a specific causal or computational graph semantics) or explicitly acknowledge that the ground truth is one of several possible decompositions and discuss how this affects the interpretation of benchmark scores.
minor comments (5)
  1. [Section 4.1] The sentence 'All FA methods performed well on the simplest test case, the Weighted Continuous models, but struggled on models with gradient discontinuities, such as those for Shattered Gradients and Pertinent Negatives' is not supported by Table 3 for the handcrafted Pertinent Negatives model, where five of seven methods achieve 0.000 MSE; after the ground truth is corrected, this sentence should be revisited.
  2. [Table 2] The table lists a single 'Boolean Formula' row but Section 3.1 describes generic Boolean formulas plus separate AND/OR entries; clarify how these map to the table's rows.
  3. [Section 5] The claim of being 'the first within the research community' to provide an end-to-end pipeline with synthetic datasets and handcrafted models is too strong given the prior existence of XAI-Bench and AttributionLab, which also use synthetic ground truths; the novelty claim should be qualified.
  4. [Appendix A.8] In the Boolean AND/OR ground-truth formula, the notation b- is used in the denominator before it is defined in the following line; reorder the presentation for clarity.
  5. [Section 3.1, Pertinent Negatives paragraph] The heart-rate example describes a continuous 0 value, but the model assumes pertinent negative features are categorical with values 0 or 1; clarify whether the benchmark's pertinent negative features are indeed binary in the generated data.

Circularity Check

1 steps flagged · score 6.0 of 10

Pertinent Negatives ground truth (Appendix A.4) forces zero attribution for the exact zero-valued features that Section 3.1 says should receive non-zero attributions, making the perfect MSE scores on this unit test an artifact of the definition.

  1. self definitional [Section 3.1 (Pertinent Negatives) and Appendix A.4, ground-truth definition F_A_xi(x)=M(x)-M(x_-i)]
    "Section 3.1: 'As the 0 heart rate is meaningful for the prediction (indicating asystole), FA methods should ideally return non-zero attributions for this feature.' Appendix A.4: 'The ground truth feature attribution is defined by ablating to a baseline reference x_ref = 0. F_A_xi(x) = M(x) - M(x_-i)' with x_-i setting x_i to 0 (A.2)."

    For a pertinent-negative feature with value 0, x_-i = x, so F_A_xi(x) = M(x) - M(x) = 0 by construction, regardless of the multiplier m that makes the zero value behaviorally meaningful. The ground truth therefore assigns zero attribution to the very feature the dataset is intended to test (non-zero attribution for zero-valued pertinent negatives). An FA method that returns 0 for this feature receives perfect MSE, while a method returning the non-zero score the text calls ideal is penalized. Table 3's all-0.000 MSE scores on the handcrafted Pertinent Negatives model are forced by this definition, not evidence that the methods satisfy the intended property. The benchmark's ground truth for this unit test is self-definitional and cannot measure what it claims to measure.

full rationale

The only load-bearing circularity found is in the Pertinent Negatives unit test. The paper's stated intent (Section 3.1) is that FA methods should return non-zero attributions for features whose zero value is meaningful (e.g., heart rate 0 = asystole). However, the ground truth in Appendix A.4 is defined by ablating to baseline 0, which for a feature already at 0 leaves the input unchanged, forcing the attribution to be exactly 0 by the paper's own equation F_A_xi(x)=M(x)-M(x_-i). Thus the dataset rewards methods that return zero and punishes methods that follow the stated ideal, making the perfect MSE scores in Table 3 an artifact of the definition rather than a meaningful evaluation. This is a self-definitional circularity in one of the seven tabular unit tests. The other components of the paper are not circular: the Captum DeepLIFT normalization discrepancy (Section 4.1) is an independent implementation check against the DeepLIFT paper's recommendations; the remaining ground truths (Weighted Continuous, Conflicting Features, Feature Interaction, Boolean AND/OR) are transparently defined by ablation and internally consistent with their stated model mechanisms; the image and text masks are derived from model behavior (foreground objects, trigger tokens). The paper's self-citations (Dejl et al. 2025) are limited to related-work context and are not load-bearing. Hence the circularity is partial: the central claim of reliable ground truth is undermined for one unit test, but the benchmark largely stands on independent definitions and external checks.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The benchmark rests on the axiomatic definition of attribution via ablation to a baseline. No free parameters are fitted to data to obtain the central results; the Captum bug finding is independent of these axioms.

free parameters (1)
  • Ablation baseline reference = 0
    All ground-truth attributions are defined relative to a baseline input, set to zero for most datasets. This choice is a definition, not fitted to data, but it determines all expected scores.
assumptions (3)
  • domain assumption Ground-truth attribution is defined by ablating each feature to the baseline and taking the output difference
    This is the central definition used to generate expected attributions (Appendix A.2-A.8). It is a choice about what 'correct' attribution means, not a derived result.
  • standard math Softmax outputs are invariant to adding a constant to all logits, so common features that shift all logits equally are irrelevant
    Used in the Uncertainty Model (Appendix A.7) to justify excluding common features from the ground-truth mask.
  • ad hoc to paper For Boolean AND/OR, importance is allocated in proportion to the number of flipped truth values that would change the output
    Defined in Appendix A.8; this allocation is specific to this benchmark and not a universally agreed notion of Boolean feature importance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of XAI-Units: Benchmarking Explainability Methods with Unit Tests." pith.science (2026). https://pith.science/paper/523LYYI7

@misc{pith2026250601059,
  author       = {Pith},
  title        = {Pith review of: XAI-Units: Benchmarking Explainability Methods with Unit Tests},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/523LYYI7}},
  note         = {Machine review of arXiv:2506.01059}
}
read the original abstract

Feature attribution (FA) methods are widely used in explainable AI (XAI) to help users understand how the inputs of a machine learning model contribute to its outputs. However, different FA models often provide disagreeing importance scores for the same model. In the absence of ground truth or in-depth knowledge about the inner workings of the model, it is often difficult to meaningfully determine which of the different FA methods produce more suitable explanations in different contexts. As a step towards addressing this issue, we introduce the open-source XAI-Units benchmark, specifically designed to evaluate FA methods against diverse types of model behaviours, such as feature interactions, cancellations, and discontinuous outputs. Our benchmark provides a set of paired datasets and models with known internal mechanisms, establishing clear expectations for desirable attribution scores. Accompanied by a suite of built-in evaluation metrics, XAI-Units streamlines systematic experimentation and reveals how FA methods perform against distinct, atomic kinds of model reasoning, similar to unit tests in software engineering. Crucially, by using procedurally generated models tied to synthetic datasets, we pave the way towards an objective and reliable comparison of FA methods.

Figures

Figures reproduced from arXiv: 2506.01059 by the authors.

Figure 1
Figure 1. Overview of the XAI-Units benchmark. The benchmark provides a set of datasets and models with controlled mechanisms and behaviour. This enables us to evaluate the attributions produced by various FA methods using various metrics, often taking into account the ground-truth expectations associated with the given dataset and model. refer to complementary efforts in prior work outlined towards the end of Section 2. More… view at source ↗
Figure 1
Figure 1. We focus on the datasets and models included in the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Variations of a foreground-background combination from the BalancedImageDataset. that closely mimic real-world conditions where objects of interest (foregrounds) appear against varying scenes (backgrounds). A good FA method applied to a well-trained model might assign higher at￾tribute scores to pixels in the foreground rather than the irrelevant background. For a given sample (combining a foreground and background)… view at source ↗
Figures from the paper (8 more)
Figure 3
Figure 3. Figure 3: Legend for the model diagrams in Appendix A. The diagrams illustrate one instance of each model but note that the number of input features can be adjusted. A.1. Baselines We first briefly introduce the used attribution baselines. Most FA methods calculate attribution s…
Figure 4
Figure 4. Figure 4: Weighted Continuous formula and model diagram. For a feature vector x ∈ R n and a given weight matrix W ∈ R 1×n , where n ∈ N is the number of features, the output y of the model. The default evaluation metric is MSE, measuring the difference from the ground truth attr…
Figure 5
Figure 5. Figure 5: Conflicting Features formula and model diagram. For a continuous feature xi and a (categorical) cancellation feature ci, together (xi, ci) contribute to output y. The ground truth feature attribution is defined by ablating to a baseline reference (xref , cref ) = (0, 0…
Figure 6
Figure 6. Figure 6: Pertinent Negative formula and model diagram. Pi denotes the set of indices of all pertinent negative features. For simplicity, we assume that pertinent negative features are categorical with values 0 or 1. When the pertinent negative feature xi takes a value of 0, the…
Figure 7
Figure 7. Figure 7: Shattered Gradients formula and model diagram. Ground truth feature attributions are not available for the Shattered Gradients model. We use SensitivityMax as the default evaluation metric. A.6. Categorical Feature Interaction y = Xn i=1 zi zi = ( wixi if xi is non-int…
Figure 8
Figure 8. Figure 8: Categorical Feature Interaction formula and model diagram. The user can define some features to be non-interacting and other features to have an interaction. For an interacting pair (xi, ci) then ci is categorical with value 0 or 1. If ci is 0, the weight applied to th…
Figure 9
Figure 9. Figure 9: Uncertainty Model formula and model diagram. The Uncertainty dataset is intended for classification problems rather than regression problems. For this dataset (like the image dataset) we provide the ground truth as a mask rather than exact feature attributions. The mas…
Figure 10
Figure 10. Figure 10: Boolean Formulae and model diagrams. For propositional atoms p, q, the mathematical and diagrammatic representation of them are illustrated in [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 46 canonical work pages

  1. [1]

    Agarwal, C., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Puri, I., Zitnik, M., and Lakkaraju, H. (2022). OpenXAI : Towards a transparent evaluation of model explanations. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A., editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informatio...

    work page 2022

  2. [2]

    Agarwal, C., Queen, O., Lakkaraju, H., and Zitnik, M. (2023). Evaluating explainability for graph neural networks. Scientific Data , 10(144)

    work page 2023

  3. [3]

    and Jaakkola, T

    Alvarez - Melis, D. and Jaakkola, T. S. (2018). On the robustness of interpretability methods

    work page 2018

  4. [4]

    Ancona, M., Ceolini, E., \" O ztireli, C., and Gross, M. (2018). Towards better understanding of gradient-based attribution methods for deep neural networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net

    work page 2018

  5. [5]

    Arras, L., Osman, A., and Samek, W. (2022). CLEVR-XAI : A benchmark dataset for the ground truth evaluation of neural network explanations. Information Fusion , 81:14--40

    work page 2022

  6. [6]

    Bach, S., Binder, A., Montavon, G., Klauschen, F., M \"u ller, K.-R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one , 10(7):e0130140

    work page 2015

  7. [7]

    W., and Kim, B

    Bilodeau, B., Jaques, N., Koh, P. W., and Kim, B. (2024). Impossibility theorems for feature attribution. Proceedings of the National Academy of Sciences , 121(2)

    work page 2024

  8. [8]

    Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science , 16(3):199--231

    work page 2001

Show all 49 references
  1. [9]

    Castro, J., G \'o mez, D., and Tejada, J. (2009). Polynomial calculation of the shapley value based on sampling. Computers & Operations Research , 36(5):1726--1730

    2009

  2. [10]

    Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. (2014). Describing textures in the wild. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014 , pages 3606--3613. IEEE Computer Society

    2014

  3. [11]

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. (2021). Training verifiers to solve math word problems. CoRR , abs/2110.14168

    2021 arXiv

  4. [12]

    Cui, Y., Liu, T., Che, W., Chen, Z., and Wang, S. (2022). Expmrc: Explainability evaluation for machine reading comprehension. Heliyon , 8:e09290

    2022

  5. [13]

    and Gal, Y

    Dabkowski, P. and Gal, Y. (2017). Real time image saliency for black box classifiers. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R., editors, Advances in Neural Information Processing Systems 30: Annual Conferenc...

    2017

  6. [14]

    Dejl, A., Zhang, D., Ayoobi, H., Williams, M., and Toni, F. (2025). Hidden conflicts in neural networks and their implications for explainability. In FAccT '25: The 2025 ACM Conference on Fairness, Accountability, and Transparency Proceedings , New York, NY, USA. Association f...

    2025

  7. [15]

    Ding, N., Chen, Y., Xu, B., Qin, Y., Hu, S., Liu, Z., Sun, M., and Zhou, B. (2023). Enhancing chat language models by scaling high-quality instructional conversations. In Bouamor, H., Pino, J., and Bali, K., editors, Proceedings of the 2023 Conference on Empirical Methods in N...

    2023

  8. [16]

    Fresz, B., L\" o rcher, L., and Huber, M. (2024). Classification metrics for image explanations: Towards building reliable xai-evaluations. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency , FAccT '24, page 1–19, New York, NY, USA. Associ...

    2024

  9. [17]

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., et al. (2024). The L lama 3 herd of models

    2024

  10. [18]

    Guidotti, R. (2021). Evaluating local explanation methods on ground truth. Artificial Intelligence , 291:103428

    2021

  11. [19]

    o m, A., Weber, L., Krakowczyk, D., Bareeva, D., Motzkus, F., Samek, W., Lapuschkin, S., and H \

    Hedstr \" o m, A., Weber, L., Krakowczyk, D., Bareeva, D., Motzkus, F., Samek, W., Lapuschkin, S., and H \" o hne, M. M. (2023). Quantus: An explainable AI toolkit for responsible evaluation of neural network explanations and beyond

    2023

  12. [20]

    and Rahnama, A

    Hossein, A. and Rahnama, A. (2024). The blame problem in evaluating local explanations and how to tackle it. In Artificial Intelligence. ECAI 2023 International Workshops , pages 66--86, Cham. Springer Nature Switzerland

    2024

  13. [21]

    Huang, W., Zhao, X., Jin, G., and Huang, X. (2023). SAFARI : Versatile and efficient evaluations for robustness of interpretability. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023 , pages 1988--1998. IEEE

    2023

  14. [22]

    Kim, J., Maathuis, H., and Sent, D. (2024). Human-centered evaluation of explainable ai applications: A systematic review. Frontiers in Artificial Intelligence , Volume 7 - 2024

    2024

  15. [23]

    Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., and Reblitz - Richardson, O. (2020). Captum: A unified and generic model interpretability library for PyTorch

    2020

  16. [24]

    Krishna, S., Han, T., Gu, A., Wu, S., Jabbari, S., and Lakkaraju, H. (2024). The disagreement problem in explainable machine learning: A practitioner's perspective. Trans. Mach. Learn. Res. , 2024

    2024

  17. [25]

    Q., Nauta, M., Nguyen, V

    Le, P. Q., Nauta, M., Nguyen, V. B., Pathak, S., Schl \" o tterer, J., and Seifert, C. (2023). Benchmarking explainable AI - A survey on available toolkits and open challenges. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI...

    2023

  18. [26]

    Li, X., Du, M., Chen, J., Chai, Y., Lakkaraju, H., and Xiong, H. (2023). M ^4 : A unified XAI benchmark for faithfulness evaluation of feature attribution methods across metrics, modalities and models. Advances in Neural Information Processing Systems , 36:1630--1643

    2023

  19. [27]

    Lin, Y., Lee, W., and Celik, Z. B. (2021). What do you see?: Evaluation of explainable artificial intelligence (XAI) interpretability through neural backdoors. In Zhu, F., Ooi, B. C., and Miao, C., editors, KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Dat...

    2021

  20. [28]

    Liu, Y., Khandagale, S., White, C., and Neiswanger, W. (2021). Synthetic benchmarks for scientific research in explainable machine learning. In Vanschoren, J. and Yeung, S., editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, N...

    2021

  21. [29]

    Lundberg, S. M. and Lee, S. (2017). A unified approach to interpreting model predictions. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R., editors, Advances in Neural Information Processing Systems 30: Annual Confe...

    2017

  22. [30]

    Mamalakis, A., Ebert-Uphoff, I., and Barnes, E. A. (2022). Neural network attribution methods for problems in geoscience: A novel synthetic benchmark dataset. Environmental Data Science , 1:e8

    2022

  23. [31]

    Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., and Seifert, C. (2023). From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai. ACM Computing Surveys , 55(13s):1–42

    2023

  24. [32]

    Ramaswamy, H. G. et al. (2020). Ablation-CAM : Visual explanations for deep convolutional network via gradient-free localization. In proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 983--991

    2020

  25. [33]

    Why should I trust you?

    Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). "Why should I trust you?": Explaining the predictions of any classifier. In Krishnapuram, B., Shah, M., Smola, A. J., Aggarwal, C. C., Shen, D., and Rastogi, R., editors, Proceedings of the 22nd ACM SIGKDD International Confe...

    2016

  26. [34]

    Roy, S., Laberge, G., Roy, B., Khomh, F., Nikanjam, A., and Mondal, S. (2022). Why don’t XAI techniques agree? Characterizing the disagreements between post-hoc explanations of defect predictions. In 2022 IEEE International Conference on Software Maintenance and Evolution (ICS...

    2022

  27. [35]

    Saha, A., Subramanya, A., and Pirsiavash, H. (2020). Hidden trigger backdoor attacks. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposiu...

    2020

  28. [36]

    and Ren, X

    Sanyal, S. and Ren, X. (2021). Discretized integrated gradients for explaining language models. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t., editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 10285--10299, Onli...

    2021

  29. [37]

    Shrikumar, A., Greenside, P., and Kundaje, A. (2017). Learning important features through propagating activation differences. In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 Au...

    2017

  30. [38]

    Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. (2016). Not just a black box: Learning important features through propagating activation differences

    2016

  31. [39]

    Speith, T. (2022). A review of taxonomies of explainable artificial intelligence (XAI) methods. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , FAccT '22, page 2239–2250, New York, NY, USA. Association for Computing Machinery

    2022

  32. [40]

    Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , volume 70 of Proceedings ...

    2017

  33. [41]

    Category:Dinosaurs with transparent background

    Wikimedia Commons (2024). Category:Dinosaurs with transparent background. Online; accessed 2-June-2024

    2024

  34. [42]

    Xu, Y., Zhang, Q., Zhang, J., and Tao, D. (2021). ViTAE : Vision transformer advanced by exploring intrinsic inductive bias. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W., editors, Advances in Neural Information Processing Systems 34: Annual Co...

    2021

  35. [43]

    Yan, J., Yadav, V., Li, S., Chen, L., Tang, Z., Wang, H., Srinivasan, V., Ren, X., and Jin, H. (2024). Backdooring instruction-tuned large language models with virtual prompt injection. In Duh, K., Gomez, H., and Bethard, S., editors, Proceedings of the 2024 Conference of the ...

    2024

  36. [44]

    and Kim, B

    Yang, M. and Kim, B. (2019). Benchmarking attribution methods with relative feature importance. CoRR , abs/1907.09701

    2019 arXiv

  37. [45]

    S., Inouye, D

    Yeh, C.-K., Hsieh, C.-Y., Suggala, A. S., Inouye, D. I., and Ravikumar, P. (2019). On the (in)fidelity and sensitivity of explanations. In Proceedings of the 33rd International Conference on Neural Information Processing Systems , Red Hook, NY, USA. Curran Associates Inc

    2019

  38. [46]

    Zhang, Y., Gu, S., Song, J., Pan, B., and Zhao, L. (2023a). XAI benchmark for visual explanation

    2023

  39. [47]

    Zhang, Y., Li, Y., Brown, H., Rezaei, M., Bischl, B., Torr, P. H. S., Khakzar, A., and Kawaguchi, K. (2023b). AttributionLab : Faithfulness of feature attribution under controllable environments

    2023

  40. [48]

    H., Chen, F., and Holzinger, A

    Zhou, J., Gandomi, A. H., Chen, F., and Holzinger, A. (2021). Evaluating the quality of machine learning explanations: A survey on methods and metrics. Electronics , 10(5):593

    2021

  41. [49]

    T., and Shah, J

    Zhou, Y., Booth, S., Ribeiro, M. T., and Shah, J. (2022). Do feature attribution methods correctly attribute features? In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 20...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.