Pith. sign in

REVIEW 2 major objections 5 minor 105 references

Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read No assumption-free feature-attribution method exists: every local additive attribution encodes hidden mathematical choices, and trustworthy use requires reporting them.

desk verdict Useful survey with a real central claim, but the five-choice taxonomy and checklist miss essential hyperparameters for surrogate and perturbation methods; worth serious refereeing with revisions. read the letter →

arxiv 2607.14271 v1 pith:SP23EMUZ submitted 2026-07-15 cs.LG cs.AI

Rebecca Afriyie Sarpong , Daniel Commey This is my paper
classification cs.LGcs.AI
keywords featureattributionexplainableAIShapleyvaluesintegratedgradientsaxiomaticmethodsreportingchecklistvaluefunctionlocaladditive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that local additive feature-attribution methods—SHAP, Integrated Gradients, LIME, LRP, Grad-CAM, and similar—are not interchangeable explainers but are defined by five hidden mathematical choices: how feature absence is encoded, what baseline is compared against, what path is integrated, what perturbation distribution defines locality, and what conservation rule propagates credit. Because methods differ along these axes, their disagreements and failures are traceable to specification choices rather than to implementation bugs. The survey's concrete deliverables are a unified taxonomy, an axiom-by-method matrix showing which guarantees each method keeps, and a ten-item reporting checklist (R1–R10) that asks papers to state model output, feature granularity, baseline, value function, path, axioms, approximation budget, sanity checks, faithfulness metrics, and known failure modes. If the central claim is right, every attribution claim that omits these specifications is underdetermined, and the checklist is the remedy the field needs.

What carries the argument

The organizing device is the five-part specification: value function v (what feature absence means), reference x' (compared to what), path γ (along what trajectory), perturbation distribution p_z (which neighbourhood is local), and conservation rule (what quantity propagates). The axiom-by-method matrix (Table IX) turns these choices into a comparison table: each method's row shows which axioms it satisfies unconditionally, conditionally, or not at all. The paper also uses the additive surrogate g(z)=φ0+Σφ_i z_i to show that LIME, SHAP, DeepLIFT, LRP, and Integrated Gradients all speak the same linear-in-mask language and differ only in how coefficients are computed.

What would settle it

Run canonical LRP-epsilon on a bias-free ReLU network where one input feature is a provable dummy (zero weight and bias everywhere); if the dummy ever receives nonzero relevance, the matrix's conditional sensitivity-(b) entry for LRP would need revision. The same check can be repeated for any conditional cell, such as DeepLIFT's implementation invariance under a specific architecture.

Watch

Extended reading notes

Core claim

The paper's central claim, stated as a principle, is that there is no assumption-free feature-attribution method: every local additive attribution defines feature importance through choices about value functions, references, paths, perturbation distributions, or conservation rules, and trustworthy use requires reporting the heatmap or ranking together with the assumptions under which it was computed and interpreted. It substantiates this by organizing Shapley, path-based, gradient/backpropagation, perturbation, and CAM-style methods in one notation, cataloguing the axioms of each family, and building an axiom-by-method matrix showing that methods differ precisely in which axioms they satisfy

Load-bearing premise

The load-bearing premise is that the axiom-by-method matrix correctly assigns axioms to canonical method variants; if even one assignment is wrong, the taxonomy's central comparison and checklist item R6 lose their trustworthiness.

Editorial extensions

If this is right

  • Any study reporting attributions without stating baseline, value function, path, and perturbation distribution has reported an underdetermined quantity; the checklist gives the minimal information needed to reproduce and interpret an attribution.
  • Method disagreements should be investigated by holding value function, baseline, path, and perturbation distribution constant before attributing disagreement to the model.
  • Known failure modes—baseline sensitivity, off-manifold perturbation, adversarial manipulation, sanity-check failure—are consequences of specification choices, so they have structural mitigations rather than being fatal to a method.
  • Axiom-equivalence results (KernelSHAP equals Shapley in expectation; DeepLIFT-Rescale equals Integrated Gradients in the small-increment limit; LRP-ε equals gradient×input on bias-free ReLU networks) mean empirical differences between methods can be traced to axioms, value functions, and paths rather than to mathematical sophistication.
  • Attribution studies should report at least one sanity-check and one faithfulness or stability metric; otherwise the attribution cannot be distinguished from a model-ignorant map.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper's thesis is accepted as a discipline norm, the field's central question shifts from 'which attribution method is best?' to 'which specification matches the user's intended question?', and method papers would be required to publish their specification as part of the method definition.
  • The taxonomy suggests a testable extension: a meta-benchmark that varies the five specification choices systematically while holding model and input fixed could quantify how much of reported method disagreement is actually specification disagreement—a prediction that follows from the paper's Principle 1.
  • The checklist, if adopted, would make attribution papers more reproducible, but it also exposes a gap the paper itself acknowledges: conditional matrix entries depend on which variant of a method is taken as canonical, so the same method under different propagation rules would need separate matrix rows.
  • The central claim extends to large-language-model attribution: sub-token, token, span, and generation-target choices are themselves specification choices, so the 'no assumption-free attribution' thesis applies directly to the open problems the paper lists.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This manuscript is a taxonomy-based survey of local additive feature-attribution methods. It proposes that every such method is specified by five mathematical choices—value function, reference/baseline, path, perturbation distribution, and conservation rule—and it compares roughly thirty methods through an axiom-by-method matrix, formalizes common failure modes as consequences of these choices, and proposes a ten-item reporting checklist (R1–R10). The central claim is that there is no assumption-free attribution method and that attribution results are meaningful only when the assumptions under which they were computed are reported.

Significance. If the central claim holds, the paper provides a valuable unifying frame for a fragmented literature and a concrete, actionable reporting standard. The internal mathematics I checked is correct: the Shapley formula (Eq. 6), the IG completeness chain-rule argument (Eqs. 15–17), the Aumann–Shapley/IG identification (Eq. 7), and the running example in Section II.F. The paper also ships per-cell axiom justifications (Appendix A) and a checklist crosswalk (Appendix B), which make the taxonomy substantially reproducible. The main weakness is that the five-choice taxonomy and the R1–R10 checklist are not exhaustive of the specification choices that actually determine several surveyed methods, notably LIME's kernel/surrogate/regularization and meaningful-perturbation regularizers.

major comments (2)
  1. [§VII.B, §VII.D, §XII] The five-choice taxonomy omits specification choices that determine LIME and meaningful-perturbation attributions, and R1–R10 do not require them. In Eq. (31), LIME is defined by a similarity kernel π_x, a mask distribution, a surrogate family G, and a complexity penalty Ω; in Eq. (33), meaningful perturbations depend on λ and μ. None of these is captured by the categories 'value function, reference, path, perturbation distribution, conservation rule', and no checklist item asks for their values. A study could follow R1–R10 and still leave the sparsity penalty, surrogate family, or regularization coefficients unstated, producing an underdetermined attribution—exactly the state that Section I.G says should be avoided. The checklist should be extended (e.g., R4/R5 or a new item) to cover kernel, surrogate family, Ω, λ, μ, and optimization details, and the claim that the five choices are ex
  2. [§XIII.D–E, Table IX, Appendix A] The manuscript itself states that Table IX entries depend on the variant/implementation taken as canonical and that the matrix 'encodes the authors' reading'. This variant dependence is load-bearing for checklist item R6, which directs users to report which axioms their method satisfies. The per-cell justifications in Appendix A are helpful, but they do not uniquely identify the canonical variant for every row (e.g., LRP rule, DeepLIFT rule, CAM layer), and the main text does not require R6 users to name their variant. I recommend adding an explicit variant column or per-row canonical statement, and requiring R6 to name the implementation and rule variant. This is a fixable verifiability issue rather than a flaw in the central thesis, but it directly affects the reliability of the central comparison artifact.
minor comments (5)
  1. [§VI.C] The heading 'Result 6' appears twice in the same subsection; the second occurrence should be renumbered or identified as a continuation, since the repeated number is confusing.
  2. [§V.B, §IV.E] Equations (12) and (18) give the same expected-gradient definition essentially verbatim. Cross-reference one to the other to reduce redundancy.
  3. [Reproducibility Statement] The supplementary materials are described as including machine-readable CSV files and a scoring sheet, but these are not attached in the arXiv source. Please state explicitly where they are available or include them as appendices.
  4. [Table IX / Appendix A] For the Grad-CAM row, the matrix and Appendix A use 'completeness' at different levels (pixel-level vs. layer-level). A one-sentence clarification distinguishing these levels would prevent misreading of the entry.
  5. [§XIII.D] The corpus-limitation caveat is appropriately transparent. Consider adding the non-exclusive nature of role counts in Table III to the main text next to the table, rather than only in an appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an externally grounded synthesis, not a self-referential derivation.

full rationale

The paper's central claim is a taxonomic thesis, not a mathematical derivation, and I find no step in which an output quantity is defined in terms of an input quantity or in which a fitted value is relabeled as a prediction. The five specification choices (Table I) are explicitly introduced as 'the organizing frame for the survey,' a heuristic device for comparing methods; the paper does not define 'assumption-free' in terms of those five choices, nor does it use the conclusion to justify the frame. Its load-bearing results are external: Shapley uniqueness [44], Lundberg-Lee additive-attribution characterization [2], Sundararajan et al. IG axioms [1], Lundstrom-Razaviyayn IG characterizations [16], Bilodeau et al. impossibility [8], Adebayo et al. sanity checks [9], and Krishna et al. disagreement results [7]. None of these is a self-citation by Sarpong and Commey, so self-citation-based circularity patterns do not apply. The paper's own constructions (e.g., the f=x1x2 running example and Eq. (40)) are illustrative, not predictive fits. The limitations the authors themselves state (Section XIII.D/E: matrix entries depend on method variants and encode the authors' reading) qualify the reliability of the axiom matrix, but they are correctness/uncertainty caveats, not circularity. The point that LIME's complexity penalty or meaningful perturbations' regularizers are not among the five choices is an argument about completeness of the taxonomy, not about an equation reducing to itself. Accordingly, no circular step meets the quoted-evidence standard, and the score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

Nothing in this survey is fitted: there are no learned constants, and the worked examples (f = x1x2, f = 1 − e^{−5x}) are illustrative. The load-bearing premises are (1) the completeness of the five-choice organizing frame (Fig. 1), (2) faithful representation of the 105 cited works in Tables IX–X, (3) the additive-surrogate form (Eq. 5) as the scope-defining frame, and (4) the accuracy of cited empirical findings, which the paper does not re-run. The manuscript itself concedes (2) and (4) in Section XIII.

assumptions (4)
  • domain assumption Every local additive attribution method is fully determined by five specification choices: value function, reference, path, perturbation distribution, and conservation rule.
    Organizing frame of the survey (Fig. 1, Sec. I). If a surveyed method resists this frame — attention rollout, XRAI region aggregation, hierarchical/contextual-decomposition methods are the marginal cases — the taxonomy's completeness claim and the inference that differences reduce to these five choices weaken accordingly. The paper partially hedges by marking boundary cases.
  • domain assumption Table IX's axiom-by-method entries accurately represent the cited method papers under a canonical-version choice.
    The paper states the matrix 'encodes the authors' reading' (Sec. XIII.E) and that entries are conditional on LRP/DeepLIFT rule and CAM-layer variants (Sec. XIII.D). The central comparison artifact is thus a secondary synthesis, not a machine-checkable certificate.
  • domain assumption The additive surrogate g(z) = φ0 + Σφizi (Eq. 5) and the completeness target Σφi = f(x) − f(x') define the surveyed class.
    Adopted from Lundberg & Lee [2] as the scope-defining frame; excludes rule-based (Anchors), counterfactual, and example-based explanations (Sec. I.C). The survey's 'additive' scope is a premise, not a result.
  • domain assumption Cited empirical findings (model/data-randomization sanity-check outcomes, median rank correlation < 0.5, adversarial constructions with unbounded attribution change) are accurate as reported by the original authors.
    Table X and Section X compile external empirics without re-running them; Result 13 explicitly disclaims a self-contained proof (Sec. X.F). The survey's failure-mode analyses inherit the epistemic status of the cited studies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist." pith.science (2026). https://pith.science/paper/SP23EMUZ

@misc{pith2026260714271,
  author       = {Pith},
  title        = {Pith review of: Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SP23EMUZ}},
  note         = {Machine review of arXiv:2607.14271}
}
read the original abstract

Feature-attribution methods are central to explainable artificial intelligence. Their assumptions are expressed in several mathematical languages: cooperative-game values, path integrals, gradient operators, perturbation distributions, and backpropagation rules. This survey proposes a common framework for local additive feature attribution. It organizes Shapley, path-based, gradient/backpropagation, perturbation, and CAM-style methods around five specification choices: value function, reference, path, perturbation distribution, and conservation rule. It then compares these methods through an axiom-by-method matrix and links common failure modes, including baseline sensitivity, off-manifold perturbations, sanity-check failures, adversarial manipulation, and method disagreement, to the assumptions that produce them. Finally, the survey proposes a ten-item reporting checklist for studies that use local additive attributions. The central message is that attribution results are meaningful only relative to the mathematical assumptions under which they are defined, and that those assumptions should be reported.

Figures

Figures reproduced from arXiv: 2607.14271 by the authors.

Figure 1
Figure 1. Attribution methods as specifications of hidden mathematical [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Inter-axiom relationships. Layer-wise conservation (Axiom 14) [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Baseline and path dependence for path-based attribution [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Taxonomy of local additive feature attribution methods used throughout this survey, with the mathematical object that each family [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Generic evaluation pipeline for attribution methods. The [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Insertion and deletion curves [89]. Features are inserted into a [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Gradient saturation along an attribution path. For a sigmoidal [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

105 extracted references · 9 linked inside Pith

  1. [1]

    Axiomatic attribution for deep networks,

    M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” inProc. ICML, ser. Proceedings of Machine Learning Research, vol. 70, 2017, pp. 3319–3328

    2017

  2. [2]

    A unified approach to interpret- ing model predictions,

    S. M. Lundberg and S.-I. Lee, “A unified approach to interpret- ing model predictions,” inProc. NeurIPS, vol. 30, 2017

    2017

  3. [3]

    “Why should I trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin, ““Why should I trust you?” Explaining the predictions of any classifier,” inProc. ACM SIGKDD, 2016, pp. 1135–1144

    2016

  4. [4]

    Grad-CAM: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” inProc. ICCV, 2017, pp. 618–626

    2017

  5. [5]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,

    S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,”PLOS ONE, vol. 10, no. 7, p. e0130140, 2015

    2015

  6. [6]

    Learning impor- tant features through propagating activation differences,

    A. Shrikumar, P. Greenside, and A. Kundaje, “Learning impor- tant features through propagating activation differences,” in Proc. ICML, 2017

    2017

  7. [7]

    The disagreement problem in explainable machine learning: A practitioner’s perspective,

    S. Krishna, T. Han, A. Gu, S. Wu, S. Jabbari, and H. Lakkaraju, “The disagreement problem in explainable machine learning: A practitioner’s perspective,”Transactions on Machine Learning Research, 2024

    2024

  8. [8]

    Impossibility theorems for feature attribution,

    B. Bilodeau, N. Jaques, P. W. Koh, and B. Kim, “Impossibility theorems for feature attribution,”Proceedings of the National Academy of Sciences, vol. 121, no. 2, 2024, art. no. e2319169121

    2024

Show all 105 references
  1. [9]

    Sanity checks for saliency maps,

    J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” inProc. NeurIPS, vol. 31, 2018, pp. 9525–9536

    2018

  2. [10]

    The (un)reliability of saliency methods,

    P.-J. Kindermans, S. Hooker, J. Adebayo, M. Alber, K. T. Schütt, S. Dähne, D. Erhan, and B. Kim, “The (un)reliability of saliency methods,” inExplainable AI: Interpreting, Explaining and Visualizing Deep Learning. Cham, Switzerland: Springer, 2019, pp. 267–280

    2019

  3. [11]

    Sanity checks for saliency metrics,

    R. Tomsett, D. Harborne, S. Chakraborty, P. Gurram, and A. Preece, “Sanity checks for saliency metrics,” inProc. AAAI, vol. 34, no. 4, 2020, pp. 6021–6029

    2020

  4. [12]

    A bench- mark for interpretability methods in deep neural networks,

    S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim, “A bench- mark for interpretability methods in deep neural networks,” in Proc. NeurIPS, vol. 32, 2019

    2019

  5. [13]

    Explanations can be manipulated and geometry is to blame,

    A.-K. Dombrowski, M. Alber, C. Anders, M. Ackermann, K.-R. Müller, and P. Kessel, “Explanations can be manipulated and geometry is to blame,” inProc. NeurIPS, vol. 32, 2019, pp. 13567–13578

    2019

  6. [14]

    Interpretation of neural networks is fragile,

    A. Ghorbani, A. Abid, and J. Zou, “Interpretation of neural networks is fragile,” inProc. AAAI, 2019

    2019

  7. [15]

    Fooling LIME and SHAP: Adversarial attacks on post-hoc explanation methods,

    D.Slack,S.Hilgard,E.Jia,S.Singh,andH.Lakkaraju,“Fooling LIME and SHAP: Adversarial attacks on post-hoc explanation methods,” inProc. AAAI/ACM Conf. on AI, Ethics, and Society (AIES), 2020, pp. 180–186

    2020

  8. [16]

    Four axiomatic characteri- zations of the integrated gradients attribution method,

    D. Lundstrom and M. Razaviyayn, “Four axiomatic characteri- zations of the integrated gradients attribution method,”Journal of Machine Learning Research, vol. 26, no. 177, pp. 1–31, 2025

    2025

  9. [17]

    The many Shapley values for model explanation,

    M. Sundararajan and A. Najmi, “The many Shapley values for model explanation,” inProc. ICML, 2020

    2020

  10. [18]

    Problems with Shapley-value-based explanations as feature importance measures,

    I. E. Kumar, C. Scheidegger, S. Venkatasubramanian, and S. A. Friedler, “Problems with Shapley-value-based explanations as feature importance measures,” inProc. ICML, 2020

    2020

  11. [19]

    Which explanation should i choose? A function approximation perspective to characterizing post hoc explanations,

    T. Han, S. Srinivas, and H. Lakkaraju, “Which explanation should i choose? A function approximation perspective to characterizing post hoc explanations,” inProc. NeurIPS, 2022

    2022

  12. [20]

    Improving KernelSHAP: Practical Shapley value estimation using linear regression,

    I. Covert and S.-I. Lee, “Improving KernelSHAP: Practical Shapley value estimation using linear regression,” inProc. AISTATS, 2021

    2021

  13. [21]

    Towards better understanding of gradient-based attribution methods for deep neural networks,

    M. Ancona, E. Ceolini, C. Öztireli, and M. Gross, “Towards better understanding of gradient-based attribution methods for deep neural networks,” inProc. ICLR, 2018

    2018

  14. [22]

    The mythos of model interpretability,

    Z. C. Lipton, “The mythos of model interpretability,”Commu- nications of the ACM, vol. 61, no. 10, pp. 36–43, 2018

    2018

  15. [23]

    Towards a rigorous science of interpretable machine learning,

    F. Doshi-Velez and B. Kim, “Towards a rigorous science of interpretable machine learning,”arXiv:1702.08608, 2017

    2017 arXiv

  16. [24]

    Feature relevance quantification in explainable AI: A causal problem,

    D. Janzing, L. Minorics, and P. Blöbaum, “Feature relevance quantification in explainable AI: A causal problem,” inProc. AISTATS, ser. Proceedings of Machine Learning Research, vol. 108, 2020, pp. 2907–2916. SARPONG AND COMMEY: A REPORTING CHECKLIST FOR LOCAL ADDITIVE ATTRIBUT...

    2020

  17. [25]

    Shapley explainability on the data manifold,

    C. Frye, D. de Mijolla, T. Begley, L. Cowton, M. Stanley, and I. Feige, “Shapley explainability on the data manifold,” inProc. ICLR, 2021

    2021

  18. [26]

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV),

    B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viégas, and R. Sayres, “Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV),” inProc. ICML, 2018

    2018

  19. [27]

    Network dissection: Quantifying interpretability of deep visual representations,

    D. Bau, B. Zhou, A. Khosla, A. Oliva, and A. Torralba, “Network dissection: Quantifying interpretability of deep visual representations,” inProc. CVPR, 2017

    2017

  20. [28]

    A survey of methods for explaining black box models,

    R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, “A survey of methods for explaining black box models,”ACM Computing Surveys, vol. 51, no. 5, 2018

    2018

  21. [29]

    Peeking inside the black-box: A surveyonexplainableartificialintelligence,

    A. Adadi and M. Berrada, “Peeking inside the black-box: A surveyonexplainableartificialintelligence,”IEEEAccess,vol.6, pp. 52138–52160, 2018

    2018

  22. [30]

    Explaining explanations: An overview of inter- pretability of machine learning,

    L. H. Gilpin, D. Bau, B. Z. Yuan, A. Bajwa, M. Specter, and L. Kagal, “Explaining explanations: An overview of inter- pretability of machine learning,” inProc. IEEE DSAA, 2018

    2018

  23. [31]

    Molnar,Interpretable Machine Learning, 2nd ed

    C. Molnar,Interpretable Machine Learning, 2nd ed. Independently published, 2022. [Online]. Available: https: //christophm.github.io/interpretable-ml-book/

    2022

  24. [32]

    Explaining deep neural networks: A survey on the global interpretation methods,

    R. Saleem, B. Yuan, F. Kurugollu, A. Anjum, and L. Liu, “Explaining deep neural networks: A survey on the global interpretation methods,”Neurocomputing, vol. 513, pp. 165– 180, 2022

    2022

  25. [33]

    Gradient based feature attribution in explainable AI: A technical review,

    Y. Wang, T. Zhang, X. Guo, and Z. Shen, “Gradient based feature attribution in explainable AI: A technical review,” arXiv:2403.10415, 2024

    2024 arXiv

  26. [34]

    Additive-feature- attribution methods: A review on explainable artificial intelli- gence for fluid dynamics and heat transfer,

    A. Cremades, S. Hoyas, and R. Vinuesa, “Additive-feature- attribution methods: A review on explainable artificial intelli- gence for fluid dynamics and heat transfer,”arXiv:2409.11992, SARPONG AND COMMEY: A REPORTING CHECKLIST FOR LOCAL ADDITIVE ATTRIBUTION 34 2024

    2024 arXiv

  27. [35]

    Shapley value: From cooperative game to explainable artificial intelligence,

    M. Li, H. Sun, Y. Huang, and H. Chen, “Shapley value: From cooperative game to explainable artificial intelligence,” Autonomous Intelligent Systems, vol. 4, no. 1, 2024, art. no. 2

    2024

  28. [36]

    From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI,

    M. Nauta, J. Trienes, S. Pathak, E. Nguyen, M. Peters, Y. Schmitt, J. Schlötterer, M. van Keulen, and C. Seifert, “From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI,”ACM Computing Surveys, vol. 55, no. 13s, 2023, art. no. 295

    2023

  29. [37]

    Ex- plainable AI: A review of machine learning interpretability methods,

    P. Linardatos, V. Papastefanopoulos, and S. Kotsiantis, “Ex- plainable AI: A review of machine learning interpretability methods,”Entropy, vol. 23, no. 1, p. 18, 2021

    2021

  30. [38]

    Improving performance of deep learning models with axiomatic attribution priors and expected gradients,

    G. Erion, J. D. Janizek, P. Sturmfels, S. M. Lundberg, and S.-I. Lee, “Improving performance of deep learning models with axiomatic attribution priors and expected gradients,”Nature Machine Intelligence, vol. 3, pp. 620–631, 2021

    2021

  31. [39]

    Explaining individual predictions when features are dependent: More accurate ap- proximations to Shapley values,

    K. Aas, M. Jullum, and A. Løland, “Explaining individual predictions when features are dependent: More accurate ap- proximations to Shapley values,”Artificial Intelligence, vol. 298, 2021

    2021

  32. [40]

    The explanation game: Explaining machine learning models using Shapley values,

    L. Merrick and A. Taly, “The explanation game: Explaining machine learning models using Shapley values,” inProc. CD- MAKE, 2020

    2020

  33. [41]

    Guided integrated gradients: An adaptive path method for removing noise,

    A. Kapishnikov, S. Venugopalan, B. Avci, B. Wedin, M. Terry, and T. Bolukbasi, “Guided integrated gradients: An adaptive path method for removing noise,” inProc. CVPR, 2021

    2021

  34. [42]

    Attribution in scale and space,

    S. Xu, S. Venugopalan, and M. Sundararajan, “Attribution in scale and space,” inProc. CVPR, 2020

    2020

  35. [43]

    Explaining image classifiers by counterfactual generation,

    C.-H. Chang, E. Creager, A. Goldenberg, and D. Duvenaud, “Explaining image classifiers by counterfactual generation,” in Proc. ICLR, 2019

    2019

  36. [44]

    A value for n-person games,

    L. S. Shapley, “A value for n-person games,” inContributions to the Theory of Games, Volume II, ser. Annals of Mathematics Studies, H. W. Kuhn and A. W. Tucker, Eds. Princeton University Press, 1953, no. 28, pp. 307–317

    1953

  37. [45]

    From local explanations to global understanding with explainable AI for trees,

    S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B.Nair,R.Katz,J.Himmelfarb,N.Bansal,andS.-I.Lee,“From local explanations to global understanding with explainable AI for trees,”Nature Machine Intelligence, vol. 2, no. 1, pp. 56–67, 2020

    2020

  38. [46]

    Weighted voting doesn’t work: A mathematical analysis,

    J. F. Banzhaf, “Weighted voting doesn’t work: A mathematical analysis,”Rutgers Law Review, vol. 19, pp. 317–343, 1965

    1965

  39. [47]

    Multilinear extensions of games,

    G. Owen, “Multilinear extensions of games,”Management Science, vol. 18, no. 5, pp. P64–P79, 1972

    1972

  40. [48]

    R. J. Aumann and L. S. Shapley,Values of Non-Atomic Games. Princeton University Press, 1974

    1974

  41. [49]

    On the robustness of interpretability methods,

    D. Alvarez-Melis and T. S. Jaakkola, “On the robustness of interpretability methods,” inICML Workshop on Human Interpretability, 2018

    2018

  42. [50]

    An efficient explanation of individual classifications using game theory,

    E. Štrumbelj and I. Kononenko, “An efficient explanation of individual classifications using game theory,”Journal of Machine Learning Research, vol. 11, pp. 1–18, 2010

    2010

  43. [51]

    Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems,

    A. Datta, S. Sen, and Y. Zick, “Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems,” inProc. IEEE Symposium on Security and Privacy, 2016, pp. 598–617

    2016

  44. [52]

    Algo- rithms to estimate Shapley value feature attributions,

    H. Chen, I. C. Covert, S. M. Lundberg, and S.-I. Lee, “Algo- rithms to estimate Shapley value feature attributions,”Nature Machine Intelligence, vol. 5, pp. 590–601, 2023

    2023

  45. [53]

    Understanding global feature contributions with additive importance measures,

    I. Covert, S. Lundberg, and S.-I. Lee, “Understanding global feature contributions with additive importance measures,” in Proc. NeurIPS, 2020

    2020

  46. [54]

    Shapley values for feature selection: The good, the bad, and the axioms,

    D. Fryer, I. Strümke, and H. Nguyen, “Shapley values for feature selection: The good, the bad, and the axioms,” arXiv:2102.10936, 2021

    2021 arXiv

  47. [55]

    The Shapley-Taylor interaction index,

    K. Dhamdhere, A. Agarwal, and M. Sundararajan, “The Shapley-Taylor interaction index,” inProc. ICML, 2020

    2020

  48. [56]

    Explaining ex- planations: Axiomatic feature interactions for deep networks,

    J. D. Janizek, P. Sturmfels, and S.-I. Lee, “Explaining ex- planations: Axiomatic feature interactions for deep networks,” Journal of Machine Learning Research, vol. 22, no. 104, pp. 1–54, 2021

    2021

  49. [57]

    Detecting statistical interac- tions from neural network weights,

    M. Tsang, D. Cheng, and Y. Liu, “Detecting statistical interac- tions from neural network weights,” inProc. ICLR, 2018

    2018

  50. [58]

    Neuron Shapley: Discovering the responsible neurons,

    A. Ghorbani and J. Zou, “Neuron Shapley: Discovering the responsible neurons,” inProc. NeurIPS, 2020

    2020

  51. [59]

    Visualizingtheimpact of feature attribution baselines,

    P.Sturmfels,S.Lundberg,andS.-I.Lee,“Visualizingtheimpact of feature attribution baselines,”Distill, 2020

    2020

  52. [60]

    XRAI: Better attributions through regions,

    A. Kapishnikov, T. Bolukbasi, F. Viégas, and M. Terry, “XRAI: Better attributions through regions,” inProc. ICCV, 2019

    2019

  53. [61]

    How important is a neuron?

    K. Dhamdhere, M. Sundararajan, and Q. Yan, “How important is a neuron?” inProc. ICLR, 2019

    2019

  54. [62]

    Deep inside convolutional networks: Visualising image classification models and saliency maps,

    K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,”arXiv:1312.6034, 2013

    2013 arXiv

  55. [63]

    How to explain individual classification decisions,

    D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R. Müller, “How to explain individual classification decisions,”Journal of Machine Learning Research, vol. 11, pp. 1803–1831, 2010

    2010

  56. [64]

    SmoothGrad: Removing noise by adding noise,

    D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wat- tenberg, “SmoothGrad: Removing noise by adding noise,” arXiv:1706.03825, 2017

    2017 arXiv

  57. [65]

    Striving for simplicity: The all convolutional net,

    J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, “Striving for simplicity: The all convolutional net,” inICLR Workshop, 2015

    2015

  58. [66]

    Visualizing and understanding convolutional networks,

    M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” inProc. ECCV, 2014, pp. 818–833

    2014

  59. [67]

    Explaining nonlinear classification decisions with deep Taylor decomposition,

    G. Montavon, S. Lapuschkin, A. Binder, W. Samek, and K.- R. Müller, “Explaining nonlinear classification decisions with deep Taylor decomposition,”Pattern Recognition, vol. 65, pp. 211–222, 2017

    2017

  60. [68]

    Full-gradient representation for neural network visualization,

    S. Srinivas and F. Fleuret, “Full-gradient representation for neural network visualization,” inProc. NeurIPS, 2019

    2019

  61. [69]

    Learning deep features for discriminative localization,

    B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” inProc. CVPR, 2016

    2016

  62. [70]

    Grad-CAM++: Improved visual explanations for deep convolutional networks,

    A. Chattopadhyay, A. Sarkar, P. Howlader, and V. N. Bala- subramanian, “Grad-CAM++: Improved visual explanations for deep convolutional networks,” inProc. WACV, 2018, pp. 839–847

    2018

  63. [71]

    Score-CAM: Score-weighted visual explanations for convolutional neural networks,

    H. Wang, Z. Wang, M. Du, F. Yang, Z. Zhang, S. Ding, P. Mardziel, and X. Hu, “Score-CAM: Score-weighted visual explanations for convolutional neural networks,” inProc. CVPR Workshops, 2020

    2020

  64. [72]

    Ablation-CAM: Visual explanations for deep convolutional network via gradient-free localization,

    S. Desai and H. G. Ramaswamy, “Ablation-CAM: Visual explanations for deep convolutional network via gradient-free localization,” inProc. WACV, 2020

    2020

  65. [73]

    LayerCAM: Exploring hierarchical class activation maps for localization,

    P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, “LayerCAM: Exploring hierarchical class activation maps for localization,”IEEE Trans. Image Processing, vol. 30, pp. 5875– 5888, 2021

    2021

  66. [74]

    Eigen-CAM: Class acti- vation map using principal components,

    M. B. Muhammad and M. Yeasin, “Eigen-CAM: Class acti- vation map using principal components,”arXiv:2008.00299, 2020

    2008 arXiv

  67. [75]

    Use HiResCAM instead of Grad- CAM for faithful explanations of CNNs,

    R. L. Draelos and L. Carin, “Use HiResCAM instead of Grad- CAM for faithful explanations of CNNs,”arXiv:2011.08891, 2020

    2011 arXiv

  68. [76]

    Shap-CAM: Visual explanationsforconvolutionalneuralnetworksbasedonShapley value,

    Q. Zheng, Z. Wang, J. Zhou, and J. Lu, “Shap-CAM: Visual explanationsforconvolutionalneuralnetworksbasedonShapley value,” inProc. ECCV, 2022

    2022

  69. [77]

    Attention is not explanation,

    S. Jain and B. C. Wallace, “Attention is not explanation,” in Proc. NAACL, 2019

    2019

  70. [78]

    Is attention interpretable?

    S. Serrano and N. A. Smith, “Is attention interpretable?” in Proc. ACL, 2019

    2019

  71. [79]

    Attention is not not explanation,

    S. Wiegreffe and Y. Pinter, “Attention is not not explanation,” inProc. EMNLP, 2019

    2019

  72. [80]

    Quantifying attention flow in transformers,

    S. Abnar and W. Zuidema, “Quantifying attention flow in transformers,” inProc. ACL, 2020

    2020

  73. [81]

    Transformer interpretability beyond attention visualization,

    H. Chefer, S. Gur, and L. Wolf, “Transformer interpretability beyond attention visualization,” inProc. CVPR, 2021

    2021

  74. [82]

    Vi- sualizing deep neural network decisions: Prediction difference analysis,

    L. M. Zintgraf, T. S. Cohen, T. Adel, and M. Welling, “Vi- sualizing deep neural network decisions: Prediction difference analysis,” inProc. ICLR, 2017

    2017

  75. [83]

    Model agnostic supervised local explanations,

    G. Plumb, D. Molitor, and A. S. Talwalkar, “Model agnostic supervised local explanations,” inProc. NeurIPS, 2018

    2018

  76. [84]

    Random forests,

    L. Breiman, “Random forests,”Machine Learning, vol. 45, no. 1, pp. 5–32, 2001

    2001

  77. [85]

    Greedy function approximation: A gradient boosting machine,

    J. H. Friedman, “Greedy function approximation: A gradient boosting machine,”The Annals of Statistics, vol. 29, no. 5, pp. 1189–1232, 2001

    2001

  78. [86]

    Anchors: High- precision model-agnostic explanations,

    M. T. Ribeiro, S. Singh, and C. Guestrin, “Anchors: High- precision model-agnostic explanations,” inProc. AAAI, 2018

    2018

  79. [87]

    Interpretable explanations of black boxes by meaningful perturbation,

    R. C. Fong and A. Vedaldi, “Interpretable explanations of black boxes by meaningful perturbation,” inProc. ICCV, 2017. SARPONG AND COMMEY: A REPORTING CHECKLIST FOR LOCAL ADDITIVE ATTRIBUTION 35

    2017

  80. [88]

    Understanding deep networks via extremal perturbations and smooth masks,

    R. Fong, M. Patrick, and A. Vedaldi, “Understanding deep networks via extremal perturbations and smooth masks,” in Proc. ICCV, 2019

    2019

  81. [89]

    RISE: Randomized input sampling for explanation of black-box models,

    V. Petsiuk, A. Das, and K. Saenko, “RISE: Randomized input sampling for explanation of black-box models,” inProc. BMVC, 2018

    2018

  82. [90]

    Real time image saliency for black box classifiers,

    P. Dabkowski and Y. Gal, “Real time image saliency for black box classifiers,” inProc. NeurIPS, 2017

    2017

  83. [91]

    Restricting the flow: Information bottlenecks for attribution,

    K. Schulz, L. Sixt, F. Tombari, and T. Landgraf, “Restricting the flow: Information bottlenecks for attribution,” inProc. ICLR, 2020

    2020

  84. [92]

    Evaluating the visualization of what a deep neural network has learned,

    W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K.-R. Müller, “Evaluating the visualization of what a deep neural network has learned,”IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 11, pp. 2660–2673, 2017

    2017

  85. [93]

    ERASER: A benchmark to evaluate rationalized NLP models,

    J. DeYoung, S. Jain, N. F. Rajani, E. Lehman, C. Xiong, R. Socher, and B. C. Wallace, “ERASER: A benchmark to evaluate rationalized NLP models,” inProc. ACL, 2020

    2020

  86. [94]

    Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?

    A. Jacovi and Y. Goldberg, “Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?” inProc. ACL, 2020

    2020

  87. [95]

    On the (in)fidelity and sensitivity of explanations,

    C.-K. Yeh, C.-Y. Hsieh, A. Suggala, D. I. Inouye, and P. Raviku- mar, “On the (in)fidelity and sensitivity of explanations,” in Proc. NeurIPS, 2019

    2019

  88. [96]

    Right for the right reasons: Training differentiable models by constraining their explanations,

    A. S. Ross, M. C. Hughes, and F. Doshi-Velez, “Right for the right reasons: Training differentiable models by constraining their explanations,” inProc. IJCAI, 2017

    2017

  89. [97]

    Interpretations are useful: Penalizing explanations to align neural networks with prior knowledge,

    L. Rieger, C. Singh, W. J. Murdoch, and B. Yu, “Interpretations are useful: Penalizing explanations to align neural networks with prior knowledge,” inProc. ICML, 2020

    2020

  90. [98]

    “will you find these shortcuts?

    J. Bastings, S. Ebert, P. Zablotskaia, A. Sandholm, and K. Filippova, ““will you find these shortcuts?” a protocol for evaluating the faithfulness of input salience methods for text classification,”arXiv:2111.07367, 2022

    2022 arXiv

  91. [99]

    The shattered gradients problem: If resnets are the answer, then what is the question?

    D. Balduzzi, M. Frean, L. Leary, J. P. Lewis, K. W.-D. Ma, and B. McWilliams, “The shattered gradients problem: If resnets are the answer, then what is the question?” inProc. ICML, ser. Proceedings of Machine Learning Research, vol. 70, 2017, pp. 342–350

    2017

  92. [100]

    Explain- ing recurrent neural network predictions in sentiment analysis,

    L. Arras, G. Montavon, K.-R. Müller, and W. Samek, “Explain- ing recurrent neural network predictions in sentiment analysis,” inEMNLP WASSA Workshop, 2017

    2017

  93. [101]

    Hierarchical interpreta- tions for neural network predictions,

    C. Singh, W. J. Murdoch, and B. Yu, “Hierarchical interpreta- tions for neural network predictions,” inProc. ICLR, 2019

    2019

  94. [102]

    Beyond word importance: Contextual decomposition to extract interactions from LSTMs,

    W. J. Murdoch, P. J. Liu, and B. Yu, “Beyond word importance: Contextual decomposition to extract interactions from LSTMs,” inProc. ICLR, 2018

    2018

  95. [103]

    How does this interaction affect me? Interpretable attribution for feature interactions,

    M. Tsang, S. Rambhatla, and Y. Liu, “How does this interaction affect me? Interpretable attribution for feature interactions,” inProc. NeurIPS, 2020

    2020

  96. [104]

    Quantus:AnexplainableAItoolkitforresponsibleevaluationof neural network explanations and beyond,

    A. Hedström, L. Weber, D. Bareeva, D. Krakowczyk, F. Motzkus, W. Samek, S. Lapuschkin, and M. M.-C. Höhne, “Quantus:AnexplainableAItoolkitforresponsibleevaluationof neural network explanations and beyond,”Journal of Machine Learning Research, vol. 24, no. 34, pp. 1–11, 2023

    2023

  97. [105]

    Interpretable decision sets: A joint framework for description and prediction,

    H. Lakkaraju, S. H. Bach, and J. Leskovec, “Interpretable decision sets: A joint framework for description and prediction,” inProc. ACM SIGKDD, 2016

    2016

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.