Stratifying marginal Shapley by feature region avoids off-manifold extrapolation and, with a causal direction and a chosen reference constant, reproduces causal Shapley values on a linear spline and an insurance example.
Feature relevance quantification in explainable AI: A causal problem
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl's seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features in contradiction to the theoretical justification of the software package SHAP. Parts of SHAP are unaffected because unconditional expectations (which we argue to be conceptually right) are used as approximation for the conditional ones, which encouraged others to `improve' SHAP in a way that we believe to be flawed.
citation-role summary
background 1
citation-polarity summary
fields
stat.ML 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
On Model Extrapolation in Marginal Shapley Values
Stratifying marginal Shapley by feature region avoids off-manifold extrapolation and, with a causal direction and a chosen reference constant, reproduces causal Shapley values on a linear spline and an insurance example.