Gradient based Feature Attribution in Explainable AI: A Technical Review
read the original abstract
The surge in black-box AI models has prompted the need to explain the internal mechanism and justify their reliability, especially in high-stakes applications, such as healthcare and autonomous driving. Due to the lack of a rigorous definition of explainable AI (XAI), a plethora of research related to explainability, interpretability, and transparency has been developed to explain and analyze the model from various perspectives. Consequently, with an exhaustive list of papers, it becomes challenging to have a comprehensive overview of XAI research from all aspects. Considering the popularity of neural networks in AI research, we narrow our focus to a specific area of XAI research: gradient based explanations, which can be directly adopted for neural network models. In this review, we systematically explore gradient based explanation methods to date and introduce a novel taxonomy to categorize them into four distinct classes. Then, we present the essence of technique details in chronological order and underscore the evolution of algorithms. Next, we introduce both human and quantitative evaluations to measure algorithm performance. More importantly, we demonstrate the general challenges in XAI and specific challenges in gradient based explanations. We hope that this survey can help researchers understand state-of-the-art progress and their corresponding disadvantages, which could spark their interest in addressing these issues in future work.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
Explanation of Dynamic Physical Field Predictions using WassersteinGrad: Application to Autoregressive Weather Forecasting
WassersteinGrad aggregates perturbed gradient attribution maps via their entropic Wasserstein barycenter to avoid blurring from geometric shifts in explanations of autoregressive weather forecasts.
-
ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding
ABLE constructs model embeddings from gradient-based input attributions, enabling training-free LLM comparison across 239 models with theoretical stability guarantees.
-
AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption
AttnTrace is an attention-weight-based context traceback method for LLMs that claims higher accuracy and efficiency than prior art like TracLLM while aiding prompt injection detection.
-
Explainable AI Isn't Enough! Rethinking Algorithmic Contestability
The paper defines algorithmic contestability as identifying evidence to overturn potentially incorrect decisions and identifies three types of such evidence that make decisions normatively indefensible under the decis...
-
Evaluating the Temporal Detection Capability of Integrated Gradients Applied on Sound Classifier
Integrated gradients on a 10-class domestic sound classifier yields 0.39 mean IoU, 0.52 frame F1 and 82.6% Pointing Game accuracy for temporal event detection, approaching weakly and strongly supervised framewise CNN ...
-
Safety Must Precede the Deployment of Open-Ended AI
Open-ended AI systems introduce distinct safety challenges including loss of predictability, emergent misalignment, and control difficulties that must be addressed preemptively before large-scale deployment.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.