By fixing CLIP-derived language similarity maps as the coefficient matrix in non-negative matrix factorization, this method produces named, faithful concept explanations for frozen image classifiers.
hub
SmoothGrad: removing noise by adding noise
51 Pith papers cite this work, alongside 755 external citations. Polarity classification is still indexing.
51
Pith papers citing it
755
external citations · Pith
abstract
Explaining the output of a deep network remains a challenge. In the case of an image classifier, one type of explanation is to identify pixels that strongly influence the final decision. A starting point for this strategy is the gradient of the class score function with respect to the input image. This gradient can be interpreted as a sensitivity map, and there are several techniques that elaborate on this basic idea. This paper makes two contributions: it introduces SmoothGrad, a simple method that can help visually sharpen gradient-based sensitivity maps, and it discusses lessons in the visualization of these maps. We publish the code for our experiments and a website with our results.
hub tools
citation-role summary
background 3
method 1
citation-polarity summary
representative citing papers
FedXDS uses propagation-based attribution to identify task-relevant features for selective data sharing in federated learning, yielding higher accuracy and faster convergence under heterogeneity with formal privacy guarantees.
KARMA constructs minimal-K Markov transition kernels as surrogates to deliver global explanations for multivariate time series forecasting models and recovers known causal structure on synthetic data.
DiffIG reformulates attribution path generation as conditional diffusion modeling trained on Stick-Breaking Process paths with guided sampling for user-controllable XAI.
Concept-based models can use controlled 'benign' information leakage to remain accurate and intervenable under real-world concept incompleteness by reframing their training objective.
Reveal-IG performs path attribution by integrating model output changes along trajectories in a space of probe distributions rather than input-space paths, retaining completeness and handling multiscale or uncertain features.
AIM is a new saliency-guided adversarial feature replacement method to evaluate faithfulness of saliency maps and reliability of masking operators on image, audio, and EEG tasks.
AGOP-based attribution methods outperform Integrated Gradients and other baselines on pixel-level ground truth benchmarks for explaining image classifier decisions, with AGOP-Global offering zero inference cost.
Transport-geodesic attribution via optimal generative flows selects principled paths for feature attributions by minimizing kinetic action.
MobileMold provides 4941 smartphone microscopy images and shows deep learning models reach 99.5% accuracy on mold detection and food classification tasks.
Introduces learning-augmented robust algorithmic recourse that trades off consistency with accurate future-model predictions against robustness to inaccurate predictions via a novel algorithm.
Defines meta-attributions as directional second-order Shapley values on attribution methods, proves hierarchical decomposition of attributions, and demonstrates applications in language models, vision-language encoders, and diffusion transformers.
An iERF-centric framework unifies local, global, and mechanistic interpretability in vision models via SRD for saliency, CAFE for concept anchoring, and ICAT for interlayer attribution.
Adversarial perturbations possess an inherently low-rank structure that enables more efficient and effective black-box adversarial attacks via subspace projection.
A single end-to-end Transformer model unifies stellar labels from heterogeneous spectroscopic surveys into a self-consistent scale without post-hoc recalibration.
The Linear Centroids Hypothesis reframes network features as directions in centroid spaces of local affine experts, unifying interpretability methods and yielding sparser, more faithful dictionaries, circuits, and saliency maps.
Sparse interventions on 8–64 neurons distributed across layers can activate task behavior in instruction-tuned LLMs, outperforming first-order linear steering approaches by modeling nonlinear neuron interactions.
Gradient saliency of teacher-forced token log-probabilities, decoded by dynamic programming, yields usable word alignments for every ASR family including speech LLMs.
CascadeFormer tapers Transformer width with depth based on gradient fan-in asymmetry to match uniform baselines in perplexity while cutting latency 8.6%.
PV-TAM uses prompt-side semantics and a bias filter to improve attention-based and IoU localization metrics for vision-language models over answer-side baselines.
WorldModelLens defines a typed adapter with four core methods and a capability descriptor to unify interpretability tooling across diverse world model architectures.
PredHydro-Net is a new dual-decoding architecture with wavelet spectral matching and adversarial training that outperforms Earthformer, PredRNNv2, and GFS on extreme-event detection and spectral fidelity in 72-hour global hydrometeor forecasts.
TEVI applies sparse autoencoders and caption-conditioned masking to edit image embeddings, yielding better retrieval on MS COCO, Flickr, IIW, DOCCI, and RoCOCO benchmarks with larger gains on richer captions.
CPC-VAR adds Gradient-based Concept Neuron Selection for continual single-concept learning and a context-aware multi-branch composition strategy to reduce forgetting and entanglement in VAR-based personalized image generation.
citing papers explorer
-
Naming the Concepts Classifiers Rely On: Language-Anchored Decomposition for Faithful Explanation
By fixing CLIP-derived language similarity maps as the coefficient matrix in non-negative matrix factorization, this method produces named, faithful concept explanations for frozen image classifiers.
-
FedXDS: Leveraging Model Attribution Methods to counteract Data Heterogeneity in Federated Learning
FedXDS uses propagation-based attribution to identify task-relevant features for selective data sharing in federated learning, yielding higher accuracy and faster convergence under heterogeneity with formal privacy guarantees.
-
Global Explanations for Multivariate Time Series Forecasting Models via $K$-Order Markov Approximations
KARMA constructs minimal-K Markov transition kernels as surrogates to deliver global explanations for multivariate time series forecasting models and recovers known causal structure on synthetic data.
-
Diffusion Integrated Gradients: Controllable Path Generation for Flexible Feature Attribution
DiffIG reformulates attribution path generation as conditional diffusion modeling trained on Stick-Breaking Process paths with guided sampling for user-controllable XAI.
-
In Defense of Information Leakage in Concept-based Models
Concept-based models can use controlled 'benign' information leakage to remain accurate and intervenable under real-world concept incompleteness by reframing their training objective.
-
Attribution via Distributional Paths for Information Revelation
Reveal-IG performs path attribution by integrating model output changes along trajectories in a space of probe distributions rather than input-space paths, retaining completeness and handling multiscale or uncertain features.
-
AIM: Adversarial Information Masking for Faithfulness Evaluation of Saliency Maps
AIM is a new saliency-guided adversarial feature replacement method to evaluate faithfulness of saliency maps and reliability of masking operators on image, audio, and EEG tasks.
-
AGOP as Explanation: From Feature Learning to Per-Sample Attribution in Image Classifiers
AGOP-based attribution methods outperform Integrated Gradients and other baselines on pixel-level ground truth benchmarks for explaining image classifier decisions, with AGOP-Global offering zero inference cost.
-
From Baselines to Transport Geodesics: Axiomatic Attribution via Optimal Generative Flows
Transport-geodesic attribution via optimal generative flows selects principled paths for feature attributions by minimizing kinetic action.
-
MobileMold: A Smartphone-Based Microscopy Dataset for Food Mold Detection
MobileMold provides 4941 smartphone microscopy images and shows deep learning models reach 99.5% accuracy on mold detection and food classification tasks.
-
Learning-Augmented Robust Algorithmic Recourse
Introduces learning-augmented robust algorithmic recourse that trades off consistency with accurate future-model predictions against robustness to inaccurate predictions via a novel algorithm.
-
Attributions All the Way Down? The Metagame of Interpretability
Defines meta-attributions as directional second-order Shapley values on attribution methods, proves hierarchical decomposition of attributions, and demonstrates applications in language models, vision-language encoders, and diffusion transformers.
-
From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Models
An iERF-centric framework unifies local, global, and mechanistic interpretability in vision models via SRD for saliency, CAFE for concept anchoring, and ICAT for interlayer attribution.
-
Low Rank Adaptation for Adversarial Perturbation
Adversarial perturbations possess an inherently low-rank structure that enables more efficient and effective black-box adversarial attacks via subspace projection.
-
Homogeneous Stellar Parameters from Heterogeneous Spectra with Deep Learning
A single end-to-end Transformer model unifies stellar labels from heterogeneous spectroscopic surveys into a self-consistent scale without post-hoc recalibration.
-
The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts
The Linear Centroids Hypothesis reframes network features as directions in centroid spaces of local affine experts, unifying interpretability methods and yielding sparser, more faithful dictionaries, circuits, and saliency maps.
-
Distributed Sparse Interventions in Language Models
Sparse interventions on 8–64 neurons distributed across layers can activate task behavior in instruction-tuned LLMs, outperforming first-order linear steering approaches by modeling nonlinear neuron interactions.
-
Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
Gradient saliency of teacher-forced token log-probabilities, decoded by dynamic programming, yields usable word alignments for every ASR family including speech LLMs.
-
CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry
CascadeFormer tapers Transformer width with depth based on gradient fan-in asymmetry to match uniform baselines in perplexity while cutting latency 8.6%.
-
Listening makes Vision Clear for VLMs
PV-TAM uses prompt-side semantics and a bias filter to improve attention-based and IoU localization metrics for vision-language models over answer-side baselines.
-
One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability
WorldModelLens defines a typed adapter with four core methods and a capability descriptor to unify interpretability tooling across diverse world model architectures.
-
Physics-Guided Dual Decoding and Spectral Supervision for Global 3D Hydrometeor Prediction
PredHydro-Net is a new dual-decoding architecture with wavelet spectral matching and adversarial training that outperforms Earthformer, PredRNNv2, and GFS on extreme-event detection and spectral fidelity in 72-hour global hydrometeor forecasts.
-
TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment
TEVI applies sparse autoencoders and caption-conditioned masking to edit image embeddings, yielding better retrieval on MS COCO, Flickr, IIW, DOCCI, and RoCOCO benchmarks with larger gains on richer captions.
-
CPC-VAR:Continual Personalized and Compositional Generation in Visual Autoregressive Models
CPC-VAR adds Gradient-based Concept Neuron Selection for continual single-concept learning and a context-aware multi-branch composition strategy to reduce forgetting and entanglement in VAR-based personalized image generation.
-
Instructions Shape Production of Language, not Processing
Instructions trigger a production-centered mechanism in language models, with task-specific information stable in input tokens but varying strongly in output tokens and correlating with behavior.
-
Causal Attribution via Activation Patching
CAAP produces patch attributions in ViTs by direct activation patching on intermediate layers to measure causal contribution to the target class score.
-
MaskDiME: Adaptive Masked Diffusion for Precise and Efficient Visual Counterfactual Explanations
MaskDiME uses adaptive masked diffusion to produce 30x faster, localized, and semantically consistent visual counterfactual explanations without training, matching or exceeding prior performance on five datasets.
-
Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models
CA-LIG is a unified hierarchical attribution method that computes layer-wise Integrated Gradients fused with class-specific attention gradients to generate signed, context-sensitive explanations for transformer models.
-
Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection
LiMA reformulates attribution as submodular subset selection and uses bidirectional greedy search to identify minimal important regions, reporting 36.3% better insertion and 39.6% better deletion scores than prior methods on eight foundation models.
-
Explaining Object Detectors via Collective Contribution of Pixels
A Shapley-value method with interaction terms that explains object detector decisions by capturing collective pixel contributions for localization and classification.
-
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
SalUn uses gradient-based weight saliency to achieve effective machine unlearning of data, classes, or concepts in image classification and generation, narrowing the gap to exact retraining.
-
Saliency-driven Word Alignment Interpretation for Neural Machine Translation
Saliency-driven interpretation methods reveal that NMT models learn word alignments of better quality than fast-align under force decoding and consistent with automatic tools under free decoding.
-
Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality
Scaling vision models by depth and parameter count does not consistently improve localisation-based explanation quality across architectures, datasets, and post-hoc methods; smaller models often perform comparably or better.
-
H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers
H-Sets detects higher-order feature interactions in image classifiers via Hessian-guided pair merging and attributes them with IDG-Vis to generate more interpretable saliency maps than existing marginal or coarse methods.
-
When Can We Trust Deep Neural Networks? Towards Reliable Industrial Deployment with an Interpretability Guide
A new reliability score computed from the IoU difference between class-specific and class-agnostic heatmaps, boosted by adversarial enhancement, detects false negatives in binary industrial defect detectors with up to 100% recall.
-
Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks
Token-level contrastive attribution yields informative signals for some LLM benchmark failures but is not universally applicable across datasets and models.
-
MONAI: An open-source framework for deep learning in healthcare
MONAI is a community-supported PyTorch framework that extends deep learning to medical data with domain-specific architectures, transforms, and deployment tools.
-
Few-class Fidelity: Evaluating Explanations of Real-conditions CNN classifiers with Optimized Perturbations
Introduces a perturbation-based fidelity metric tailored to few-class CNN classifiers for real-conditions XAI evaluation, tested on medical and natural imaging against human-centric metrics.
-
Testing Neural Networks via Bayesian-Guided Exploration of Decision Landscapes
BayesWarp is a testing framework combining saliency techniques and uncertainty-aware Bayesian optimization to discover diverse model failures on image classification tasks while preserving data proximity.
-
ExECG: An Explainable AI Framework for ECG models
ExECG is a Python framework providing Wrapper, Explainer, and Visualizer stages to unify XAI methods for ECG models and improve reproducibility.
-
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
The survey organizes mechanistic interpretability techniques into a Locate-Steer-Improve framework to enable actionable improvements in LLM alignment, capability, and efficiency.
-
Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
Attribution-guided pruning with contrastive relevance identifies behavior-specific circuits in small LLMs and removes as little as 0.03-0.3% of components to reduce toxicity or repetition while preserving general performance.
-
ELF: Embedded Localisation of Features in pre-trained CNN
ELF derives keypoint locations via gradients on pre-trained CNN feature maps and reaches repeatability and matchability scores comparable to specialized detectors on HPatches, Webcam, and photo-tourism data.
-
CAMAL: Improving Attention Alignment and Faithfulness with Segmentation Masks
CAMAL adds an auxiliary regularizer during training that aligns model attention with segmentation masks to improve both spatial accuracy and causal faithfulness of attention in deep learning and deep reinforcement learning vision models.
-
Path-Sampled Integrated Gradients
Path-sampled integrated gradients generalizes integrated gradients by averaging gradients over sampled baselines on the linear path, proving equivalence to a weighted version that improves convergence rate to O(m^{-1}) and reduces variance by a factor of 1/3 under uniform sampling.
-
Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making
This survey synthesizes XAI methods with surrogate modeling workflows for simulations and outlines a research agenda to embed explainability into simulation-driven design and decision-making.
-
Event-Level Detection of Surgical Instrument Handovers in Videos with Interpretable Vision Models
A ViT-LSTM spatiotemporal model detects surgical instrument handovers and classifies direction in videos, achieving F1 of 0.84 for detection and 0.72 mean F1 for direction on kidney transplant data.
-
PECKER: A Precisely Efficient Critical Knowledge Erasure Recipe For Machine Unlearning in Diffusion Models
Spline numerical encodings can match or beat standard scaling on tabular nets, but PLE is most robust for classification and learnable knots add substantial training cost.
-
Using Explainability as a Training-Time Reliability Signal for Efficient ECG Classification
ERTS applies explanation quality from Grad-CAM to select reliable samples in progressive data dropout for ECG classification, reporting higher macro-F1 and lower training cost across three datasets and multiple architectures.
-
Parameter Space Analysis through Guided Visual Interpolations
ParamInter is a guided visual interpolation tool for high-dimensional parameter space analysis that integrates t-SNE and XAI to support optimization, demonstrated on blast furnace modeling.