Abstraction-refinement framework with SHAP-guided timestep selection improves certified robustness verification success and margin tightness for RNNs over abstraction-only baselines.
hub
Axiomatic Attribution for Deep Networks
28 Pith papers cite this work. Polarity classification is still indexing.
28
Pith papers citing it
abstract
We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms---Sensitivity and Implementation Invariance that attribution methods ought to satisfy. We show that they are not satisfied by most known attribution methods, which we consider to be a fundamental weakness of those methods. We use the axioms to guide the design of a new attribution method called Integrated Gradients. Our method requires no modification to the original network and is extremely simple to implement; it just needs a few calls to the standard gradient operator. We apply this method to a couple of image models, a couple of text models and a chemistry model, demonstrating its ability to debug networks, to extract rules from a network, and to enable users to engage with models better.
hub tools
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 14 cs.CV 4 astro-ph.GA 2 cs.CL 2 astro-ph.IM 1 astro-ph.SR 1 cs.CR 1 cs.GR 1 cs.IR 1 eess.IV 1roles
background 1polarities
background 1representative citing papers
A training-free method using Fourier-parameterized star-convex contours optimized via gradients to generate compact, faithful visual attributions for image classifiers on benchmarks like ImageNet.
Neurons exhibit concept-conditioned activation ranges forming Gaussian-like distributions with minimal overlap, and range-based interventions via NeuronLens outperform neuron-level masking in targeted manipulation with reduced collateral effects.
A single end-to-end Transformer model unifies stellar labels from heterogeneous spectroscopic surveys into a self-consistent scale without post-hoc recalibration.
The C-Score quantifies intra-class explanation consistency for CAM methods via confidence-weighted pairwise soft IoU and detects AUC-consistency dissociation as an early warning for model instability on chest X-ray classification.
Data-similarity and data-influence produce significantly overlapping rankings of training documents for LLM outputs, with asymmetry allowing a favorable cost-accuracy trade-off.
TIRA attacks with PMiS and PRSMP push fairness metrics to ideal values and reduce SHAP attribution for protected features to zero in black-box settings.
XtrAIn shifts occlusion from input space to parameter space along the training trajectory to produce cleaner feature attributions than standard methods.
Physical plausibility is linearly decodable from diffusion transformer states in video models at 81.27% accuracy on IntPhys and InfLevel, absent from VAE latents and outperforming V-JEPA and VideoMAE.
Retraining all 31 subsets of five vision encoders shows Capacity and Necessity are distinct, pre-projector effective rank predicts residual performance at fixed parameter count, and high-Capacity plus adaptive complement pairs match the full five-encoder model.
A triplet-based plateau search algorithm is proposed to adaptively determine a near-minimal number of trees for random forests by monitoring relative OOB score changes across forest size triplets, removing n_trees from the TPE search space.
The paper proposes an architecture-aware explanation audit protocol demonstrating that perturbation-based faithfulness is bounded by structural compatibility between explainer and model readout rather than architecture family.
Reasoning models encode step importance in their activations pre-hoc, and this signal is more informative than tokens for identifying critical reasoning steps.
Rhamba uses region-aware masking strategies and hybrid Attention-Mamba models pretrained on ABIDE fMRI data to achieve top AUROC on schizophrenia and ADHD classification tasks while outperforming prior methods.
Counterfactual prompting effects on LLMs are often indistinguishable from those caused by meaning-preserving paraphrases, causing most previously reported demographic sensitivities to disappear under proper statistical comparison.
Cross-modal averaging maps ECG model attributions to CineECG 3D space, raising Dice overlap with expert annotations from 0.47 to 0.56 on 20 cases while filtering attribution noise.
TwinSpecNet uses empirical paired learning on spectral twins to denoise low-S/N APOGEE spectra and predict stellar parameters and abundances with lower scatter than the standard pipeline.
Empirical Jacobian analysis reveals that token influence in trained language models decays as a power law with distance (exponent ~0.8), a learned property not present in random models.
SIREN is a ~100k-parameter Transformer that detects SIRs with ROC-AUC 0.93 on held-out data and attributes 24% importance to proton density and 13-17% to transverse velocity, identifying flow deflection as a consistent signature.
Transformers trained on cosmic ray simulations learn physically plausible features in positional encodings for symmetric air showers and in attention mechanisms for galaxy-origin particles.
Researchers train AI detectors on a large photorealistic fake image dataset, apply 16 XAI methods, and use human survey feedback to assess alignment between machine explanations and human perception of AI-generated images.
CONFIDE applies conformal prediction to transformer embeddings for valid prediction sets, improving accuracy up to 4.09% and efficiency over baselines on models like BERT-tiny.
Hybrid RL-PID controllers track angle of attack better and show greater robustness than PID alone within a defined operational envelope for re-entry attitude control.
Spatial Learning Entropy Maps derived from MLP weight adaptations during spatial pixel prediction tasks highlight image points with high learning impact.
citing papers explorer
-
Robustness Verification of Recurrent Neural Networks with Abstraction Refinement
Abstraction-refinement framework with SHAP-guided timestep selection improves certified robustness verification success and margin tightness for RNNs over abstraction-only baselines.
-
Extremal Contours: Gradient-driven contours for compact visual attribution
A training-free method using Fourier-parameterized star-convex contours optimized via gradients to generate compact, faithful visual attributions for image classifiers on benchmarks like ImageNet.
-
Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution
Neurons exhibit concept-conditioned activation ranges forming Gaussian-like distributions with minimal overlap, and range-based interventions via NeuronLens outperform neuron-level masking in targeted manipulation with reduced collateral effects.
-
Homogeneous Stellar Parameters from Heterogeneous Spectra with Deep Learning
A single end-to-end Transformer model unifies stellar labels from heterogeneous spectroscopic surveys into a self-consistent scale without post-hoc recalibration.
-
Quantifying Explanation Consistency: The C-Score Metric for CAM-Based Explainability in Medical Image Classification
The C-Score quantifies intra-class explanation consistency for CAM methods via confidence-weighted pairwise soft IoU and detects AUC-consistency dissociation as an early warning for model instability on chest X-ray classification.
-
Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior
Data-similarity and data-influence produce significantly overlapping rankings of training documents for LLM outputs, with asymmetry allowing a favorable cost-accuracy trade-off.
-
The Unseen Hand: Manipulating Model Fairness and SHAP with Targeted Identity Re-Association Attacks
TIRA attacks with PMiS and PRSMP push fairness metrics to ideal values and reduce SHAP attribution for protected features to zero in black-box settings.
-
XtrAIn: Training-Guided Occlusion for Feature Attribution
XtrAIn shifts occlusion from input space to parameter space along the training trajectory to produce cleaner feature attributions than standard methods.
-
The Invisible Hand of Physics: When Video Diffusion Models Know More Than They Show
Physical plausibility is linearly decodable from diffusion transformer states in video models at 81.27% accuracy on IntPhys and InfLevel, absent from VAE latents and outperforming V-JEPA and VideoMAE.
-
Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs
Retraining all 31 subsets of five vision encoders shows Capacity and Necessity are distinct, pre-projector effective rank predicts residual performance at fixed parameter count, and high-Capacity plus adaptive complement pairs match the full five-encoder model.
-
How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration
A triplet-based plateau search algorithm is proposed to adaptively determine a near-minimal number of trees for random forests by monitoring relative OOB score changes across forest size triplets, removing n_trees from the TPE search space.
-
Architecture-Aware Explanation Auditing for Industrial Visual Inspection
The paper proposes an architecture-aware explanation audit protocol demonstrating that perturbation-based faithfulness is bounded by structural compatibility between explainer and model readout rather than architecture family.
-
Reasoning Models Know What's Important, and Encode It in Their Activations
Reasoning models encode step importance in their activations pre-hoc, and this signal is more informative than tokens for identifying critical reasoning steps.
-
Rhamba: Region-Aware Hybrid Attention-Mamba Framework for Self-Supervised Learning in Resting-State fMRI
Rhamba uses region-aware masking strategies and hybrid Attention-Mamba models pretrained on ABIDE fMRI data to achieve top AUROC on schizophrenia and ADHD classification tasks while outperforming prior methods.
-
Compared to What? Baselines and Metrics for Counterfactual Prompting
Counterfactual prompting effects on LLMs are often indistinguishable from those caused by meaning-preserving paraphrases, causing most previously reported demographic sensitivities to disappear under proper statistical comparison.
-
Validating the Clinical Utility of CineECG 3D Reconstructions through Cross-Modal Feature Attribution
Cross-modal averaging maps ECG model attributions to CineECG 3D space, raising Dice overlap with expert annotations from 0.47 to 0.56 on 20 cases while filtering attribution noise.
-
TwinSpecNet: Extending APOGEE's chemical reach to low-S/N spectra via empirical paired learning
TwinSpecNet uses empirical paired learning on spectral twins to denoise low-S/N APOGEE spectra and predict stellar parameters and abundances with lower scatter than the standard pipeline.
-
How Token Influence Decays with Distance: A Green-Function View of Trained Language Models
Empirical Jacobian analysis reveals that token influence in trained language models decays as a power law with distance (exponent ~0.8), a learned property not present in random models.
-
Finding Novel Precursors for Solar Wind Stream Interaction Regions with Interpretable Deep Learning
SIREN is a ~100k-parameter Transformer that detects SIRs with ROC-AUC 0.93 on held-out data and attributes 24% importance to proton density and 13-17% to transverse velocity, identifying flow deflection as a consistent signature.
-
What exactly did the Transformer learn from our physics data?
Transformers trained on cosmic ray simulations learn physically plausible features in positional encodings for symmetric air showers and in attention mechanisms for galaxy-origin particles.
-
AI-Generated Images: What Humans and Machines See When They Look at the Same Image
Researchers train AI detectors on a large photorealistic fake image dataset, apply 16 XAI methods, and use human survey feedback to assess alignment between machine explanations and human perception of AI-generated images.
-
Uncertainty-Aware Transformers: Conformal Prediction for Language Models
CONFIDE applies conformal prediction to transformer embeddings for valid prediction sets, improving accuracy up to 4.09% and efficiency over baselines on models like BERT-tiny.
-
Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry
Hybrid RL-PID controllers track angle of attack better and show greater robustness than PID alone within a defined operational envelope for re-entry attitude control.
-
Learning Entropy and Spatial Adaptation Dynamics of Multilayer Perceptrons for Structural Point Extraction
Spatial Learning Entropy Maps derived from MLP weight adaptations during spatial pixel prediction tasks highlight image points with high learning impact.
-
ClinQueryAgent: A Conversational Agent for Population Health Management
The paper introduces ClinQueryAgent, a conversational agent that converts natural language queries into database queries for population health management while keeping patient data secure, and reports its use by 128 staff across 15 NHS practices covering 148,319 patients.
-
A Giant-Step Baby-Step Classifier For Scalable and Real-Time Anomaly Detection In Industrial Control Systems and Water Treatment Systems
A Giant-Step Baby-Step Classifier for real-time anomaly detection in ICS via linearization of sensor-actuator interactions, achieving millisecond response and 97.72% accuracy on a water treatment testbed with built-in explainability.
-
Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data
Benchmark of local explainability methods on tabular data finds explanation quality driven primarily by dataset complexity rather than model predictive performance.
-
Platonic Projection Structures: Operator-Induced Observability in Representation Learning
The paper introduces 'Platonic Projection Structures,' a reformulation of standard PSD operator theory applied to representation learning, with experiments that verify definitions rather than test predictions.