REVIEW 36 cited by
Captum: A unified and generic model interpretability library for PyTorch
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Captum: A unified and generic model interpretability library for PyTorch
read the original abstract
In this paper we introduce a novel, unified, open-source model interpretability library for PyTorch [12]. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms, also known as feature, neuron and layer importance algorithms, as well as a set of evaluation metrics for these algorithms. It can be used for both classification and non-classification models including graph-structured models built on Neural Networks (NN). In this paper we give a high-level overview of supported attribution algorithms and show how to perform memory-efficient and scalable computations. We emphasize that the three main characteristics of the library are multimodality, extensibility and ease of use. Multimodality supports different modality of inputs such as image, text, audio or video. Extensibility allows adding new algorithms and features. The library is also designed for easy understanding and use. Besides, we also introduce an interactive visualization tool called Captum Insights that is built on top of Captum library and allows sample-based model debugging and visualization using feature importance metrics.
Forward citations
Cited by 36 Pith papers
-
Aligning Molecular Graph Explanations with Chemical Identity via InChIfied Invariants
InChIfied Invariants based on InChI achieve 99.62% identical representations for chemically equivalent molecular graphs versus 0.35% for standard Daylight invariants on one million PubChem molecules, while preserving ...
-
MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs
MetaBackdoor shows that LLMs can be backdoored using positional triggers like sequence length, enabling stealthy activation on clean inputs to leak system prompts or trigger malicious behavior.
-
Many-Shot CoT-ICL: Making In-Context Learning Truly Learn
Many-shot CoT-ICL functions as test-time learning when demonstrations are ordered for smooth conceptual progression rather than similarity, enabling a new selection method that improves reasoning performance.
-
Modeling Subjective Urban Perception with Human Gaze
Gaze data from eye-tracking carries predictive signals for subjective urban perception and improves accuracy when fused with image-based scene representations.
-
Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?
FASS benchmark shows post-hoc attributions remain unstable under geometric perturbations even after filtering for unchanged predictions, with Grad-CAM exhibiting the highest stability across ImageNet, COCO, and CIFAR-10.
-
What is Missing? Explaining Neurons Activated by Absent Concepts
DNNs encode concept absences that raise neuron activation; standard XAI misses them, but non-target attribution and minimization visualization uncover them and improve debiasing.
-
MobileMold: A Smartphone-Based Microscopy Dataset for Food Mold Detection
MobileMold provides 4941 smartphone microscopy images and shows deep learning models reach 99.5% accuracy on mold detection and food classification tasks.
-
Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs
Attribution-method rankings are architecture-dependent: CNN conclusions do not reliably transfer to Vision Transformers, and apparent CAM localization wins under bounding boxes are largely metric saturation.
-
Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering
A token-attribution metric (ASI) and attribution-guided steering are proposed for LLM sycophancy, with large in-sample reductions reported.
-
Scaling Time Series Classification via XAI-Driven Data Reduction
drXAI uses XAI attributions from a fast classifier to choose important channels/time points, achieving 80–90% data reduction with comparable classification accuracy.
-
Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts
An imputation-free transformer (NITROGEN) trained on ADNI generalizes with competitive discrimination and superior calibration to OASIS-3 and AIBL while quantifying missing-modality uncertainty.
-
STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Training
Normalized entropy locates trajectory-neglect steps; STAPO selectively optimizes them with a trajectory-aware reward and trajectory-independent penalty, beating group-based RL baselines on agent tasks.
-
Two Black Boxes, One Solver: Encoder Probing and Decoder Attribution for Neural Multi-Attribute VRP under Hard-Mask and Recourse Decoders
Graph encoders and recourse training improve latent constraint representation and actionable decoder explanations in neural MAVRP solvers; make-feasible counterfactuals arise from the training regime, not the mask.
-
Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
Prediction agreement between open and closed LLMs substantially overstates agreement on attributions and causal reasons.
-
One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability
WorldModelLens defines a typed adapter with four core methods and a capability descriptor to unify interpretability tooling across diverse world model architectures.
-
Human-Centered Benchmarking of Driver Monitoring Models
Proposes HCBF to benchmark four lightweight vision models on the MRL Eye Dataset across four dimensions, finding each excels in one area and that aggregate scores mask robustness vulnerabilities.
-
AIMing for Standardised Explainability Evaluation in GNNs: A Framework and Case Study on Graph Kernel Networks
AIM is a new evaluation framework for explainability in GNNs that combines accuracy, instance-level, and model-level measures, applied to graph kernel networks to create an improved model xGKN.
-
Many-Shot CoT-ICL: Making In-Context Learning Truly Learn
Many-shot CoT-ICL improves when demonstrations are ordered for smooth conceptual progression, with CDS delivering up to 5.42 percentage-point gains on math tasks using 64 examples.
-
Instructions Shape Production of Language, not Processing
Instructions trigger a production-centered mechanism in language models, with task-specific information stable in input tokens but varying strongly in output tokens and correlating with behavior.
-
Enabling Performant and Flexible Model-Internal Observability for LLM Inference
DMI-Lib delivers 0.4-6.8% overhead for offline batch LLM inference and ~6% for moderate online serving while exposing rich internal signals across backends, cutting latency overhead 2-15x versus prior observability baselines.
-
Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality
Scaling vision models by depth and parameter count does not consistently improve localisation-based explanation quality across architectures, datasets, and post-hoc methods; smaller models often perform comparably or better.
-
Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models
Hallucinations in diffusion models are driven by local intrinsic dimension instabilities on the manifold, which Intrinsic Quenching corrects by deflating it.
-
X-SYS: A Reference Architecture for Interactive Explanation Systems
X-SYS is a reference architecture for interactive explanation systems organized around STAR quality attributes and five service components, demonstrated via SemanticLens for vision-language models.
-
Interpreto: An Explainability Library for Transformers
Interpreto provides a unified, documented Python API for attribution and end-to-end concept-based explanation of HuggingFace classification and generation models.
-
Delta-XAI: A Unified Framework for Explaining Prediction Changes in Online Time Series Monitoring
Delta-XAI wraps existing XAI methods for online time series and introduces SWING to explain prediction changes while accounting for temporal dependencies.
-
Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts
An imputation-free transformer with masked and intersample attention predicts Alzheimer’s status and scores across cohorts with better calibration than tree ensembles.
-
Automated Solar Radio Burst Detection Using Deep Learning on Augmented e-Callisto Data
FlareSense, a ResNet detector trained on 304,750 e-Callisto spectrograms with SpecAugment and TimeWarp, reaches 93% precision and 73.15% recall, outperforming routine expert cataloging at matched precision.
-
A Preliminary Study on Explaining Risk of Code Changes using LLM-Based Prediction Models
Top-2 attention-highlighted hunks cover expert-labeled outage root causes 53.85% of the time while requiring review of 26.28% of changed lines.
-
Reliability, Faithfulness, and the Limits of Post-hoc Explanations of Opaque Scientific Models
Reliability and faithfulness of post-hoc explanations do not suffice to support claims about how a scientific phenomenon is structured.
-
Personalized Deep Learning for Short-Term Forecasting of Impending Atrial Fibrillation from Continuous Wearable ECG Signals
Fine-tuning a global DL model on patient-specific ECG segments raises AUROC for 5-minute AF prediction from 0.614 to 0.711 (ICENTIA11K) and 0.585 to 0.686 (MobiCARE).
-
OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models
OPTIMUS generates minimal and sufficient concept-based visual explanations for deep classifiers using prime implicant theory to enforce logical sufficiency and minimality.
-
Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text
eXTC learns a natural-language SOP via structured prompt optimization, distills it into a compact LM, and extends it with RL to deliver fast inference plus global rules and local traces while claiming benchmark gains ...
-
ExECG: An Explainable AI Framework for ECG models
ExECG is a Python framework providing Wrapper, Explainer, and Visualizer stages to unify XAI methods for ECG models and improve reproducibility.
-
Instructions Shape Production of Language, not Processing
Instructions primarily shape the production stage of language models rather than the processing stage, with task-specific information and causal effects stronger in output tokens than input tokens.
-
Predicting the thermodynamics in the chromosphere from the translation of SDO data into the IRIS$^{2}$ inversion results using a visual transformer model
A visual transformer model trained on IRIS inversions predicts chromospheric temperature and density from SDO data with correlations around 0.8 on 80% of test cases.
-
Functional Groups are All you Need for Chemically Interpretable Molecular Property Prediction
FGR encodes molecules as functional-group bit vectors, embeds them with an autoencoder, and reports competitive or better benchmark accuracy while enabling chemical attribution.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.