REVIEW 39 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
RISE: Randomized Input Sampling for Explanation of Black-box Models
read the original abstract
Deep neural networks are being used increasingly to automate data analysis and decision making, yet their decision-making process is largely unclear and is difficult to explain to the end users. In this paper, we address the problem of Explainable AI for deep neural networks that take images as input and output a class probability. We propose an approach called RISE that generates an importance map indicating how salient each pixel is for the model's prediction. In contrast to white-box approaches that estimate pixel importance using gradients or other internal network state, RISE works on black-box models. It estimates importance empirically by probing the model with randomly masked versions of the input image and obtaining the corresponding outputs. We compare our approach to state-of-the-art importance extraction methods using both an automatic deletion/insertion metric and a pointing metric based on human-annotated object segments. Extensive experiments on several benchmark datasets show that our approach matches or exceeds the performance of other methods, including white-box approaches. Project page: http://cs-people.bu.edu/vpetsiuk/rise/
Forward citations
Cited by 39 Pith papers
-
Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability
Introduces Synergistic Faithfulness metric based on Shapley Interaction Index to evaluate cross-modal synergy in VLM explainers, revealing over-reliance on visual salience in existing methods.
-
How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence
ProSA auditing framework shows that structural loss metrics track OCR instability and downstream QA degradation far better than area-based footprint measures across two document parsers on 1,000 pages.
-
How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence
A new output-level auditing framework with B-SLR and exposure descriptors shows that structure-targeted perturbations better predict OCR instability and downstream degradation than footprint size in document parsers.
-
How to Evaluate and Refine your CAM
Introduces synthetic ground-truth dataset for CAM evaluation, proposes ARCC composite metric, and RefineCAM method that aggregates layers for higher-resolution maps outperforming baselines.
-
Architecture-Aware Explanation Auditing for Industrial Visual Inspection
An audit protocol on wafer maps finds that ViT-Tiny with Attention Rollout achieves better deletion faithfulness than other models and explainers, with readout structure as the key factor and RISE outperforming native...
-
Architecture-Aware Explanation Auditing for Industrial Visual Inspection
Explanation faithfulness for deep classifiers on wafer maps is highest when the explainer matches the model's native readout structure, with ViT-Tiny plus Attention Rollout achieving lower Deletion AUC than mismatched...
-
Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models
Introduces ISS and NMR as interventional metrics to diagnose causal misalignment in VLA policies and link it to generalization performance.
-
Adjoint Inversion Reveals Holographic Superposition and Destructive Interference in CNN Classifiers
CNN classifiers work by holographic superposition and destructive interference in pixel space rather than selecting cleaned features, as proven by a new adjoint inversion framework that also yields a covariance-volume...
-
From Baselines to Transport Geodesics: Axiomatic Attribution via Optimal Generative Flows
Transport-geodesic attribution via optimal generative flows selects principled paths for feature attributions by minimizing kinetic action.
-
TreeGrad-Ranker: Feature Ranking via $O(L)$-Time Gradients for Decision Trees
TreeGrad-Ranker produces feature rankings for decision trees by optimizing a joint insertion-deletion objective with O(L)-time gradients derived from the multilinear extension, outperforming probabilistic values like ...
-
CPG-PAD: Concept-Informed Prompts Guided Presentation Attack Detection
CPG-PAD uses XAI-derived visual concepts to guide prompt learning in VLMs, enabling better cross-domain generalization for presentation attack detection on nine benchmarks.
-
Validating Causal Abstraction Metrics on Simulated Complex Systems
Authors create a benchmark across discrete/continuous and static/dynamical systems and introduce the Causal Abstraction Error (CAE) metric that reliably distinguishes valid from invalid causal abstractions when it inc...
-
FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning
FedLAB organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context via federated semantic barycenter pre-training, improving performance by up to ...
-
GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosis
GRAPE augments prototype medical image classifiers with graph attention for co-occurrence, a mismatch safety check, and open-vocabulary anchoring to support incremental findings without retraining.
-
GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosis
GRAPE augments prototype medical image classifiers with graph attention for co-occurrence, a mismatch safety check, and open-vocabulary anchoring to support incremental addition of findings from single examples.
-
Partition-Guided Distance Saliency: Bridging Decision and Objective Spaces in Many-Objective Optimization
PGDS is a new explainable AI method for many-objective optimization that automates target selection via partitioning and identifies influential decision variables through distance-based sensitivity analysis.
-
XtrAIn: Training-Guided Occlusion for Feature Attribution
XtrAIn shifts occlusion from input space to parameter space along the training trajectory to produce cleaner feature attributions than standard methods.
-
Landseer: Exploring the Machine Learning Defense Landscape
Landseer offers a containerized modular system to integrate and evaluate combinations of machine learning defenses, with an initial analysis of 35 defenses highlighting replicability challenges.
-
Bridging the Disciplinary Gap in Explainable AI: From Abstract Desiderata to Concrete Tasks
The authors introduce a taxonomy with target, functional role, and mode of justification axes plus a framework that decomposes abstract XAI desiderata into concrete benchmarkable tasks via identified dependency structures.
-
OCCAM: Open-set Causal Concept explAnation and Ontology induction for black-box vision Models
OCCAM discovers open-set visual concepts, estimates causal contributions via object-level interventions on black-box vision models, and induces a global concept ontology from aggregated dataset evidence.
-
From Weight Perturbation to Feature Attribution for Explaining Fully Connected Neural Networks
XWP and XWP_c are novel attribution methods for FCNNs that estimate feature importance by perturbing attached weights to avoid added bias and out-of-distribution issues in occlusion approaches.
-
Architecture-Aware Explanation Auditing for Industrial Visual Inspection
The paper proposes an architecture-aware explanation audit protocol demonstrating that perturbation-based faithfulness is bounded by structural compatibility between explainer and model readout rather than architectur...
-
Evaluation Cards for XAI Metrics
The authors introduce the XAI Evaluation Card template to standardize how XAI evaluation metrics are defined, validated, and reported.
-
Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models
Interventional attribution via ISS and NMR diagnoses causal misalignment in VLA policies and predicts their generalization performance across manipulation tasks.
-
DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
DRAGON is a new benchmark with 11,664 annotated instances from six diagram QA datasets that requires models to localize visual evidence regions supporting their answers.
-
Explainable AI in Speaker Recognition -- Making Latent Representations Understandable
Speaker recognition networks form hierarchical clusters in latent space that can be matched to semantic classes using new HCCM algorithm and quantified by Liebig's score.
-
H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers
H-Sets detects higher-order feature interactions in image classifiers via Hessian-guided pair merging and attributes them with IDG-Vis to generate more interpretable saliency maps than existing marginal or coarse methods.
-
PhiNet: Speaker Verification with Phonetic Interpretability
PhiNet adds phonetic interpretability to speaker verification while matching the accuracy of standard black-box models on VoxCeleb, SITW, and LibriSpeech.
-
AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption
AttnTrace is an attention-weight-based context traceback method for LLMs that claims higher accuracy and efficiency than prior art like TracLLM while aiding prompt injection detection.
-
Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIP
Grad-ECLIP produces gradient-based visual and textual explanation heatmaps for CLIP by applying channel and spatial weights to token features instead of relying on sparse self-attention maps.
-
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
SalUn uses gradient-based weight saliency to achieve effective machine unlearning of data, classes, or concepts in image classification and generation, narrowing the gap to exact retraining.
-
Few-class Fidelity: Evaluating Explanations of Real-conditions CNN classifiers with Optimized Perturbations
Introduces a perturbation-based fidelity metric tailored to few-class CNN classifiers for real-conditions XAI evaluation, tested on medical and natural imaging against human-centric metrics.
-
Explainable AI in Speaker Recognition -- Attention Map Visualisation and Evaluation
The paper introduces Modified RISE-eval to evaluate GradCAM and LayerCAM attention maps on speaker recognition networks and reports distinct advantages for each method under different conditions.
-
EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers
EIVE reformulates decoder cross-attention in Detection Transformers to produce instance-specific saliency maps via cross-layer fusion and attention-aware training, matching post-hoc methods in quality while improving speed.
-
Learning Quantifiable Visual Explanations Without Ground-Truth
A perturbation-based metric for XAI quality that formalizes sufficiency and necessity, paired with an adapter trained via differentiable supervision to generate causal explanations on black-box models.
-
Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentation
An optimized KernelSHAP method for 3D medical image segmentation restricts computation to ROI and receptive fields, uses patch logit caching for 15-30% savings, and compares organ units versus supervoxels for clinical...
-
Explainable AI in Speaker Recognition -- Making Latent Representations Understandable
Hierarchical clustering of speaker recognition embeddings reveals that the network organizes representations by gender at upper levels and by gender-nationality conjunctions at lower levels, interpreted via a proposed...
-
FM-G-CAM: A Holistic Approach for Explainable AI in Computer Vision
FM-G-CAM extends Grad-CAM by fusing explanations across multiple top classes for holistic CNN prediction understanding and ships an open-source library.
-
Position: Genomic Model Research Must Move Beyond Anecdotal Evaluation of Interpretability Methods
Benchmarking on transcription factor binding shows that different interpretability methods often contradict each other, miss known motifs, and fail to match model behavior, so the field needs a systematic tiered valid...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.