Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models
Reviewed by Pithpith:RB4XBSXUopen to challenge →
read the original abstract
Medical image classification is a critical problem for healthcare, with the potential to alleviate the workload of doctors and facilitate diagnoses of patients. However, two challenges arise when deploying deep learning models to real-world healthcare applications. First, neural models tend to learn spurious correlations instead of desired features, which could fall short when generalizing to new domains (e.g., patients with different ages). Second, these black-box models lack interpretability. When making diagnostic predictions, it is important to understand why a model makes a decision for trustworthy and safety considerations. In this paper, to address these two limitations, we propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model. We systematically evaluate our method on eight medical image classification datasets to verify its effectiveness. On challenging datasets with strong confounding factors, our method can mitigate spurious correlations thus substantially outperform standard visual encoders and other baselines. Finally, we show how classification with a small number of concepts brings a level of interpretability for understanding model decisions through case studies in real medical data.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
When Interpretability Becomes a Liability: Adversarial Attacks on CBM Concept Layers
Concept-level adversarial attacks exploit CBM interpretability on the CUB dataset, but SPECTRA raises required perturbation norm from 0.46 to over 4200 while keeping accuracy loss under 2.2%.
-
TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessment
TRACE is a RANO 2.0-aligned concept bottleneck model for 4-class glioblastoma response classification on longitudinal 3D MRI that reports 0.4769 macro F1 on the LUMIERE dataset via 5-fold patient-wise cross-validation.
-
TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessment
TRACE is a RANO 2.0-aligned concept bottleneck model for 4-class longitudinal glioblastoma response classification on 3D MRI that reports 0.4769 macro F1 on the LUMIERE dataset via 5-fold patient-wise cross-validation.
-
Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models
Introduces synthetic benchmarks for concept bottleneck models that control data modality, concept choice, annotation quality, and completeness to evaluate performance in decision support and automation.
-
Mitigating Label Bias with Interpretable Rubric Embeddings
Rubric embeddings from expert criteria mitigate label bias in models trained on historical evaluations, reducing group disparities while improving cohort quality on a master's program dataset.
-
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
Sparse feature circuits are introduced as interpretable causal subnetworks in language models, supporting unsupervised discovery of thousands of circuits and a method called SHIFT to improve classifier generalization ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.