REVIEW 17 cited by
Interpreting CLIP's Image Representation via Text-Based Decomposition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We investigate the CLIP image encoder by analyzing how individual model components affect the final representation. We decompose the image representation as a sum across individual image patches, model layers, and attention heads, and use CLIP's text representation to interpret the summands. Interpreting the attention heads, we characterize each head's role by automatically finding text representations that span its output space, which reveals property-specific roles for many heads (e.g. location or shape). Next, interpreting the image patches, we uncover an emergent spatial localization within CLIP. Finally, we use this understanding to remove spurious features from CLIP and to create a strong zero-shot image segmenter. Our results indicate that a scalable understanding of transformer models is attainable and can be used to repair and improve models.
Forward citations
Cited by 17 Pith papers
-
Multimodal Model Diffing for Feature Discovery and Control
By diffing base-language and multimodal sparse autoencoder features, MMDiff isolates causally relevant features that can be ablated or steered to control spatial, OCR, and safety behaviors in multimodal LLMs.
-
ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features
ConceptAttention shows that linear projections in the output space of DiT attention layers yield sharper concept-localizing saliency maps than cross-attention maps, reaching state-of-the-art zero-shot segmentation.
-
Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models
A variance-based, retraining-free pruning framework for vision-language models that allocates per-layer sparsity and outperforms Wanda and SparseGPT at high sparsity.
-
BrainExplore: Large-Scale Discovery of Interpretable Visual Representations in the Human Brain
A new automated pipeline decomposes fMRI activity into components and labels them with visual concepts, claiming thousands of interpretable patterns across the human visual cortex.
-
The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail
Reliable concept presence in transformers is concentrated in the extreme high-activation tail of in-concept tokens; thresholding that tail improves concept detection and localization.
-
How Visual Representations Map to Language Feature Space in Multimodal LLMs
Visual tokens in a fully frozen-backbone VLM with a linear adapter only become well-represented by the LLM's sparse autoencoder features in middle-to-late layers, converging around layer 18.
-
Debiasing CLIP: Interpreting and Correcting Bias in Attention Heads
Using wrong/correct hard-sample head comparisons, LTC finds spurious CLIP attention heads and corrects them to raise worst-group accuracy on biased benchmarks.
-
Understanding Design Fixation in Generative AI
Generative AI models exhibit a design fixation phenomenon that limits the diversity and originality of their design outputs, according to a small lab study and a proposed theoretical framework.
-
Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models
Padding tokens in text-to-image models can carry semantic information or act as diffusion-time registers, depending on training and attention architecture.
-
Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights
A gradient alignment score against a reference model's weights selects and reweights training samples, improving data efficiency in image classification and CLIP pretraining.
-
Cross-Attention Head Position Patterns Can Align with Human Visual Concepts in Text-to-Image Generative Models
Head Relevance Vectors, built by counting which attention head best matches each of 34 concept words, reveal concept-specific cross-attention head patterns and enable targeted concept steering in Stable Diffusion.
-
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering
The paper shows LLaVA's visual QA mechanism parallels textual QA: visual embeddings encode animal and color features, attention heads extract and match them, and visual instruction tuning refines existing Vicuna heads.
-
MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
MEDIC-AD adds anomaly-aware and difference tokens to a medical VLM, claiming SOTA lesion detection, temporal tracking, and visual grounding; the zero-shot claim is undermined by likely train/test overlap.
-
Discovering Hidden Visual Concepts Beyond Linguistic Input in Infant Learning
Neuron dissection of the infant-trained CVCL model claims to find visual concepts beyond its training vocabulary, but the evaluation leaks test data into neuron selection.
-
Model Science: getting serious about verification, explanation and control of AI systems
Proposes 'Model Science' as a model-centric paradigm for AI with four pillars: verification, explanation, control, and interface.
-
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future
A historical review that organizes multimodal explainability methods into four chronological eras and three explainability types, extending coverage to generative LLMs.
-
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey
A survey maps the field of MLLM explainability and interpretability into data, model, and training and inference perspectives.
Discussion (0). Continue with ORCID to comment.