Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An attribution score shows which hidden concepts actually drive CLIP predictions.

desk verdict Solid, useful toolkit for component-level CLIP interpretability with an explicit derivation and honest experiments, but the headline faithfulness claim is slightly ahead of the evidence: the benchmark measures probed-class score, not the model's decision. read the letter →

arxiv 2505.20229 v1 pith:NHOAOUIM submitted 2025-05-26 cs.LG cs.AI

Maximilian Dreyer , Lorenz Hufe , Jim Berend , Thomas Wiegand , Sebastian Lapuschkin , Wojciech Samek This is my paper · ORCID
classification cs.LGcs.AI
keywords CLIPsparseautoencodersattributionpatchingLogitLensmechanisticinterpretabilityspuriouscorrelationstext-imageprobing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks which sparse, interpretable components inside a CLIP model actually change a text-image prediction, not just which concepts they encode. The authors adapt attribution patching to CLIP's latent space, producing an instance-wise Act×Grad score whose closed form shows what Logit Lens misses: component magnitude and a local correction for components already aligned with the output. Deletion and insertion experiments on ImageNet indicate that Act×Grad ranks components more faithfully than Logit Lens, energy-based scores, or random ordering. Combined with semantic alignment labels, the score automatically flags components that are highly relevant yet poorly aligned with expected concepts, and the paper reports hundreds of such components across CLIP variants, encoding polysemy, compound nouns, visual typography, and dataset artifacts. A melanoma-detection case study shows that a linear classifier on CLIP embeddings relies on a red-hue background cue, and that latent-space augmentation along that component direction improves robustness.

What carries the argument

The central object is the instance-wise attribution score Act×Grad, a latent-space Input×Gradient-style attribution-patching score $R_j(x,t)=a_j\,\partial y/\partial a_j$ evaluated at reference activation zero. For CLIP's final class-token layer the paper derives the closed form $$R_j = a_j\,\frac{\|v_j-\mu_{v_j}\|}{\|x-\mu_x\|}\,\frac{\|v_j\|}{\|x\|}\,\Bigl(\mathrm{LogitLens}_j(t)-\frac{v_j\cdot x}{\|v_j\|\|x\|}\,y(x,t)\Bigr),$$ which makes explicit two things Logit Lens omits: a magnitude factor with the activation $a_j$ and the dictionary-direction norm, and a subtraction term that discounts a component whose direction already agrees with the current embedding. The framework around this object uses a top-k sparse autoencoder ($k=64$, $30{,}000$ components) to supply interpretable directions, a semantic alignment score to label each one, and a z-score over reference relevance values to flag components whose relevance on a new sample is a statistical outlier.

What would settle it

A direct test would ablate components in reverse Act×Grad order; if the output drop is not minimized relative to deleting in Act×Grad order, the score is not ranking causal effect. A second check would measure the LayerNorm bias β on real CLIP models, since the closed-form derivation sets it to zero and never empirically verifies that approximation.

Watch

Extended reading notes

Core claim

The paper's central claim is that component-level explanations of CLIP must move from what a component encodes to how much it moves a prediction, and that latent-space attribution patching provides a faithful way to measure this. Its proposal, Act×Grad, is an Input×Gradient-style score $R_j(x,t)=a_j\,\partial y/\partial a_j$ evaluated at zero reference activation; for CLIP's final class-token layer the paper derives a closed form in which this becomes a magnitude-weighted Logit Lens score minus a correction term that discounts components already aligned with the current embedding. The authors argue that this correction is exactly why Act×Grad beats Logit Lens, activation-only Energy scores, and random ordering in deletion and insertion tests, and that combining the score with semantic alignment labels makes unexpected concept reliance discoverable at scale. Applied across CLIP variants, the framework surfaces hundreds of components encoding polysemous words, compound nouns, visual typography, and dataset artifacts, and a melanoma-detection case shows that linear classifiers on image embeddings amplify a red-hue background shortcut.

Load-bearing premise

The faithfulness evaluation assumes that zeroing out a component's activation and watching the output fall is a valid measure of that component's true causal influence, which holds only if attributions are locally linear and components act independently.

Editorial extensions

If this is right

  • Logit Lens alone cannot rank components faithfully for individual predictions, because it ignores activation magnitude and interaction with other components; Act×Grad supersedes it for instance-wise analysis.
  • Automatic outlier detection on per-component relevance scores will flag concepts absent from standard label sets, so the same pipeline can audit a CLIP model before deployment.
  • CLIP text embeddings are more robust to dataset-specific spurious correlations than linear classifiers trained on image embeddings, so the choice of probing head changes the failure profile.
  • Prompt enrichment improves robustness to visual typography but does little for spurious correlations and only partially resolves polysemy.
  • In melanoma detection, a standard linear probe on CLIP embeddings uses a red-hue background component as a shortcut; latent-space augmentation along that component direction reduces the shortcut and stabilizes accuracy when input color changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The formula depends only on the cosine output and the LayerNorm-then-project structure, so the same Act×Grad score should transfer to other contrastively trained multimodal encoders and to earlier layers, making the framework a general audit tool rather than a CLIP-specific one.
  • Because semantic labels come from one auxiliary model, some of the reported unexpected concepts may reflect that labeler's biases; a two-labeler consistency check would separate model-side spuriousness from annotation-side ambiguity.
  • The latent-space augmentation used for red hue suggests a general shortcut-mitigation recipe: any flagged spurious component defines a direction along which additional training data can be synthesized.
  • The stated assumption that the LayerNorm bias β is negligible is not empirically tested, so measuring β across CLIP variants would be a direct extension that tightens the approximation's validity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a framework for attributing predictions of CLIP models to sparse autoencoder (SAE) components. The central technical contribution is an instance-wise attribution score, Activation×Gradient (Act×Grad), derived in closed form in Eq. (29) and presented in Eq. (6). The paper claims that Act×Grad is more faithful than Logit Lens, activation×LogitLens, Energy, and random baselines, as measured by deletion and insertion experiments over the probed-class output score (Fig. 3). It combines these attributions with semantic alignment scores computed via Mobile-CLIP-S2 to automatically flag components that encode semantically unexpected or spurious concepts. The framework is applied to several CLIP variants and to a melanoma detection case study using WhyLesion-CLIP, where the authors identify and partially mitigate reliance on red-hue background artifacts. The paper also claims that CLIP text-embedding probing is more robust to spurious correlations than linear classifiers trained on image embeddings.

Significance. If the central claims hold, this is a useful step toward mechanistic interpretability of vision-language models: it provides a scalable, closed-form attribution method for SAE components, evaluates it against multiple baselines with uncertainty estimates, releases code, and demonstrates a concrete medical robustness application. The derivation in Appendix B is explicit, and the faithfulness experiments include deletion and insertion protocols across multiple model variants. The unexpected-concept discovery and the melanoma case study are potentially valuable for debugging spurious correlations. However, the strength of the headline faithfulness claim currently exceeds what the evaluation measures, and two other load-bearing points (the zero- LayerNorm-bias assumption and the construction of the failure-mode benchmarks from the same model being evaluated) need to be addressed before the paper's conclusions are fully supported.

major comments (4)
  1. [Sec. 5.1, Eq. (29), Fig. 3] The faithfulness experiments measure change in the cosine similarity y(x,t) to the probed class text embedding, not the model's zero-shot prediction (the argmax over class text embeddings) and not the classification accuracy used in the downstream case study. The claim that Act×Grad identifies components that 'drive a prediction' (Q1, Abstract, Sec. 5.1) is therefore stronger than the evidence supports. A component can lower the probed-class score without changing the predicted class, and a decision flip can occur with only a small change in the probed-class score. I recommend either narrowing the wording to 'drive the probed-class output score' or adding a decision-level faithfulness metric, e.g., the fraction of images whose predicted class changes when the top attributed components are deleted or reinserted.
  2. [Appendix B, Eqs. (19)–(29)] The closed-form Act×Grad derivation assumes that the LayerNorm bias β is zero. CLIP's final LayerNorm generally has an affine bias term, and the paper provides no empirical evidence that β is negligible for the models used. Since Eq. (6) is the foundation of the proposed method and is described as an adaptation of attribution patching, this unvalidated approximation is load-bearing. Please report the norm of β for the evaluated models or compare Eq. (29) against autograd-computed a_j ∂y/∂a_j on a sample of images; if they differ, Eq. (29) should be presented as an additional approximation rather than as the exact Activation×Gradient score.
  3. [Sec. 5.3, Fig. 5, App. D.1] The failure-mode benchmark sets are constructed from images that activate components flagged as outliers by the CLIP ViT-B/16 model itself (the z-score procedure in Sec. 4.2b with threshold 3.0). Using these same sets to compare CLIP text probing against linear classifiers makes the comparison asymmetric: the distractors are adversarially selected for the CLIP model, not for the linear classifiers. The claim that text embeddings are 'more robust to spurious correlations' than linear classifiers therefore goes beyond what the experiment demonstrates. The Neuhaus et al. distractors provide one independent subset, but a substantial part of the benchmark comes from the automatic flagging. Please either construct the benchmark independently, e.g., by flagging components for both model types, or restrict the claim to separating CLIP-identified distractors.
  4. [Sec. 5.4, Fig. 6] The melanoma case study attributes SAE components for a linear classifier trained on CLIP image embeddings, but the paper never specifies how Eq. (29), which is derived for CLIP's cosine similarity to a text embedding, is adapted to that linear head. The relevance percentages in Fig. 6 must come from some formula, but neither the main text nor Appendix E defines it. Please provide the exact attribution formula used for the linear classifier (for example, the exact linear attribution a_j w·v_j for the logit, or the corresponding gradient expression) and state whether the LayerNorm/projection assumptions of Appendix B still apply in that setting.
minor comments (5)
  1. [Throughout] The manuscript contains several typographical errors, including 'Prelimenaries' (Sec. 3), 'Througout' (Sec. 3.1), 'begging' (Appendix introduction), 'independt' (App. B.1), 'erstimated' (App. B.2), 'componens' (App. B.2), 'mimicks' (App. B.2), 'visal typography' (Fig. 14), and 'Chosing' (Sec. 4).
  2. [Sec. 3.3, Eq. (3)] The notation is inconsistent: the text says that after LayerNorm and projection 'we obtain the embedding v_k', but the Logit Lens score in Eq. (3) is written in terms of v_j. Please align the subscript notation.
  3. [Sec. 5.1] The main text refers to 'our proposed Act×Grad score as in Eq. (29)', but Eq. (29) appears only in Appendix B while the main-text derivation is Eq. (6). Please renumber or add an explicit cross-reference to Eq. (6) as well.
  4. [Fig. 3] The caption repeats 'AUC score:' and the curves do not display the SEM values that are reported in the legend. Consider plotting confidence bands or stating more explicitly that the SEM is reported only in the legend/text.
  5. [App. D.1] The description of the failure-mode discovery pipeline states that flagged components are 'manually reviewed', but it does not report how many flagged components were rejected, how many samples per component were used in the benchmark sets, or whether the manual review was blinded. This information is important for reproducibility of the benchmark construction.

Circularity Check

1 steps flagged · score 2.0 of 10

Act×Grad derivation is first-principles and benchmarked externally; the only partial circularity is that 'unexpected semantics' is defined by the Mobile-CLIP-S2 labeler used to detect it, a dependency the authors acknowledge.

  1. self definitional [Sec. 3.2 Eq. (2); Sec. 4.2a Eq. (7); Limitations]
    "For textual labeling of a componentj, we follow SemanticLens, which uses Mobile-CLIP-S2 [50] ... We then compute the alignment between average embedding ¯xj and each text embedding t ∈ Tof predefined labels ... sj(t) = ... We then select components that satisfy: Rj(x, t) ≥ τrel, s j(t) ≤ τalign ... Lastly, our semantic alignment relies on CLIP-Mobile-S2, which may itself be affected by spurious correlations or ambiguous language."

    The paper's 'unexpected concept reliance' is operationalized as low alignment under Mobile-CLIP-S2: labels are assigned by argmax over sj(t) (Eq. 2), and hidden/unexpected components are selected by the threshold sj(t) ≤ τalign (Eq. 7). The 'surprising components' catalog is therefore the direct output of the definition and of the chosen labeler, not an independent empirical discovery about the analyzed CLIP models. The authors' limitation statement acknowledges the labeler may itself encode spurious correlations or ambiguous language, but the quantitative 'hundreds of surprising components' claims in Secs. 5.2-5.3 still treat this self-defined expectation as ground truth.

full rationale

The core attribution contribution is not circular. Act×Grad is derived from first principles in App. B: R_j = a_j ∂y/∂a_j is the standard attribution-patching Taylor term, and the closed form follows by differentiating CLIP's cosine similarity through LayerNorm (with the stated β=0 assumption) and projection. The faithfulness evaluation in Sec. 5.1 is an external deletion/insertion benchmark against five other methods, and the ordering of methods is an empirical result, not a fitted consequence of the derivation. The paper cites several prior tools by overlapping authors (SemanticLens [12], CRP [1], CAV [38]), but these are used as implemented methods and not as uniqueness theorems, so self-citation is not load-bearing. The skeptical concern that the benchmark probes the probed-class score rather than the argmax decision is a validity gap, not a circularity, and therefore does not raise the circularity score. The one partial self-definitional loop is the 'unexpected semantics' discovery, which defines expectedness via Mobile-CLIP-S2 alignment and then selects low-alignment components as unexpected; this is acknowledged in the Limitations and does not affect the main attribution claim. Overall the central derivation is self-contained, so the circularity score is low.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The core freedom in the paper is the choice of SAE hyperparameters and detection thresholds, all of which shape which components are reported as unexpected. The attribution formula itself is derived with a stated approximation (β≈0) and is benchmarked, so it is not an invented entity. No new particles, forces, or external entities are postulated.

free parameters (7)
  • SAE top-k = 64
    The number of active components per embedding is chosen by the authors; the central analysis depends on this sparsity setting but no sensitivity analysis is provided.
  • SAE dictionary size = 30000
    The number of latent components is a hand-chosen hyperparameter; the discovered concepts and their interpretability could change with dictionary size.
  • Relevance threshold tau_rel
    Used in Eq. (7) to filter high-relevance components; value is not specified in the main text, so the reported counts of unexpected components depend on it.
  • Alignment threshold tau_align
    Used in Eq. (7) to classify components as semantically unexpected; threshold selection is not justified independently.
  • Activation threshold tau_act = 3.0
    Used to filter components with high average activation in Section 4.2a and Appendix C; this affects which components are labeled unexpected.
  • Z-score outlier threshold = 3.0
    Used in Eq. (8) to flag failure cases; this is a standard but arbitrary cutoff.
  • Latent augmentation strength alpha = 0.5
    Used in the robustness mitigation (Eq. 34); the improvement is reported only for this chosen value, with alternatives in the appendix.
assumptions (4)
  • domain assumption Sparse autoencoder reconstruction is accurate enough that components capture meaningful latent directions (Eq. 1).
    The whole analysis operates on SAE components of the final class token embedding; reconstruction error is not reported in the main text.
  • ad hoc to paper LayerNorm bias beta is negligible (Appendix B, beta=0).
    The closed-form Act×Grad formula in Eq. (6) relies on this assumption; no empirical check of its magnitude is given.
  • domain assumption First-order Taylor approximation adequately captures the effect of zeroing a component (Eq. 5).
    Attribution patching assumes the output change under ablation is well approximated by activation times gradient; the faithfulness experiments indirectly test this but not exhaustively.
  • domain assumption Mobile-CLIP-S2 text-image alignment provides a valid semantic ground truth for labels (Eq. 2).
    The paper acknowledges this model may itself have biases; the expected semantics are therefore model-dependent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance." pith.science (2026). https://pith.science/paper/NHOAOUIM

@misc{pith2026250520229,
  author       = {Pith},
  title        = {Pith review of: From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NHOAOUIM}},
  note         = {Machine review of arXiv:2505.20229}
}
read the original abstract

Transformer-based CLIP models are widely used for text-image probing and feature extraction, making it relevant to understand the internal mechanisms behind their predictions. While recent works show that Sparse Autoencoders (SAEs) yield interpretable latent components, they focus on what these encode and miss how they drive predictions. We introduce a scalable framework that reveals what latent components activate for, how they align with expected semantics, and how important they are to predictions. To achieve this, we adapt attribution patching for instance-wise component attributions in CLIP and highlight key faithfulness limitations of the widely used Logit Lens technique. By combining attributions with semantic alignment scores, we can automatically uncover reliance on components that encode semantically unexpected or spurious concepts. Applied across multiple CLIP variants, our method uncovers hundreds of surprising components linked to polysemous words, compound nouns, visual typography and dataset artifacts. While text embeddings remain prone to semantic ambiguity, they are more robust to spurious correlations compared to linear classifiers trained on image embeddings. A case study on skin lesion detection highlights how such classifiers can amplify hidden shortcuts, underscoring the need for holistic, mechanistic interpretability. We provide code at https://github.com/maxdreyer/attributing-clip.

Figures

Figures reproduced from arXiv: 2505.20229 by the authors.

Figure 1
Figure 1. A framework for interpreting CLIP via latent attributions. a) Sparse autoencoders (SAEs) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Methodological overview: a) For each SAE component, we collect its most activating [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Faithfulness evaluation of latent attributions on the ImageNet test set. We measure output [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Analysis of components extracted by SAEs in CLIP. a) Most latent components have [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Analysis of failure modes in text-image probing. We evaluate CLIP’s robustness to textual [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Melanoma detection case study. a) Among the most relevant SAE components for the [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Evaluating Faithfulness of attributions of different attribution methods under three evaluation [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Evaluating Faithfulness of attributions of different attribution methods when attribution [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Correlation between interpretability (clarity) and component activation metrics across [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Examples of unexpected concepts found in CLIP ViT-L/14 for different levels of inter [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Examples of unexpected concepts found in CLIP ViT-L/14 for different levels of inter [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Examples of sample sets where text-image probing led to false positives, corresponding to [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Examples of sample sets where text-image probing led to false positives, corresponding [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: A detailed plot variant of Fig. 5 for the analysis of failure modes in text-image probing. We [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: Robustness analysis of linear classifiers trained on WhyLesion-CLIP visual embeddings [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Attribution to Action: A Human-Centered Application of Activation Steering

    cs.AI 2026-04 unverdicted novelty 6.5 of 10

    Activation steering paired with attribution enables intervention-based debugging in vision models, as all 8 interviewed experts shifted to hypothesis testing, most trusted observed responses, and highlighted risks lik...

  2. Model Science: getting serious about verification, explanation and control of AI systems

    cs.AI 2025-08 conditional novelty 4.0 of 10

    Proposes 'Model Science' as a model-centric paradigm for AI with four pillars: verification, explanation, control, and interface.

Reference graph

Works this paper leans on

60 extracted references · 46 canonical work pages · cited by 2 Pith papers

  1. [1]

    Achtibat, M

    R. Achtibat, M. Dreyer, I. Eisenbraun, S. Bosse, T. Wiegand, W. Samek, and S. Lapuschkin. From attribution maps to human-understandable explanations through concept relevance propa- gation. Nature Machine Intelligence, 5(9):1006–1019, 2023

    work page 2023

  2. [2]

    Y . H. Ahn, H. B. Kim, and S. T. Kim. Www: a unified framework for explaining what where and why of neural networks by interpretation of neuron concepts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10968–10977, 2024

    work page 2024

  3. [3]

    S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015

    2015

  4. [4]

    Belrose, Z

    N. Belrose, Z. Furman, L. Smith, D. Halawi, I. Ostrovsky, L. McKinney, S. Biderman, and J. Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08112, 2023

    arXiv 2023

  5. [5]

    Bhalla, A

    U. Bhalla, A. Oesterling, S. Srinivas, F. Calmon, and H. Lakkaraju. Interpreting CLIP with sparse linear concept embeddings (spliCE). InThe Thirty-eighth Annual Conference on Neural Informa- tion Processing Systems, 2024. URL https://openreview.net/forum?id=7UyBKTFrtd

    work page 2024

  6. [6]

    Bloom and J

    J. Bloom and J. Lin. Understanding sae features with the logit lens. In AI Alignment Forum, pages 24–25, 2024

    work page 2024

  7. [7]

    Bricken, A

    T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2, 2023

    work page 2023

  8. [8]

    N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler, et al. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th international sy...

    work page 2017

Show all 60 references
  1. [9]

    I. S. I. Collaboration et al. Slice-3d 2024 challenge dataset. International Skin Imaging Collaboration, 10, 2024

    2024

  2. [10]

    Dorszewski, L

    T. Dorszewski, L. Tˇetková, R. Jenssen, L. K. Hansen, and K. K. Wickstrøm. From colors to classes: Emergence of concepts in vision transformers. arXiv preprint arXiv:2503.24071, 2025

    2025 arXiv

  3. [11]

    Dreyer, R

    M. Dreyer, R. Achtibat, W. Samek, and S. Lapuschkin. Understanding the (extra-) ordinary: Validating deep model decisions with prototypical concept-based explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 3491–3501, 2024

    2024

  4. [12]

    Dreyer, J

    M. Dreyer, J. Berend, T. Labarta, J. Vielhaben, T. Wiegand, S. Lapuschkin, and W. Samek. Mechanistic understanding and validation of large ai models with semanticlens. arXiv preprint arXiv:2501.05398, 2025

    2025 arXiv

  5. [13]

    Elhage, T

    N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022. 10 From What to How: Attributing CLIP’s Latent Components Reveals Unexpected Semanti...

    2022 arXiv

  6. [14]

    A. Fang, A. M. Jose, A. Jain, L. Schmidt, A. T. Toshev, and V . Shankar. Data filtering networks. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=KAk6ngZ09F

    2024

  7. [15]

    T. Fel, V . Boutin, L. Béthune, R. Cadène, M. Moayeri, L. Andéol, M. Chalvidal, and T. Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems, 36:54805–54818, 2023

    2023

  8. [16]

    T. Fel, A. Picard, L. Bethune, T. Boissin, D. Vigouroux, J. Colin, R. Cadène, and T. Serre. Craft: Concept recursive activation factorization for explainability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2711–2721, 2023

    2023

  9. [17]

    S. Y . Gadre, G. Ilharco, A. Fang, J. Hayase, G. Smyrnis, T. Nguyen, R. Marten, M. Wortsman, D. Ghosh, J. Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems, 36:27092–27112, 2023

    2023

  10. [18]

    Gandelsman, A

    Y . Gandelsman, A. A. Efros, and J. Steinhardt. Interpreting CLIP’s image representation via text- based decomposition. In The Twelfth International Conference on Learning Representations,

  11. [19]

    Gandelsman, A

    Y . Gandelsman, A. A. Efros, and J. Steinhardt. Interpreting the second-order effects of neurons in CLIP. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=GPDcvoFGOL

    2025

  12. [20]

    L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu. Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=tcsZt9ZNKD

    2025

  13. [21]

    Hernández-Pérez, M

    C. Hernández-Pérez, M. Combalia, S. Podlipnik, N. C. Codella, V . Rotemberg, A. C. Halpern, O. Reiter, C. Carrera, A. Barreiro, B. Helba, et al. Bcn20000: Dermoscopic lesions in the wild. Scientific data, 11(1):641, 2024

    2024

  14. [22]

    Huben, H

    R. Huben, H. Cunningham, L. R. Smith, A. Ewart, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=F76bwRSLeK

    2024

  15. [23]

    Iglewicz and D

    B. Iglewicz and D. C. Hoaglin. Volume 16: how to detect and handle outliers. Quality Press, 1993

    1993

  16. [24]

    Ilharco, M

    G. Ilharco, M. Wortsman, R. Wightman, C. Gordon, N. Carlini, R. Taori, A. Dave, V . Shankar, H. Namkoong, J. Miller, H. Hajishirzi, A. Farhadi, and L. Schmidt. Openclip, July 2021. URL https://doi.org/10.5281/zenodo.5143773

    2021 doi

  17. [25]

    Joseph, P

    S. Joseph, P. Suresh, E. Goldfarb, L. Hufe, Y . Gandelsman, R. Graham, D. Bzdok, W. Samek, and B. A. Richards. Steering clip’s vision transformer with sparse autoencoders. In Mechanistic Interpretability for Vision at CVPR 2025 (Non-proceedings Track), 2025

    2025

  18. [26]

    C. Kim, S. U. Gadgil, A. J. DeGrave, J. A. Omiye, Z. R. Cai, R. Daneshjou, and S.-I. Lee. Transparent medical image ai via an image–text foundation model grounded in medical literature. Nature Medicine, 2024. doi: 10.1038/s41591-024-02887-x. URL https://doi.org/10. 1038/s41591...

    2024 doi

  19. [27]

    Krizhevsky, I

    A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012

    2012

  20. [28]

    Kumar, S

    S. Kumar, S. Ghosh, S. Sakshi, U. Tyagi, and D. Manocha. Do vision-language models understand compound nouns? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Pape...

    2024

  21. [29]

    J. Li, D. Li, S. Savarese, and S. Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  22. [30]

    H. Lim, J. Choi, J. Choo, and S. Schneider. Sparse autoencoders reveal selective remapping of visual concepts during adaptation. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=imT03YXlG2

    2025

  23. [31]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019. 11 From What to How: Attributing CLIP’s Latent Components Reveals Unexpected Semantic Reliance

    2019

  24. [32]

    Marks, C

    S. Marks, C. Rager, E. J. Michaud, Y . Belinkov, D. Bau, and A. Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/f...

    2025

  25. [33]

    A. J. Muinonen-Martin, S. J. O’Shea, and J. Newton-Bishop. Amelanotic melanoma. BMJ (Clinical research ed.), 360:k826, 2018

    2018

  26. [34]

    N. Nanda. Attribution patching: Activation patching at industrial scale. URL: https://www. neelnanda. io/mechanistic-interpretability/attribution-patching, 2023

    2023

  27. [35]

    Neuhaus, M

    Y . Neuhaus, M. Augustin, V . Boreiko, and M. Hein. Spurious features everywhere-large- scale detection of harmful spurious features in imagenet. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20235–20246, 2023

    2023

  28. [36]

    B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research, 37(23):3311–3325, 1997

    1997

  29. [37]

    M. Pach, S. Karthik, Q. Bouniot, S. Belongie, and Z. Akata. Sparse autoencoders learn monosemantic features in vision-language models. arXiv preprint arXiv:2504.02821, 2025

    2025

  30. [38]

    Pahde, M

    F. Pahde, M. Dreyer, M. Weckbecker, L. Weber, C. J. Anders, T. Wiegand, W. Samek, and S. Lapuschkin. Navigating neural space: Revisiting concept activation vectors to overcome directional divergence. In The Thirteenth International Conference on Learning Representations,

  31. [39]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021

    2021

  32. [40]

    S. Rao, S. Mahajan, M. Böhle, and B. Schiele. Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, pages 444–461. Springer, 2024

    2024

  33. [41]

    Ridnik, E

    T. Ridnik, E. Ben-Baruch, A. Noy, and L. Zelnik-Manor. Imagenet-21k pretraining for the masses. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. URL https://openreview.net/forum?id=Zkj_ VcZ6ol

    2021

  34. [42]

    Rotemberg, N

    V . Rotemberg, N. Kurtansky, B. Betz-Stablein, L. Caffery, E. Chousakos, N. Codella, M. Com- balia, S. Dusza, P. Guitera, D. Gutman, et al. A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific data, 8(1):34, 2021

    2021

  35. [43]

    Schuhmann, R

    C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021

    2021 arXiv

  36. [44]

    Schuhmann, R

    C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35: 25278–25294, 2022

    2022

  37. [45]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017

    2017

  38. [46]

    Shrikumar, P

    A. Shrikumar, P. Greenside, and A. Kundaje. Learning important features through propagating activation differences. In International conference on machine learning , pages 3145–3153. PMlR, 2017

    2017

  39. [47]

    A. Syed, C. Rager, and A. Conmy. Attribution patching outperforms automated circuit discovery. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 407–416, 2024

    2024

  40. [48]

    Thasarathan, J

    H. Thasarathan, J. Forsyth, T. Fel, M. Kowal, and K. Derpanis. Universal sparse autoencoders: Interpretable cross-model concept alignment. arXiv preprint arXiv:2502.03714, 2025

    2025

  41. [49]

    Tschandl, C

    P. Tschandl, C. Rosendahl, and H. Kittler. The ham10000 dataset, a large collection of multi- source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1–9, 2018. 12 From What to How: Attributing CLIP’s Latent Components Reveals Unexpected Semantic Reliance

    2018

  42. [50]

    P. K. A. Vasu, H. Pouransari, F. Faghri, R. Vemulapalli, and O. Tuzel. Mobileclip: Fast image-text models through multi-modal reinforced training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15963–15974, 2024

    2024

  43. [51]

    J. Vig, S. Gehrmann, Y . Belinkov, S. Qian, D. Nevo, Y . Singer, and S. Shieber. Investigating gender bias in language models using causal mediation analysis.Advances in neural information processing systems, 33:12388–12401, 2020

    2020

  44. [52]

    Q. Wang, Y . Lin, Y . Chen, L. Schmidt, B. Han, and T. Zhang. A sober look at the robustness of CLIPs to spurious features. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=wWyumwEYV8

    2024

  45. [53]

    Y . Yang, M. Gandhi, Y . Wang, Y . Wu, M. S. Yao, C. Callison-Burch, J. Gee, and M. Yatskar. A textbook remedy for domain shifts: Knowledge priors for medical image analysis. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. URL https://op...

    2024

  46. [54]

    T. Yun, U. Bhalla, E. Pavlick, and C. Sun. Do vision-language pretrained models learn composable primitive concepts? Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=YwNrPLjHSL

    2023

  47. [55]

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. InThe Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=1tZbq88f27. 13 From What to ...

    2024

  48. [58]

    Mean activation on the top-5 most activating images per component,

  49. [59]

    Average activation over the entire dataset,

  50. [60]

    object”, “an image of a object

    Firing rate, defined as the number of non-zero activations per component. All measurements are now computed on the ImageNet-1k test set (results in the main manuscript were performed on the train set). Results for ViT-L/14, ViT-H/14, ViT-B/16, and ViT-B/32 models are shown in ...

    2019

  51. [2024]

    URL https://openreview.net/forum?id=5Ca9sSzuDp

  52. [2025]

    URL https://openreview.net/forum?id=Q95MaWfF4e

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.