Pith. sign in

REVIEW 4 major objections 4 minor 99 references

Multimodal Model Diffing for Feature Discovery and Control

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding vision to a language model rewrites a small set of internal features, and those rewritten features act as specific control handles: removing one degrades a targeted skill while general question answering is left intact.

desk verdict A new diffing pipeline with careful controls, but the headline effect sizes are best-of-feature numbers on feature-selected subsets and need to be reported as such. read the letter →

arxiv 2608.09928 v1 pith:P25W5XUJ submitted 2026-08-10 cs.CV cs.AIcs.CLcs.LG

Hunar Batra , Lachin Naghashyar , Ashkan Khakzar , Philip Torr , Christian Schroeder de Witt , Constantin Venhoff , Ronald Clark This is my paper · ORCID
classification cs.CVcs.AIcs.CLcs.LG
keywords multimodalmodeldiffingsparseautoencodersfeature-levelinterpretabilityactivationsteeringsafetyspatialreasoningOCRcausalablation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MMDiff is a pipeline that turns a multimodal AI's internal feature dictionary into a set of behavior-specific control handles. The paper's central claim is that the features a language model rewrites when it is adapted to see images — found by training a sparse autoencoder on the multimodal model, warm-started from the text backbone's own dictionary, and then diffing the two dictionaries — are exactly the features that carry specific visual behaviors. Removing one such feature direction at inference drops target-behavior accuracy by an average of 12% on spatial tasks and 17% on OCR, and cuts multimodal attack success rate by 24%, while leaving generic visual question answering essentially unchanged. Steering the same directions at their home layer beats a standard single-layer steering baseline by +3.6% spatial and +1.8% OCR. If right, this makes sparse autoencoders a practical interface for auditing, steering, and controlling multimodal behavior rather than just a post-hoc explanation tool.

What carries the argument

The central object is the MMDiff pipeline, a three-stage filter that turns two SAE dictionaries into control handles. The load-bearing pieces are: (i) a text-only warm-started SAE trained on the frozen MLLM's text-token activations, which preserves the base-LM feature basis so that same-index feature comparisons stay meaningful; (ii) the adapted-feature filter, defined by visual energy $E_v(f)$ above a threshold together with decoder cosine $c_f$ in the bottom 25%, which isolates the roughly 5–20% of features that multimodal training actually rewrote; (iii) per-token contrastive firing screened by a Fisher exact test (odds ratio $\geq 3$, firing-frequency gap $\Delta p \geq 0.05$) plus a neutral-prompt lexical-invariance filter, which extracts the task-specific subset; and (iv) two intervention primitives — three-point all-layer orthogonal projection for causal removal, and MMDiff-CAA steering, which injects the feature's decoder direction at its feature-associated layer alongside multi-layer CAA directions.

What would settle it

Ablate the same discovered spatial and OCR features but score on the complete VSR and OCRBench benchmarks instead of the per-feature top-activating subsets; if the average deltas collapse toward zero, or concentrate on a handful of near-duplicate niche samples, the selectivity claim is an artifact of the evaluation subsets. A complementary check steers a feature selected on one spatial dataset and tests it on a different spatial benchmark the feature never saw, which would reveal whether the feature encodes the behavior or the dataset.

Watch

Extended reading notes

Core claim

The paper claims that the difference between a base language model's feature dictionary and its multimodal-adapted counterpart is the right discovery signal for multimodal behavior. MMDiff warm-starts a multimodal SAE from the base-LM SAE, then selects features whose decoder directions rotate most under adaptation (bottom quartile of cosine similarity) while becoming visually responsive (positive visual energy), and further narrows this adapted set by per-token contrastive firing between a target distribution — spatial, OCR, or unsafe prompts — and a generic VQA baseline, followed by a lexical-invariance filter. The surviving sets are sparse: roughly 700 to 1,400 features out of dictionaries of hundreds of thousands to a million. Projecting a single discovered direction out of the residual stream at text-token positions degrades the target behavior by 6–31% per feature across three model families (means of −10.1, −12.3 and −14.6% on spatial tasks, −16.9% on OCR), with VQA spillover at or below 1.5%; safety features cut attack success by 17–28% per category with no measurable cost on benign controls. Steering the same directions together with multi-layer contrastive activation addition improves over vanilla single-layer steering, supporting the paper's conclusion that multimodal SAEs can function as control interfaces, not merely interpretability tools.

Load-bearing premise

The causal-effect numbers assume that a feature's importance on the samples where it fires hardest measures its importance for the whole behavior, because the headline 12% and 17% averages are computed on per-feature subsets built from each feature's top-activating samples rather than on the full benchmarks.

Editorial extensions

If this is right

  • A single sparse feature direction can carry substantial causal weight for a specific behavior: per-feature removal drops spatial accuracy by 6–31% and OCR category accuracy by up to 28%, with $|\Delta\mathrm{VQA}| \leq 1.5\%$ across all three model families.
  • Safety features found by contrastive firing reduce VLSBench attack success rate by 17–28% per category, with a mean of −9.67% over 1,061 candidates and essentially unchanged benign controls, offering a feature-level defense handle against image-grounded jailbreaks.
  • Cross-stage ablation on PaliGemma 2 shows spatial feature effects amplify roughly 3× after instruction tuning, and two features reverse sign, indicating that these spatial behaviors are acquired during multimodal fine-tuning rather than inherited from the pretrained model.
  • MMDiff-CAA's steering gain decomposes into comparable contributions from moving CAA to the feature's discovered layers (+1.82) and injecting the feature's decoder direction (+1.81) on top of vanilla single-layer CAA (+8.96).
  • Only the target distribution changes between applications, so the same recipe can be pointed at new behaviors and new MLLM families without per-domain retuning, as the paper itself argues in its conclusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper reports deltas on per-feature subsets built from each feature's top-activating samples; measured on complete VSR and OCRBench, the average effect is likely smaller, so a deployment-grade estimate of control strength should re-run the ablations on the full benchmarks. This is an editorial inference about the evaluation protocol, not a paper claim.
  • The attribution-patching result — driving attention heads cluster near a feature's home layer — suggests a mechanistic explanation for why layer-targeted steering works, and implies a testable predictor: features whose driving heads are more tightly co-located with the feature's home layer should steer more effectively.
  • The diffing recipe does not depend on the LM-to-MLLM transition being special; applying the same diff across other adjacent training stages (base to instruction-tuned, instruction-tuned to safety-tuned) would localize when each behavior is acquired, turning MMDiff into a training-stage audit tool.
  • Image counterfactuals show OCR features lose 36.8% of activation when the image is blanked; the natural stress test is whether the safety features survive adversarially constructed images, or whether attackers can re-elicit unsafe behavior through features outside the adapted set.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript introduces MMDiff, a pipeline that trains multimodal SAEs warm-started from base-LM SAEs, identifies features whose decoder directions rotate and that prefer visual input, and then applies per-token contrastive firing analysis to isolate task-specific features for spatial reasoning, multimodal safety, and OCR. The discovered features are intervened on by projection ablation and by a combined multi-layer CAA plus decoder-direction steering method, evaluated across LLaVA-MORE, PaliGemma 2, and InternVL3.5-2B. The central claims are that feature-level removal selectively degrades target behaviors by 12% on spatial tasks and 17% on OCR, reduces attack success rate by 24% on multimodal safety attacks, and that steering improves spatial and OCR accuracy over a single-layer CAA baseline.

Significance. If the headline results held at the level claimed, MMDiff would be a valuable contribution: it combines model diffing with SAE-based feature discovery for MLLMs, and it provides feature-level handles for both causal analysis and control. The paper has notable strengths, including multiple control conditions (random-feature ablation, a from-scratch SAE control, VQA spillover checks, and benign control sets), a cross-stage ablation on pretrained versus instruction-tuned checkpoints, image-counterfactual diagnostics, and explicit matching checks for feature correspondence across dictionaries. These controls support the qualitative conclusion that the diffing-based selection carries information beyond random or from-scratch alternatives. However, the headline causal-effect sizes are computed on outcome-ranked top features evaluated on feature-specific subsets, so the reported magnitudes do not yet support the benchmark-level selectivity claims made in the abstract.

major comments (4)
  1. [§5.1, Table 1; Abstract] The abstract's "average of 12% on spatial tasks" is not an average over MMDiff-discovered features on a full spatial benchmark. It is the mean over the top ten features per model ranked by ΔVSR, and each feature is scored on a VSR subset constructed from that feature's top-activating samples. This is a best-of-feature effect on feature-specific subsets, so the claimed "selectively degrades target behaviors" is substantially weaker than the headline suggests. Please report the mean ablation effect over the full discovered feature set and on the full VSR benchmark, and qualify the abstract accordingly.
  2. [§5.2, Table 4; Abstract] The "24% reduction in attack success rate" reported in the abstract and introduction is the mean over the single best feature per VLSBench category, whereas the same section reports a mean ΔASR of −9.67% over the full sweep of 1,061 candidate safety features. Both numbers appear in the text, but the headline selects the per-category best-case figure. The abstract should report the full-sweep mean, or at minimum present the 24% figure explicitly as the per-category top-feature result.
  3. [§5.3, Tables 5–6] The OCR results are based on five features, with ΔCat measured on each feature's own OCRBench category subset and steering gains measured on the same five features. The means of −16.9% for ablation and +1.8% for steering are therefore small-sample, feature-specific-subset numbers rather than full-benchmark results. To support the benchmark-level claim, the paper should report full OCRBench ablation results and the distribution of effects over the 1,070 discovered OCR-selective features.
  4. [§3.3, §5.1] The selection procedure uses the target distribution: features are retained because they fire more on D_tgt than on D_base, and the causal effect is then measured on subsets or categories of the same target distribution. This creates a structural correlation between selection and evaluation that inflates effect sizes. The random-feature and from-scratch-SAE controls show that selection beats chance, but they do not quantify the gap between top-feature/subset effects and all-feature/benchmark effects. I request an evaluation of the selected features on a held-out full benchmark and a report of the effect distribution over the whole discovered set.
minor comments (4)
  1. [§3.2] The main text does not give the numerical values of the visual-energy threshold ε and the cosine percentile pcos; they appear only in appendix figures. Please state the chosen values in the main text for reproducibility.
  2. [§5.1, Table 1 caption] The caption describes the features as "Top spatial SAE features ranked by ΔVSR." It should clarify whether this ranking is part of the MMDiff discovery pipeline or a post hoc outcome-based selection, since outcome-based ranking can inflate the reported deltas.
  3. [Table 3] The sentence "Non-improvements correspond to features already strongly encoded before fine-tuning or resistant to amplification" is a post hoc explanation without supporting evidence; please either provide data for it or remove it.
  4. [§8] The limitation that safety and OCR are evaluated on PaliGemma 2 only should be reflected in the abstract and introduction, which currently present the results with cross-model generality.

Circularity Check

2 steps flagged · score 4.0 of 10

Core diffing pipeline is self-contained, but the headline causal-effect averages are outcome-ranked and subset-selected, inflating the reported 12%/17%/24% figures.

  1. fitted input called prediction [Sec. 5.1, Table 1 caption and evaluation protocol; Abstract]
    "Top spatial SAE features ranked by ∆VSR; ... Ablating top spatial features lowers VSR accuracy by 6–31%, with means of −10.1, −12.3 and −14.6% ... each feature is scored on a VSR subset constructed from its top-activating samples"

    The abstract's "average of 12% on spatial tasks" is the mean of Table 1's ΔVSR column, and the rows of that column are selected by ranking on ΔVSR itself. Averaging the top-k values of the outcome variable guarantees a large negative mean even if most MMDiff features have small or zero causal effects. In addition, each feature is evaluated on a VSR subset built from its own top-activating samples, so the "target behavior" is defined by the feature's firing distribution; removing the feature from the residual stream then changes that subset almost by construction. The ΔVQA and ΔCtrl controls show no spillover, but they do not quantify the gap between top-feature/subset effects and typical-feature/full-benchmark effects.

  2. fitted input called prediction [Sec. 5.2, Table 4 and sweep paragraph; Abstract]
    "for each of the six VLSBench categories we identify the top unsafe feature in the adapted set A and ablate it. ... Each top feature reduces VLSBench ASR by 17–28% ... Across a sweep of 1,061 candidate safety features, the mean effect is ∆ASR = −9.67 %"

    The abstract's 24% ASR reduction is the average of the six per-category top-feature deltas in Table 4, i.e., the best feature within each category selected after seeing the effect. The paper discloses that the mean over all 1,061 safety candidates is only −9.67%, so the 24% headline is a selected-extreme statistic, not the typical MMDiff safety feature's effect. Since the selection criterion (per-category ASR drop) is the same variable being summarized, the headline number is statistically forced by the choice of top features.

full rationale

The MMDiff derivation chain—warm-started SAE adaptation, adapted-feature filter (visual energy + decoder rotation), contrastive firing against a generic VQAv2 baseline, lexical-invariance filtering, and downstream causal removal/steering—does not reduce to its inputs. The random-feature control (ΔVSR −0.5), from-scratch SAE control (+0.22), ΔVQA/ΔCtrl controls, cross-stage ablation, and steering decomposition provide independent grounding; self-citations such as [83] are not load-bearing because the paper reports its own FVU and decoder-cosine measurements. The partial circularity is confined to how the headline effect sizes are aggregated: Table 1's spatial means are averages over rows selected by ranking on ΔVSR itself, and each feature is evaluated on a VSR subset built from its own top activations, so the 12% abstract figure is an order statistic of the outcome variable rather than a typical discovered feature's effect; the same applies to the 17% OCR mean over five top features. For safety, the 24% abstract figure is the mean of per-category top features while the disclosed full-sweep mean is −9.67%, so the headline is a best-case selection. Because the paper discloses the selection in table captions and the safety sweep mean, and because the discovery pipeline has independent controls, this is a partial circularity in the strength of the headline claims, not a collapse of the derivation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-chosen thresholds and on the assumption that per-feature evaluation subsets reflect general behavior. No new physical or architectural entities are introduced. The diffing and control pipeline is operational, and its components are validated through ablation controls.

free parameters (6)
  • Visual energy threshold ε = 0.01 (MMDiff-Gemma); otherwise unspecified
    Chosen by hand to discard features with zero mean squared activation; directly determines the adapted set A and is swept in App. D.3.
  • Decoder cosine percentile pcos = 25% (bottom quartile)
    Features with cosine in the bottom 25% are considered rotated; the paper states downstream results are stable across reasonable choices, but this is an operational definition.
  • Odds ratio threshold τ_OR = 3
    Minimum odds ratio from Fisher exact test for a feature to be a task-specific candidate; set by hand.
  • Firing frequency gap threshold τ_Δp = 0.05
    Minimum increase in per-token firing frequency for a feature to be retained; set by hand.
  • Lexical filter activation cutoff η = 0.01
    A feature passes if it exceeds this activation on at least one neutral prompt; set by hand.
  • Steering strengths α and γf = α unspecified; γf in {1, 3, 10}
    The paper does not report which α is used or whether γf is tuned per feature; the reported steering results likely use the best value, which is a hand-chosen parameter.
assumptions (4)
  • domain assumption Text-only SAE training preserves base-LM feature identity sufficiently for index-aligned diffing.
    Sec. 3.1 and App. C justify this through FVU and decoder cosine alignment, but it remains an assumption about representational stability that could fail in other architectures.
  • ad hoc to paper Decoder cosine and visual energy jointly define features 'altered by multimodal training'.
    The adapted-feature filter is operational and lacks an external ground truth; it is a modeling choice specific to this paper.
  • ad hoc to paper Per-feature evaluation subsets built from top-activating samples measure a feature's true causal contribution to the target behavior.
    Sec. 5.1 states each feature is scored on a VSR subset constructed from its top-activating samples; this is load-bearing for the spatial and OCR effect sizes.
  • domain assumption Three-point all-layers projection ablation is a faithful causal intervention on the residual stream.
    Sec. 4 defines the intervention; it is partially validated by random-feature ablation and the from-scratch SAE control.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Model Diffing for Feature Discovery and Control." pith.science (2026). https://pith.science/paper/P25W5XUJ

@misc{pith2026260809928,
  author       = {Pith},
  title        = {Pith review of: Multimodal Model Diffing for Feature Discovery and Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P25W5XUJ}},
  note         = {Machine review of arXiv:2608.09928}
}
read the original abstract

Multimodal Large Language Models (MLLMs) exhibit strong visual understanding, yet the internal features that cause these behaviors remain difficult to identify, audit, or control. While applicable to post-hoc inspection, hidden states that are decomposed into interpretable feature directions using sparse autoencoders (SAEs) neither readily isolate which features are changed by multimodal training, nor are they directly useful for targeted control. We introduce MMDiff, a multimodal model-diffing framework that trains multimodal SAEs and turns them into feature-level interfaces for discovering and controlling multimodal behavior. MMDiff supports three uses: (i) feature isolation, by diffing a base-LM SAE against its multimodal-adapted counterpart to identify features altered by multimodal training; (ii) task-specific feature detection, via per-token contrastive firing analysis that isolates causal features; and (iii) feature-level control, by causally removing or steering the discovered feature directions. We train multimodal SAEs for three MLLM families, LLaVA-MORE, PaliGemma 2, and InternVL3.5, and evaluate on visual-spatial understanding, multimodal safety, and OCR. MMDiff discovers sparse, causally specific features whose removal selectively degrades target behaviors by an average of 12% on spatial tasks and 17% on OCR, and reduces attack success rate by 24% on multimodal safety attacks, with no impact on VQA performance. Steering these features improves spatial and OCR accuracy by +3.6% and +1.8% on average over a standard single-layer steering baseline. These results show that multimodal SAEs can serve not only as interpretability tools, but as mechanisms for auditing, steering, and controlling MLLMs behavior toward safer and more capable generations.

Figures

Figures reproduced from arXiv: 2608.09928 by the authors.

Figure 1
Figure 1. MMDiff turns multimodal SAE features into an interface for auditing and control. Isolating the features altered by multimodal training yields directions that are causally tied to specific behaviors: steering them improves spatial and OCR accuracy and suppresses unsafe generations, with no measurable impact on general visual question answering. multimodal training. Model diffing resolves this ambiguity by comparing S… view at source ↗
Figure 2
Figure 2. The MMDiff pipeline. Starting from a base-LM SAE, MMDiff (1) trains a multimodal SAE on VLM activations, (2) identifies adapted features adapted by multimodal training and prefer vision input, and (3) discovers task-specific features (multimodal safety, spatial reasoning, OCR) via per-token contrastive firing analysis. Discovered features enable causal ablation and targeted steering. defines a feature direction in t… view at source ↗
Figure 3
Figure 3. Qualitative MMDiff interventions. Examples of MMDiff-CAA steering and feature-level removal across spatial reasoning, multimodal safety, and OCR. 5 Applications We apply the protocols from Sec. 4 to three domains, each instantiated by varying only Dtgt [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Attribution patching across related spatial features. Top: a recurring top-scoring head (L13H1) localizes to relevant regions in queries about “on top of” relations. Middle: bottom-ranked heads on the same samples fail to capture spatial structure. Bottom: unrelated qu…
Figure 5
Figure 5. Figure 5: Auto-Interp example (Layer 16, Feature 176, MMDiff-Llama). Top VQA and VSR samples highlight facing direction, activating on objects facing toward, away, or relative to others. 7 Related Work Model Diffing. Model diffing studies how internal representations change acro…
Figure 6
Figure 6. Figure 6: summarizes aggregated FVU trends for MMDiff-Llama, and [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]
Figure 7
Figure 7. Figure 7: Per-layer FVU across regimes (MMDiff-Llama). Text-only SAEs converge to near-zero error rapidly; image and full-sequence regimes plateau higher. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: Decoder cosine similarity vs. layer (MMDiff-Llama). Text-only SAEs remain aligned with the base-LM dictionary across layers; image-only and full-sequence SAEs diverge in shallow layers and only re-align deeper. Random initialization stays decorrelated. C.5 Seed stabili…
Figure 9
Figure 9. Figure 9: Distribution of SAE features by visual energy and cosine similarity (MMDiff-Llama). All features are shown in gray; adapted features are highlighted in pink. Task-specific candidates (here: spatial) are marked with blue squares, and the subset used for downstream analy…
Figure 10
Figure 10. Figure 10: Distribution of SAE features by visual energy and cosine similarity (MMDiff-Gemma). All features in gray; high-visual-energy features (Ev > ϵ = 0.01) in light pink; the adapted set A (high Ev and bottom-25% cosine, τ = 0.881) in dark pink. Adapted features comprise ∼2…
Figure 11
Figure 11. Figure 11: Per-layer adapted-feature stats (MMDiff-Llama). Counts and mean cosine for the adapted set A (Sec. 3.2). lexical-invariance filtering (Sec. 3.3, step 2). The same three-stage funnel is applied uniformly across spatial, safety, and OCR target distributions; only Dtgt a…
Figure 12
Figure 12. Figure 12: Threshold sweep. Adapted-set size as a function of ϵ and pcos. Model Target All features A (adapted) Discovered MMDiff-Llama Spatial ∼1,024K ∼51K (5%) 711 MMDiff-Gemma Spatial ∼416K ∼85K (∼20%) ∼1,400 MMDiff-Gemma Safety ∼416K ∼85K (∼20%) 1,061 MMDiff-Gemma OCR ∼416K …
Figure 13
Figure 13. Figure 13: Firing-frequency distributions for Dbase and Dsp. D.7 Image counterfactuals Ev alone does not separate image-driven from text-driven activation. To disentangle the two we hold the text fixed and replace the image, on every feature reported in the paper and 300 evaluat…
Figure 14
Figure 14. Figure 14: Auto-Interp example (L16/F176, MMDiff-Llama). Top VQA + VSR samples highlight facing direction. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_14.png]
Figure 15
Figure 15. Figure 15: Additional Auto-Interp examples. Two adapted features; top VQA + VSR samples show consistent spatial relations. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_15.png]
Figure 16
Figure 16. Figure 16: Attribution patching on related spatial features. Top: top head L13H1 localizes to relevant regions for “on top of”. Middle: bottom-ranked heads fail to localize. Bottom: unrelated queries do not trigger spurious activation. Scalar objective. Given a target SAE featur…
Figure 17
Figure 17. Figure 17: Layer-wise aggregated attribution curves for Method B (left) and Method A (right). Both [PITH_FULL_IMAGE:figures/full_fig_p034_17.png]
Figure 18
Figure 18. Figure 18: Attention head aggregated attribution maps for Method B (left) and Method A (right). [PITH_FULL_IMAGE:figures/full_fig_p035_18.png]
Figure 19
Figure 19. Figure 19: Attribution patching on individual spatial features. Each subfigure displays aggregated head/layer attribution maps (left) and attention overlays (right) using the strongest heads on the feature’s top-activating samples across both VSR and VQA. 36 [PITH_FULL_IMAGE:fi…
Figure 20
Figure 20. Figure 20: Attention head visualizations across queries. Each row shows one image with attention overlays from a single high-attribution head across multiple spatial and non-spatial custom queries. The same heads consistently focus on semantically relevant regions. 37 [PITH_FUL…
Figure 21
Figure 21. Figure 21: Bottom-ranked heads yield weak localization. For each feature, we show overlays from the lowest-scoring heads under Methods A and B on the feature’s top-activating samples across VSR and VQA. In contrast to [PITH_FULL_IMAGE:figures/full_fig_p038_21.png]
Figure 22
Figure 22. Figure 22: Distribution of OCR features. Top OCR candidates (purple) cluster among adapted units (pink), paralleling the spatial subset (blue) [PITH_FULL_IMAGE:figures/full_fig_p039_22.png]
Figure 23
Figure 23. Figure 23: Layer 25, Feature 25568. Activates on storefront and clothing text; top heads align to characters. 39 [PITH_FULL_IMAGE:figures/full_fig_p039_23.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 28 canonical work pages

  1. [1]

    Pixtral 12b: A new frontier in image and text understanding

    Mistral AI. Pixtral 12b: A new frontier in image and text understanding. https://mistral. ai/news/pixtral-12b/, September 2024. Accessed: 2024-12-21

    2024

  2. [2]

    Golden gate Claude

    Anthropic. Golden gate Claude. https://www.anthropic.com/news/ golden-gate-claude, May 2024

    2024

  3. [3]

    SAE on activation differences

    Santiago Aranguri, Jacob Drori, and Neel Nanda. SAE on activation differences. AI Align- ment Forum, 2025. https://www.alignmentforum.org/posts/XPNJSa3BxMAN4ZXc7/ sae-on-activation-differences

    2025

  4. [4]

    Refusal in language models is mediated by a single direction

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. InAdvances in Neural Information Processing Systems (NeurIPS), 2024. arXiv:2406.11717

    arXiv 2024

  5. [5]

    Revisiting model stitching to compare neural representations.Advances in neural information processing systems, 34:225–236, 2021

    Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revisiting model stitching to compare neural representations.Advances in neural information processing systems, 34:225–236, 2021

    2021

  6. [6]

    Representation topology divergence: A method for comparing neural network representations.arXiv preprint arXiv:2201.00058, 2021

    Serguei Barannikov, Ilya Trofimov, Nikita Balabin, and Evgeny Burnaev. Representation topology divergence: A method for comparing neural network representations.arXiv preprint arXiv:2201.00058, 2021

    arXiv 2021

  7. [7]

    Understanding information storage and transfer in multi-modal large language models.Advances in Neural Information Processing Systems, 37:7400–7426, 2024

    Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, and Daniela Massiceti. Understanding information storage and transfer in multi-modal large language models.Advances in Neural Information Processing Systems, 37:7400–7426, 2024. 11

    2024

  8. [8]

    Towards monosemanticity: Decomposing language models with dictionary learning.Transformer Circuits Thread, 2023

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and ...

    2023

Show all 99 references
  1. [9]

    Stage-wise model diffing

    Trenton Bricken, Siddharth Mishra-Sharma, Jonathan Marcus, Adam Jermyn, Christopher Olah, Kelley Rivoire, and Thomas Henighan. Stage-wise model diffing. 2024. https: //transformer-circuits.pub/2024/model-diffing/index.html

    2024

  2. [10]

    Observing and controlling features in vision-language-action models.arXiv preprint arXiv:2603.05487, 2026

    Hugo Buurmeijer, Carmen Amo Alonso, Aiden Swann, and Marco Pavone. Observing and controlling features in vision-language-action models.arXiv preprint arXiv:2603.05487, 2026

    2026

  3. [11]

    Improving steering vectors by targeting sparse autoencoder features.arXiv preprint arXiv:2411.02193, 2024

    Sviatoslav Chalnev, Matthew Siu, and Arthur Conmy. Improving steering vectors by targeting sparse autoencoder features.arXiv preprint arXiv:2411.02193, 2024

    2024 arXiv

  4. [12]

    Pappas, Florian Tramer, Hamed Hassani, and Eric Wong

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, and Eric Wong. JailbreakBench: An open robustness benchmark for jailbreaking large la...

    2024

  5. [13]

    Interpreting and controlling vision foundation models via text explanations.arXiv preprint arXiv:2310.10591, 2023

    Haozhe Chen, Junfeng Yang, Carl V ondrick, and Chengzhi Mao. Interpreting and controlling vision foundation models via text explanations.arXiv preprint arXiv:2310.10591, 2023

    2023 arXiv

  6. [14]

    Llava-more: A comparative study of llms and visual backbones for enhanced visual instruction tuning.arXiv preprint arXiv:2503.15621, 2025

    Federico Cocchi, Nicholas Moratelli, Davide Caffagni, Sara Sarto, Lorenzo Baraldi, Marcella Cornia, and Rita Cucchiara. Llava-more: A comparative study of llms and visual backbones for enhanced visual instruction tuning.arXiv preprint arXiv:2503.15621, 2025

    2025 arXiv

  7. [15]

    Explaining how visual, textual and multimodal encoders share concepts.arXiv preprint arXiv:2507.18512, 2025

    Clement Cornet, Romaric Besancon, and Herve Le Borgne. Explaining how visual, textual and multimodal encoders share concepts.arXiv preprint arXiv:2507.18512, 2025

    2025 arXiv

  8. [16]

    Sparse autoen- coders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600, 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600, 2023

    2023 arXiv

  9. [17]

    Case study: Interpreting, manipulating, and controlling CLIP with sparse autoencoders

    Gytis Daujotas. Case study: Interpreting, manipulating, and controlling CLIP with sparse autoencoders. LessWrong, August 2024. https://www.lesswrong.com/posts/ iYFuZo9BMvr6GgMs5/

    2024

  10. [18]

    Toy models of superposition, 2022

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposi...

    2022 arXiv

  11. [19]

    Why does unsupervised pre-training help deep learning? 11:625–660, March

    Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why does unsupervised pre-training help deep learning? 11:625–660, March

  12. [20]

    Interpreting clip’s image representa- tion via text-based decomposition.arXiv preprint arXiv:2310.05916, 2023

    Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip’s image representa- tion via text-based decomposition.arXiv preprint arXiv:2310.05916, 2023

    2023 arXiv

  13. [21]

    Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093, 2024

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093, 2024

    2024 arXiv

  14. [22]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024. 12

    2024 arXiv

  15. [23]

    FigStep: Jailbreaking large vision-language models via typographic visual prompts

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. FigStep: Jailbreaking large vision-language models via typographic visual prompts. InProceedings of the AAAI Conference on Artificial Intelligence, 2025

    2025

  16. [24]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913, 2017

    2017

  17. [25]

    Not all features are created equal: A mechanistic study of vision-language-action models

    Bryce Grant, Xijia Zhao, and Peng Wang. Not all features are created equal: A mechanistic study of vision-language-action models. InICLR Workshop on Multimodal Intelligence, 2026. arXiv:2603.19233

    2026

  18. [26]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  19. [27]

    Mechanistic interpretability for steering vision-language-action models

    Bear Häon, Kaylene Stocking, Ian Chuang, and Claire Tomlin. Mechanistic interpretability for steering vision-language-action models. InConference on Robot Learning (CoRL), 2025. arXiv:2509.00328

    2025 arXiv

  20. [28]

    Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders.arXiv preprint arXiv:2410.20526, 2024

    Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, et al. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders.arXiv preprint arXiv:2410.20526, 2024

    arXiv

  21. [29]

    In-context learning creates task vectors

    Roee Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9318–9333, 2023

    2023

  22. [30]

    VLSBench: Unveiling visual leakage in multimodal safety.arXiv preprint arXiv:2411.19939, 2024

    Xuhao Hu, Dongrui Liu, Hao Li, Xuanjing Huang, and Jing Shao. VLSBench: Unveiling visual leakage in multimodal safety.arXiv preprint arXiv:2411.19939, 2024

    2024 arXiv

  23. [31]

    Sleeper agents: Training deceptive llms that persist through safety training.arXiv preprint arXiv:2401.05566, 2024

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive llms that persist through safety training.arXiv preprint arXiv:2401.05566, 2024

    2024 arXiv

  24. [32]

    Interpreting and editing vision-language representations to mitigate hallucinations.arXiv preprint arXiv:2410.02762, 2024

    Nick Jiang, Anish Kachinthaya, Suzie Petryk, and Yossi Gandelsman. Interpreting and editing vision-language representations to mitigate hallucinations.arXiv preprint arXiv:2410.02762, 2024

    2024 arXiv

  25. [33]

    A “diff” tool for AI: Finding behavioral differences in new models

    Thomas Jiralerspong and Trenton Bricken. A “diff” tool for AI: Finding behavioral differences in new models. Anthropic Research, March 2026. https://www.anthropic.com/research/ diff-tool

    2026

  26. [34]

    Bridging the VLM and mech interp communities for multimodal interpretability

    Sonia Joseph. Bridging the VLM and mech interp communities for multimodal interpretability. LessWrong, October 2024.https://www.lesswrong.com/posts/aa5fzGr8JA3pqvhYC/

    2024

  27. [35]

    Steering CLIP’s vision transformer with sparse autoencoders.arXiv preprint arXiv:2504.08729, 2025

    Sonia Joseph, Praneet Suresh, Ethan Goldfarb, Lorenz Hufe, Yossi Gandelsman, Robert Graham, Danilo Bzdok, Wojciech Samek, and Blake Aaron Richards. Steering CLIP’s vision transformer with sparse autoencoders.arXiv preprint arXiv:2504.08729, 2025

    2025 arXiv

  28. [36]

    Prisma: An open source toolkit for mechanistic interpretability in vision and video

    Sonia Joseph, Praneet Suresh, Yash Vadi, Rob Graham, Lorenz Hufe, Edward Stevinson, Ethan Goldfarb, Sebastian Lapuschkin, Lee Sharkey, and Danilo Bzdok. Prisma: An open source toolkit for mechanistic interpretability in vision and video. InCVPR Workshop on Mechanistic Interpre...

    2025 arXiv

  29. [37]

    Analyzing fine-tuning representation shift for multimodal llms steering alignment.arXiv preprint arXiv:2501.03012, 2025

    Pegah Khayatan, Mustafa Shukor, Jayneel Parekh, and Matthieu Cord. Analyzing fine-tuning representation shift for multimodal llms steering alignment.arXiv preprint arXiv:2501.03012, 2025

    2025 arXiv

  30. [38]

    Saes (usually) transfer between base and chat models

    Connor Kissane, Robert Krzyzanowski, Arthur Conmy, and Neel Nanda. Saes (usually) transfer between base and chat models. AI Alignment Forum post, July 18 2024. 13

    2024

  31. [39]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. InInternational conference on machine learning, pages 3519–3529. PMlR, 2019

    2019

  32. [40]

    Sakla, and Kowshik Thopalli

    Akshay Kulkarni, Tsui-Wei Weng, Vivek Narayanaswamy, Shusen Liu, Wesam A. Sakla, and Kowshik Thopalli. Interpretable and steerable concept bottleneck sparse autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),

  33. [41]

    Understanding image representations by measuring their equivariance and equivalence

    Karel Lenc and Andrea Vedaldi. Understanding image representations by measuring their equivariance and equivalence. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 991–999, 2015

    2015

  34. [42]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024

    2024 arXiv

  35. [43]

    Inference- time intervention: Eliciting truthful answers from a language model

    Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  36. [44]

    Images are Achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. Images are Achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  37. [45]

    Convergent learning: Do different neural networks learn the same representations?arXiv preprint arXiv:1511.07543, 2015

    Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John Hopcroft. Convergent learning: Do different neural networks learn the same representations?arXiv preprint arXiv:1511.07543, 2015

    2015 arXiv

  38. [46]

    Gemma Scope: Open sparse autoencoders everywhere all at once on Gemma 2.arXiv preprint arXiv:2408.05147, 2024

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma Scope: Open sparse autoencoders everywhere all at once on Gemma 2.arXiv preprint arXiv:2408.05147, 2024

    2024 arXiv

  39. [47]

    Sparse autoencoders reveal selective remapping of visual concepts during adaptation

    Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. Sparse autoencoders reveal selective remapping of visual concepts during adaptation. InInternational Conference on Learning Representations (ICLR), 2025. arXiv:2412.05276

    2025 arXiv

  40. [48]

    A survey on mecha- nistic interpretability for multi-modal foundation models.arXiv preprint arXiv:2502.17516, 2025

    Zihao Lin, Samyadeep Basu, Mohammad Beigi, Varun Manjunatha, et al. A survey on mecha- nistic interpretability for multi-modal foundation models.arXiv preprint arXiv:2502.17516, 2025

    2025 arXiv

  41. [49]

    Sparse crosscoders for cross-layer features and model diffing, October 25 2024

    Jack Lindsey, Adly Templeton, Jonathan Marcus, Thomas Conerly, Joshua Batson, and Christo- pher Olah. Sparse crosscoders for cross-layer features and model diffing, October 25 2024. Published on Transformer Circuits Thread; https://transformer-circuits.pub/2024/ crosscoders/index.html

    2024

  42. [50]

    Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 2023

    Fangyu Liu, Guy Edward Toh Emerson, and Nigel Collier. Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 2023

    2023

  43. [51]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  44. [52]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024

    2024

  45. [53]

    MM-SafetyBench: A benchmark for safety evaluation of multimodal large language models

    Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. MM-SafetyBench: A benchmark for safety evaluation of multimodal large language models. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  46. [54]

    OCRBench: On the hidden mystery of OCR in large multimodal models.Science China Information Sciences, 2024

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. OCRBench: On the hidden mystery of OCR in large multimodal models.Science China Information Sciences, 2024. arXiv:2305.07895. 14

    2024 arXiv

  47. [55]

    Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  48. [56]

    Locating and editing factual associations in GPT

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  49. [57]

    Robustly identifying concepts introduced during chat fine-tuning using crosscoders.arXiv preprint arXiv:2504.02922, 2025

    Julian Minder, Clément Dumas, Caden Juang, Bilal Chugtai, and Neel Nanda. Robustly identifying concepts introduced during chat fine-tuning using crosscoders.arXiv preprint arXiv:2504.02922, 2025

    2025

  50. [58]

    What we learned trying to diff base and chat models (and why it matters).LessWrong, 2025

    Julian Minder, Clément Dumas, and Neel Nanda. What we learned trying to diff base and chat models (and why it matters).LessWrong, 2025

    2025

  51. [59]

    Insights on crosscoder model diffing

    Siddharth Mishra-Sharma, Trenton Bricken, Jack Lindsey, Adam Jermyn, Jonathan Marcus, Kelley Rivoire, Christopher Olah, and Thomas Henighan. Insights on crosscoder model diffing. Transformer Circuits Thread, February 2025. https://transformer-circuits. pub/2025/crosscoder-diff...

    2025

  52. [60]

    Attribution patching: Activation patching at industrial scale

    Neel Nanda. Attribution patching: Activation patching at industrial scale. https://www. neelnanda.io/mechanistic-interpretability, 2023. Accessed: 2025-08-23

    2023

  53. [61]

    To- wards interpreting visual information processing in vision-language models.arXiv preprint arXiv:2410.07149, 2024

    Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. To- wards interpreting visual information processing in vision-language models.arXiv preprint arXiv:2410.07149, 2024

    2024 arXiv

  54. [62]

    Steering language model refusal with sparse autoencoders.arXiv preprint arXiv:2411.11296, 2024

    Kyle O’Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangdeh. Steering language model refusal with sparse autoencoders.arXiv preprint arXiv:2411.11296, 2024

    2024 arXiv

  55. [63]

    Zoom in: An introduction to circuits.Distill, 2020

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits.Distill, 2020. doi: 10.23915/distill.00024.001. https://distill.pub/2020/circuits/zoom-in

    2020 doi

  56. [64]

    Visualizing representations: Deep learning and human beings

    Christopher Olah. Visualizing representations: Deep learning and human beings. https:// colah.github.io/posts/2015-01-Visualizing-Representations/ , 2015. Accessed: 2025-08-23

    2015

  57. [65]

    Probing the representational power of sparse autoencoders in vision models

    Matthew Lyle Olson, Neale Ratzlaff, Musashi Hinck, Shao-Yen Tseng, Changbai Li, Phillip Howard, and Vasudev Lal. Probing the representational power of sparse autoencoders in vision models. InICCV Workshops, 2025. arXiv:2508.11277

    2025

  58. [66]

    Gpt-4o-mini: Advancing cost-efficient intelligence

    OpenAI. Gpt-4o-mini: Advancing cost-efficient intelligence. https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/ , 2024. Accessed: 2024-12- 21

    2024

  59. [67]

    Sparse autoencoders learn monosemantic features in vision-language models

    Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, and Zeynep Akata. Sparse autoencoders learn monosemantic features in vision-language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2025. arXiv:2504.02821

    2025

  60. [68]

    Towards vision-language mechanistic interpretability: A causal tracing tool for blip

    Vedant Palit, Rohan Pandey, Aryaman Arora, and Paul Pu Liang. Towards vision-language mechanistic interpretability: A causal tracing tool for blip. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2856–2861, 2023

    2023

  61. [69]

    I’m sorry, I can’t

    Nirmalendu Prakash, Wei Jie Yeo, Amir Abdullah, Ranjan Satapathy, Erik Cambria, and Roy Ka-Wei Lee. Beyond “I’m sorry, I can’t”: Dissecting large language model refusal.arXiv preprint arXiv:2509.09708, 2025

    2025 arXiv

  62. [70]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21527–21536, 2024

    2024

  63. [71]

    Qwen-Scope: An open sparse autoencoder suite for the Qwen model family

    Qwen Team. Qwen-Scope: An open sparse autoencoder suite for the Qwen model family. Technical report, Alibaba Qwen, April 2026. https://qianwen-res.oss-accelerate. aliyuncs.com/qwen-scope/Qwen_Scope.pdf. 15

    2026

  64. [72]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  65. [73]

    Jumping ahead: Improving reconstruction fidelity with JumpReLU sparse autoencoders.arXiv preprint arXiv:2407.14435, 2024

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with JumpReLU sparse autoencoders.arXiv preprint arXiv:2407.14435, 2024

    2024 arXiv

  66. [74]

    Steering Llama 2 via contrastive activation addition

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–1...

    2024 arXiv

  67. [75]

    Multi- modal neurons in pretrained text-only transformers

    Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. Multi- modal neurons in pretrained text-only transformers. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 2862–2867, 2023

    2023

  68. [76]

    SteerVLM: Robust model control through lightweight activation steering for vision language models

    Anushka Sivakumar, Andrew Zhang, Zaber Ibn Abdul Hakim, and Chris Thomas. SteerVLM: Robust model control through lightweight activation steering for vision language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, 2025

    2025

  69. [77]

    Lvlm-interpret: an interpretability tool for large vision-language models.arXiv preprint arXiv:2404.03118, 2024

    Gabriela Ben Melech Stan, Estelle Aflalo, Raanan Yehezkel Rohekar, Anahita Bhiwandiwalla, Shao-Yen Tseng, Matthew Lyle Olson, Yaniv Gurwicz, Chenfei Wu, Nan Duan, and Vasudev Lal. Lvlm-interpret: an interpretability tool for large vision-language models.arXiv preprint arXiv:24...

    2024 arXiv

  70. [78]

    PaliGemma 2: A family of versatile VLMs for transfer.arXiv preprint arXiv:2412.03555, 2024

    Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sahar Kazemzadeh, Thomas Mesnard, Ibrahim Alab- dulmohsin, ...

    2024 arXiv

  71. [79]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua ...

    2024

  72. [80]

    Li, Arnab Sen Sharma, Aaron Mueller, Byron C

    Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. Function vectors in large language models. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  73. [81]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024

    2024

  74. [82]

    Vazquez, Ulisse Mini, and Monte MacDiarmid

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering.arXiv preprint arXiv:2308.10248, 2023

    2023 arXiv

  75. [83]

    Too late to recall: The two-hop problem in multimodal knowledge retrieval

    Constantin Venhoff, Ashkan Khakzar, Sonia Joseph, Philip Torr, and Neel Nanda. Too late to recall: The two-hop problem in multimodal knowledge retrieval. InMechanistic Interpretability for Vision (Non-proceedings Track), CVPR 2025, 2025. URL https://openreview.net/ forum?id=VUhRdZp8ke

    2025

  76. [84]

    How visual representations map to language feature space in multimodal llms.arXiv preprint arXiv:2506.11976, 2025

    Constantin Venhoff, Ashkan Khakzar, Sonia Joseph, Philip Torr, and Neel Nanda. How visual representations map to language feature space in multimodal llms.arXiv preprint arXiv:2506.11976, 2025. 16

    2025 arXiv

  77. [85]

    Steering away from harm: An adaptive approach to defending vision language model against jailbreaks

    Han Wang, Gang Wang, and Huan Zhang. Steering away from harm: An adaptive approach to defending vision language model against jailbreaks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  78. [86]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Z...

    2025 arXiv

  79. [87]

    AdaShield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting

    Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. AdaShield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  80. [88]

    Llava-o1: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440, 2024

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440, 2024

    2024 arXiv

  81. [89]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  82. [90]

    SafeSteer: Adaptive subspace steering for efficient jailbreak defense in vision-language models.arXiv preprint arXiv:2509.21400, 2025

    Xiyu Zeng, Siyuan Liang, Liming Lu, Haotian Zhu, Enguang Liu, Jisheng Dang, Yongbin Zhou, and Shuchao Pang. SafeSteer: Adaptive subspace steering for efficient jailbreak defense in vision-language models.arXiv preprint arXiv:2509.21400, 2025

    2025

  83. [91]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11975–11986, 2023

    2023

  84. [92]

    Towards best practices of activation patching in language models: Metrics and methods

    Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods. InInternational Conference on Learning Representations (ICLR), 2024. URLhttps://doi.org/10.48550/arXiv.2309.16042. arXiv:2309.16042

    2024 doi arXiv

  85. [93]

    Cross-modal information flow in multimodal large language models

    Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. Cross-modal information flow in multimodal large language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  86. [94]

    Multimodal situational safety

    Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Anderson Compalas, Dawn Song, and Xin Eric Wang. Multimodal situational safety. InInternational Conference on Learning Representations (ICLR), 2025. 17 Appendix: Table of Contents A Preliminaries18 A.1 Multimodal Large Language Models ...

    2025

  87. [95]

    Relocated

    for PaliGemma 2, and Qwen-Scope Top-K SAEs [71] for the Qwen3 backbone of InternVL3.5- 2B). Where a property is shared across the models we report it as a property of MMDiff; where it is specific to one MLLM we say so explicitly. A.2 Sparse Autoencoders Motivation.Internal rep...

  88. [96]

    Select up tok=5top-activating samples (deduped across VQA / VQA-spatial / VSR)

  89. [97]

    Call the API once to generate a single concise description

  90. [98]

    Validate using held-out positive samples and random VQA negatives (two short rounds)

  91. [99]

    this neuron activates for

    Compute F1 as a lightweight proxy for description confidence. Outputs are stored per feature as JSON ( description, examples, classification results). Adding overlays improves interpretability, with early results showing a typical gain of about+0.2F1. Prompt A: Description (RA...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.