A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle

Deyu Meng; Guancheng Zhou; Wentao Shu; Xipeng Qiu; Xuyang Ge; Yisi Luo; Zhengfu He; Zhenyu Jin

arxiv: 2605.17504 · v1 · pith:TURPOF7Dnew · submitted 2026-05-17 · 💻 cs.CV · cs.AI

A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle

Guancheng Zhou , Yisi Luo , Zhengfu He , Zhenyu Jin , Xuyang Ge , Wentao Shu , Deyu Meng , Xipeng Qiu This is my paper

Pith reviewed 2026-05-20 13:49 UTC · model grok-4.3

classification 💻 cs.CV cs.AI

keywords mechanistic interpretabilityvisual MIKL divergencediffusion posterior samplingfeature visualizationDINOv3

0 comments

The pith

A distributional view frames visual mechanistic interpretability as finding minimal shifts from the natural image distribution under a KL constraint.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper models the effect of activating an internal feature as a change to the overall distribution of natural images rather than as an isolated optimization target. This distributional framing produces a KL-minimal optimization problem whose solutions expose two common failures in earlier methods: visualizations that look artificial to humans or that fail to actually drive the model's feature. To correct both failures at once, the authors introduce a soft-constraint principle that keeps generated images close to the natural distribution while still reliably activating the chosen feature. They implement the principle by guiding a diffusion model with an energy term derived from the target feature and test the resulting images on the DINOv3 vision model.

Core claim

We model the influence of a feature activation on the natural image distribution, thereby formulating a Kullback-Leibler minimal optimization problem to model the MI task. Under this framework, statistical biases are identified within previous MI paradigms, which reveal that they may either be perceptually uninterpretable to humans or mechanistically unfaithful to the vision models. To resolve the biases under the distributional view, we propose a model with a KL-minimal soft-constraint principle for visual MI that theoretically balances interpretability and faithfulness, realized via energy-guided diffusion posterior sampling.

What carries the argument

The KL-minimal soft-constraint principle, which optimizes images to minimize divergence from the natural image distribution while ensuring the target feature activates inside the vision model.

If this is right

Earlier heuristic methods for visual MI are shown to suffer systematic biases that make them either perceptually unnatural or mechanistically unfaithful.
The soft-constraint principle supplies a single optimization objective that trades off the two requirements in a controlled way.
Energy-guided diffusion posterior sampling supplies a concrete algorithm that satisfies the principle in practice.
Experiments on DINOv3 confirm that the resulting visualizations improve both human ratings and feature-activation fidelity over prior approaches.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same distributional framing could be tried on other vision architectures by swapping in a different generative model as the reference distribution.
If the soft constraint generalizes, it may reduce reliance on hand-tuned regularization terms in optimization-based visualization pipelines.
One could measure whether the method also improves human ability to debug model mistakes by inspecting the generated images.

Load-bearing premise

The natural image distribution captured by the diffusion model is the right reference for judging both human interpretability and mechanistic faithfulness of the visualizations.

What would settle it

If images produced by the energy-guided sampling activate the target feature less often than top-K retrieval or regularized optimization while also receiving lower naturalness ratings from human observers, the claimed balance would not hold.

Figures

Figures reproduced from arXiv: 2605.17504 by Deyu Meng, Guancheng Zhou, Wentao Shu, Xipeng Qiu, Xuyang Ge, Yisi Luo, Zhengfu He, Zhenyu Jin.

**Figure 1.** Figure 1: Induced distributions under matched moments. Top: baseline p(x) and KLSC; bottom: score field s(x) and hard truncation. For m ∈ {5, 6, 7}, both are calibrated to satisfy Eq[s] = m; white curves are density contours. See Appendix H [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

**Figure 2.** Figure 2: Object drift under CLIP task injection. EnergyDPS is guided by an intrinsic SAE score s(x) and a CLIP alignment score fc(x) (fixed prompt c). While s(x) is matched, larger fc(x) yields qualitatively different samples, showing task dominance over the intrinsic feature. See Appendix L.5. can express. Consistently, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Figure 3: Sampling comparisons under matched faithfulness in a toy GMM. Left: prior p(x), intrinsic score s(x), and regularization r(x). Columns vary the target E[s] ∈ {5, 6, 7} (calibrated per method). Rows show retrieval (Re.), optimization w/ regularization (OR), and EnergyDPS (Section 2.3). Numbers report KL(q∥p) (lower is closer to p). Experimental details in Appendix I. optimization in the prompt space. Let p… view at source ↗

**Figure 4.** Figure 4: Qualitative comparison on DINOv3 features. Columns show Top-4 dataset samples, MACO, DiffExplainer, and EnergyDPS (ours). Rows show feature 81 (top) and feature 9863 (bottom), which visually suggest heart-shaped and wing-like patterns, respectively. These labels are post-hoc summaries for reader guidance, not ground-truth feature labels. Implementation details are provided in Appendix J [PITH_FULL_IMAG… view at source ↗

**Figure 5.** Figure 5: DEXTER fails on SAE feature visualization. Qualitative comparison for a DINOv3 SAE feature (index 9863). We report representative outputs from Top-K dataset search, MACO, DiffExplainer, DEXTER (hard-prompt optimization), and EnergyDPS. While other methods can produce samples with high feature activation, DEXTER frequently collapses to low-activation outputs (final activation s(x) = 1.4 in our run), indicat… view at source ↗

**Figure 6.** Figure 6: Neuron-level interpretation is unstable. Top-4 retrieval and three synthesis methods (MACO, DiffExplainer, EnergyDPS) for the same neuron yield heterogeneous or texture-like results, consistent with superposition and a lack of a single coherent semantic [PITH_FULL_IMAGE:figures/full_fig_p036_6.png] view at source ↗

**Figure 7.** Figure 7: Randomly selected neurons vs. SAE features under EnergyDPS. Each unit is visualized by four images generated using the same EnergyDPS sampler. The top row shows randomly selected individual neurons, and the bottom row shows randomly selected SAE features [PITH_FULL_IMAGE:figures/full_fig_p037_7.png] view at source ↗

**Figure 8.** Figure 8: Image-level deviation under increasing guidance. For feature 81, we compare EnergyDPS samples generated with different activation levels against a reference image and report LPIPS values. samples with progressively increasing activation scores s(x). We then compute LPIPS between each generated sample and the reference image to quantify the perceptual image-level deviation. The results are shown in [PITH_F… view at source ↗

**Figure 9.** Figure 9: Semantic shift across activation levels. Left: the Top-4 highest-activating dataset sample for the same feature (a retrieval baseline). Right: samples generated by our sampler, ordered by increasing intrinsic score sk(x) (annotated below each image). As sk(x) increases, the dominant evidence shifts from coarse, context-like court/field cues to a localized object cue (the tennis ball), illustrating that ext… view at source ↗

**Figure 10.** Figure 10: shows that samples can exhibit markedly different semantics as fc(x) increases, even when s(x) remains nearly unchanged, indicating that the injected prompt objective can dominate the resulting visualization. s(x) = 16.8034 fc(x) = 4.2130 s(x) = 16.6693 fc(x) = 10.1400 s(x) = 16.8788 fc(x) = 19.0615 s(x) = 17.7417 fc(x) = 31.3358 s(x) = 15.0999 fc(x) = 35.9034 s(x) = 16.0728 fc(x) = 40.8028 [PITH_FULL_IM… view at source ↗

**Figure 11.** Figure 11: Object drift under reference injection (pixel ℓ2 guidance). EnergyDPS is guided by the intrinsic SAE score s(x) and a reference objective fy(x) = −∥x − y∥ 2 2 for a fixed reference image y. Even when s(x) is approximately matched, reducing MSE increasingly forces samples to resemble y, showing that pixel-level task injection can override intrinsic feature semantics. L.6. More Comparisons in Features We pr… view at source ↗

**Figure 12.** Figure 12: Additional feature visualizations on 100 random SAE features. Each panel shows a 10 × 10 grid of feature visualizations produced by the corresponding method under the same evaluation protocol as in the main experiment. 42 [PITH_FULL_IMAGE:figures/full_fig_p042_12.png] view at source ↗

**Figure 13.** Figure 13: Representative feature examples. For each feature, we show Top-4 dataset samples (retrieval baseline), MACO, DiffExplainer, and EnergyDPS. DiffExplainer often produces qualitatively different semantics from the other references, while EnergyDPS remains closer to the dataset exemplars and intrinsic-guided optimization under the same protocol. These features visually appears to capture eye-shaped, mesh text… view at source ↗

read the original abstract

Most current paradigms in visual mechanistic interpretability (MI) remain confined to interpreting internal units of the vision model via heuristic methods (e.g., top-$K$ activation retrieval or optimization with regularization). In this work, we establish a theoretical distributional view for visual MI, which models the influence of a feature activation on the natural image distribution, thereby formulating a Kullback-Leibler (KL)-minimal optimization problem to model the MI task. Under this framework, statistical biases are identified within previous MI paradigms, which reveal that they may either be perceptually uninterpretable to humans (i.e., deviate from the natural image distribution), or mechanistically unfaithful to the vision models (i.e., unable to activate model features). To resolve the biases under the distributional view, we propose a model with a KL-minimal soft-constraint principle for visual MI that theoretically balances interpretability and faithfulness. We realize this principle via energy-guided diffusion posterior sampling. Extensive experiments validate the theoretical soundness of the proposed distributional view and demonstrate the practical effectiveness of our paradigm on the DINOv3 vision model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper recasts visual feature visualization as KL minimization to a diffusion-captured natural image distribution, with a soft constraint for activation, but the link from that objective to true mechanistic faithfulness is not clearly derived.

read the letter

The main takeaway is that this work tries to replace heuristic feature visualization with a distributional objective: minimize KL divergence to natural images while keeping the target unit active inside the vision model, implemented through energy-guided diffusion posterior sampling on DINOv3. That framing and the sampling trick look like the actual novelty compared with standard activation maximization or regularization tricks in the MI literature. It does a reasonable job flagging that earlier methods can drift into perceptually odd images or ones that do not strongly drive the internal feature. The experiments are presented as evidence that the approach produces more plausible outputs without losing activation strength. The soft spot is the central claim that the KL-minimal soft constraint theoretically balances interpretability and faithfulness. Faithfulness in this context should mean the visualization actually elicits the model's internal response on the target unit, yet nothing in the setup shows why closeness to the diffusion model's natural-image distribution necessarily preserves or maximizes that internal activation rather than just producing nicer-looking pictures. The guidance scale appears to act as an empirical knob for the tradeoff, not a proof that the constrained optimum aligns with the model's true feature direction. There is also the open question of whether the pre-trained diffusion model makes the KL term somewhat circular, since its parameters already embed natural-image statistics. This is aimed at interpretability researchers who want a more principled optimization route for vision transformers instead of ad-hoc regularizers. A reader working on activation-based analysis would find the perspective and the concrete sampling method useful to try. It is worth sending to peer review because the framing is distinct and the method is executable, even though the theoretical connection between the distributional objective and internal faithfulness will need tighter justification during review.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes a distributional framework for visual mechanistic interpretability that models feature influence through KL divergence to the natural image distribution captured by a pre-trained diffusion model. It identifies statistical biases in prior heuristic approaches (top-K retrieval and regularized optimization) that produce either perceptually uninterpretable or mechanistically unfaithful visualizations. To correct these, the authors introduce a KL-minimal soft-constraint principle claimed to theoretically balance interpretability and faithfulness, realized via energy-guided diffusion posterior sampling, and validate the approach through experiments on the DINOv3 vision model.

Significance. If the central construction holds, the work could supply a more principled alternative to heuristic feature visualization by grounding interpretations in distributional properties of natural images. The energy-guided sampling provides a concrete mechanism for trading off the two objectives. However, the significance is limited by the absence of a derivation showing that the KL objective preserves internal feature activation rather than merely producing plausible images.

major comments (3)

[Abstract, §4] Abstract and §4: the claim that the KL-minimal soft-constraint principle 'theoretically balances interpretability and faithfulness' is stated without an explicit loss function, derivation, or proof that the constrained optimum coincides with the model's true feature direction; the guidance scale appears to function as an empirical knob rather than a derived quantity.
[§5] §5 (energy-guided diffusion posterior sampling): it is unclear whether the KL-minimal objective remains independent of the diffusion model's training or reduces to a quantity already implicit in the pre-trained parameters, raising the risk that the method is circular with respect to the reference distribution.
[§3, §6] The weakest assumption (natural image distribution as reference for mechanistic faithfulness) is load-bearing: faithfulness is defined by reliable elicitation of the target unit's internal response, yet no argument or experiment demonstrates that minimizing KL to p_natural necessarily maximizes or preserves this activation rather than trading it off empirically.

minor comments (2)

[§5] Notation for the energy function and guidance scale should be introduced with an explicit equation in the main text rather than deferred to the appendix.
[§6] Quantitative comparisons in the experiments would benefit from reporting both the achieved KL values and the corresponding feature activation strengths to allow direct verification of the claimed balance.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and insightful comments on our manuscript. We address each of the major comments point by point below, providing clarifications and indicating where revisions will be made to strengthen the presentation of our distributional view for visual mechanistic interpretability.

read point-by-point responses

Referee: [Abstract, §4] Abstract and §4: the claim that the KL-minimal soft-constraint principle 'theoretically balances interpretability and faithfulness' is stated without an explicit loss function, derivation, or proof that the constrained optimum coincides with the model's true feature direction; the guidance scale appears to function as an empirical knob rather than a derived quantity.

Authors: We agree that the manuscript would benefit from a more explicit derivation. The KL-minimal soft-constraint principle is defined as the optimization problem: minimize KL(q_θ || p_natural) subject to E_{x~q_θ}[f(x)] >= c, where f is the feature activation and c is the target level. This is converted to a soft constraint using a Lagrangian multiplier λ, leading to the energy function E(x) = KL term + λ * (c - f(x)). The guidance scale in the diffusion sampling corresponds to this λ, which can be tuned but is theoretically grounded in the constraint strength. In the revision, we will include this explicit loss function and a sketch of the derivation in §4 to show how the optimum balances the objectives without deviating from the feature direction. revision: yes
Referee: [§5] §5 (energy-guided diffusion posterior sampling): it is unclear whether the KL-minimal objective remains independent of the diffusion model's training or reduces to a quantity already implicit in the pre-trained parameters, raising the risk that the method is circular with respect to the reference distribution.

Authors: The KL-minimal objective is independent in the sense that it uses the diffusion model solely as an approximator of p_natural to define the reference distribution for interpretability. The optimization itself is driven by the energy term derived from the target vision model's (DINOv3) feature activation, which is external to the diffusion model's parameters. The sampling process modifies the trajectory using this external energy, rather than relying on implicit quantities from the diffusion training. We will revise §5 to explicitly state this separation and provide a step-by-step explanation of how the posterior sampling incorporates the KL term without circularity. revision: yes
Referee: [§3, §6] The weakest assumption (natural image distribution as reference for mechanistic faithfulness) is load-bearing: faithfulness is defined by reliable elicitation of the target unit's internal response, yet no argument or experiment demonstrates that minimizing KL to p_natural necessarily maximizes or preserves this activation rather than trading it off empirically.

Authors: This is a fair point regarding the foundational assumption. Our argument is that by using the soft-constraint, we explicitly enforce the activation level while minimizing deviation from p_natural, thus avoiding the trade-off seen in hard optimization or retrieval methods. Experiments in §6 demonstrate higher activation scores and better perceptual quality compared to baselines. However, we do not claim a universal proof that KL minimization always maximizes activation for arbitrary models; it holds under the distributional view where faithfulness is tied to natural statistics. We will add a brief discussion in §3 and an additional plot in §6 showing the correlation between KL reduction and activation preservation to support this empirically. revision: partial

Circularity Check

0 steps flagged

Derivation self-contained; no reduction of claims to fitted inputs or self-citations

full rationale

The paper introduces a distributional framing that defines the MI task as KL minimization against an externally pre-trained diffusion model capturing the natural image distribution. The KL-minimal soft-constraint principle is explicitly proposed as a new modeling choice to trade off two separately defined objectives (perceptual closeness to natural images versus internal feature activation in the target vision model). Realization via energy-guided diffusion posterior sampling relies on standard conditional sampling techniques rather than any derivation internal to the present work. No equation or step reduces the claimed balance or faithfulness property to a quantity already implicit in the diffusion training objective; the diffusion model remains an independent reference distribution. The central claims therefore retain independent content beyond the inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the existence of a well-defined natural image distribution that can be approximated by a diffusion model and on the assumption that KL divergence to that distribution is a suitable proxy for both human interpretability and model faithfulness.

axioms (1)

domain assumption There exists a natural image distribution that serves as the reference for both perceptual interpretability and mechanistic faithfulness.
Invoked when the paper states that previous methods deviate from the natural image distribution or fail to activate model features.

pith-pipeline@v0.9.0 · 5748 in / 1319 out tokens · 30228 ms · 2026-05-20T13:49:56.482958+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

KLSC Principle defines the induced distribution as the KL-minimal distribution in Q_m: q_KLSC_β ∈ arg min_{q∈Q_m} KL(q∥p). Lemma D.4 ... q_KLSC_β(x) = p(x) exp(β s(x)) / Z(β)
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We establish a theoretical distributional view for visual MI, which models the influence of a feature activation on the natural image distribution

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages · 5 internal anchors

[1]

Quality-aware image-text alignment for opinion-unaware image quality assessment.arXiv preprint arXiv:2403.11176,

Agnolucci, L., Galteri, L., and Bertini, M. Quality-aware image-text alignment for opinion-unaware image quality assessment.arXiv preprint arXiv:2403.11176,

work page arXiv
[2]

Advanc- ing llm safe alignment with safety representation ranking

Du, T., Wei, Z., Chen, Q., Zhang, C., and Wang, Y . Advanc- ing llm safe alignment with safety representation ranking. arXiv preprint arXiv:2505.15710,

work page arXiv
[3]

Toy Models of Superposition

URL https://arxiv.org/ abs/2209.10652. Fel, T., Boissin, T., Boutin, V ., Picard, A., Novello, P., Colin, J., Linsley, D., Rousseau, T., Cad`ene, R., Goetschalckx, L., et al. Unlocking feature visualization for deep network with magnitude constrained optimization.37th Advances in Neural Information Processing Systems (NeurIPS), 36: 37813–37826, 2023a. Fel...

work page internal anchor Pith review Pith/arXiv arXiv
[4]

Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry

URLhttps://arxiv.org/abs/2510.08638. Gorgun, A., Schiele, B., and Fischer, J. Vital: More un- derstandable feature visualization through distribution alignment and relevant information flow,

work page internal anchor Pith review Pith/arXiv arXiv
[5]

He, Z., Wang, J., Lin, R., Ge, X., Shu, W., Tang, Q., Zhang, J., and Qiu, X

URL https://arxiv.org/abs/2503.22399. He, Z., Wang, J., Lin, R., Ge, X., Shu, W., Tang, Q., Zhang, J., and Qiu, X. Towards understanding the nature of attention with low-rank sparse decomposition.arXiv preprint arXiv:2504.20938,

work page arXiv
[6]

URL https://distill.pub/2017/ feature-visualization

doi: 10.23915/ distill.00007. URL https://distill.pub/2017/ feature-visualization. Park, K., Choe, Y . J., and Veitch, V . The linear representa- tion hypothesis and the geometry of large language mod- els. InInternational Conference on Machine Learning (ICML),

work page 2017
[7]

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B

doi: 10.1109/CVPR.2016.91. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models,

work page doi:10.1109/cvpr.2016.91 2016
[8]

High-Resolution Image Synthesis with Latent Diffusion Models

URL https://arxiv.org/ abs/2112.10752. Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Bengio, Y . and LeCun, Y . (eds.),International Conference on Learning Representa- tions (ICLR),

work page internal anchor Pith review Pith/arXiv arXiv
[9]

DINOv3

URLhttps://arxiv.org/abs/2508.10104. Song, Y ., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. InInternational Conference on Learning Representations (ICLR),

work page internal anchor Pith review Pith/arXiv arXiv
[10]

Understanding Neural Networks Through Deep Visualization

Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., and Lipson, H. Understanding neural networks through deep visual- ization.arXiv preprint arXiv:1506.06579,

work page internal anchor Pith review Pith/arXiv arXiv
[11]

Related Work A.1

11 A Distributional View for Visual Mechanistic Interpretability A. Related Work A.1. Mechanistic Interpretability in Vision Models To understand the internal mechanisms of the model, we need to study its internal computational units. Neurons are the most direct internal units of a model, and abundant researches have focused on them to reveal the internal...

work page 2024
[12]

To move beyond attribution, concept-based methods (Poeta et al., 2023; Kim et al.,

trace the models in under specific stimuli, but they do not reveal “what” internal features are being computed (Fel et al., 2025). To move beyond attribution, concept-based methods (Poeta et al., 2023; Kim et al.,

work page 2025
[13]

shift the focus to interpreting the latent representations within the model (e.g., neurons). However, this neuron-centric methods faces fundamental theoretical limitations: individual neurons frequently exhibit polysemanticity, lack a privileged basis representation, and implicitly premise that the semantic feature space is dimensionally bounded by the nu...

work page 2024
[14]

Under the LRH, we can learn this polysemantic concepts through the dictionary learning methods

crystallized intoLinear Representation Hypothesis(LRH), which holds that models contain many more features than neurons, arranged as sparse, quasi-orthogonal directions (Fel et al., 2025). Under the LRH, we can learn this polysemantic concepts through the dictionary learning methods. SAE has been proven to be an effective research method. The polysemantic...

work page 2025
[15]

The vast image distribution may contain “fooling” inputs that excite the internal units without resembling natural images (Pennisi et al., 2025)

demonstrate that unconstrained optimization may lead to uninterpretable patterns. The vast image distribution may contain “fooling” inputs that excite the internal units without resembling natural images (Pennisi et al., 2025). To address this issue, previous activation maximization paradigms design specific regularization to constraint the image distribu...

work page 2025
[16]

These methods rely on prompt-guided diffusion models to obtain the natural image prior, but this process from image to prompt inevitably results in information loss

introduce adiffusion-based proxy paradigms, which optimize a soft or hard prompt to generate the preferred stimuli for specific internal units. These methods rely on prompt-guided diffusion models to obtain the natural image prior, but this process from image to prompt inevitably results in information loss. This information loss is not severe in visualiz...

work page 2008
[17]

The Detailed Formulation of SAE We first restate the sparse autoencoder (SAE) formulation, which serves as the base model throughout this paper

C. The Detailed Formulation of SAE We first restate the sparse autoencoder (SAE) formulation, which serves as the base model throughout this paper. Let x∈RB×S×ddenote a batch of intermediate representations (e.g., patch-/token-wise activations), whereB is the batch size, S the number of spatial positions/tokens, andd the channel dimension. A standard SAE ...

work page 2003
[18]

and applies guidance via∇xts(ˆx0). This guidance can be viewed as a practical 30 A Distributional View for Visual Mechanistic Interpretability surrogate for the ideal correction in (193): it injects the intrinsic signal directly into the reverse dynamics while retaining the diffusion prior. A key advantage is that this construction isscore-agnostic: given...

work page 2023
[19]

J. Implementation Details for DINOv3 Experiments This appendix specifies the exact experimental configurations used in Section 3, including (i) transcoder training on DINOv3 activations, (ii) intrinsic-score construction and stabilization, and (iii) method-specific settings for activation maximization (MACO (Fel et al., 2023a)), proxy-guided generation (D...

work page 2025
[20]

We sample 1,000,000images from ImageNet-1K, resized to256×256

For an input imagex∈X, the hooked feature map has shaped×H×W= 768×16×16. We sample 1,000,000images from ImageNet-1K, resized to256×256. Training objective and hyperparameters.We train a transcoder (Appendix C) with top- 64 sparsification and reconstruc- tion loss only (no auxiliary losses). We use learning rate 1×10−4. Training is continued until the expl...

work page 2025

[1] [1]

Quality-aware image-text alignment for opinion-unaware image quality assessment.arXiv preprint arXiv:2403.11176,

Agnolucci, L., Galteri, L., and Bertini, M. Quality-aware image-text alignment for opinion-unaware image quality assessment.arXiv preprint arXiv:2403.11176,

work page arXiv

[2] [2]

Advanc- ing llm safe alignment with safety representation ranking

Du, T., Wei, Z., Chen, Q., Zhang, C., and Wang, Y . Advanc- ing llm safe alignment with safety representation ranking. arXiv preprint arXiv:2505.15710,

work page arXiv

[3] [3]

Toy Models of Superposition

URL https://arxiv.org/ abs/2209.10652. Fel, T., Boissin, T., Boutin, V ., Picard, A., Novello, P., Colin, J., Linsley, D., Rousseau, T., Cad`ene, R., Goetschalckx, L., et al. Unlocking feature visualization for deep network with magnitude constrained optimization.37th Advances in Neural Information Processing Systems (NeurIPS), 36: 37813–37826, 2023a. Fel...

work page internal anchor Pith review Pith/arXiv arXiv

[4] [4]

Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry

URLhttps://arxiv.org/abs/2510.08638. Gorgun, A., Schiele, B., and Fischer, J. Vital: More un- derstandable feature visualization through distribution alignment and relevant information flow,

work page internal anchor Pith review Pith/arXiv arXiv

[5] [5]

He, Z., Wang, J., Lin, R., Ge, X., Shu, W., Tang, Q., Zhang, J., and Qiu, X

URL https://arxiv.org/abs/2503.22399. He, Z., Wang, J., Lin, R., Ge, X., Shu, W., Tang, Q., Zhang, J., and Qiu, X. Towards understanding the nature of attention with low-rank sparse decomposition.arXiv preprint arXiv:2504.20938,

work page arXiv

[6] [6]

URL https://distill.pub/2017/ feature-visualization

doi: 10.23915/ distill.00007. URL https://distill.pub/2017/ feature-visualization. Park, K., Choe, Y . J., and Veitch, V . The linear representa- tion hypothesis and the geometry of large language mod- els. InInternational Conference on Machine Learning (ICML),

work page 2017

[7] [7]

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B

doi: 10.1109/CVPR.2016.91. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models,

work page doi:10.1109/cvpr.2016.91 2016

[8] [8]

High-Resolution Image Synthesis with Latent Diffusion Models

URL https://arxiv.org/ abs/2112.10752. Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Bengio, Y . and LeCun, Y . (eds.),International Conference on Learning Representa- tions (ICLR),

work page internal anchor Pith review Pith/arXiv arXiv

[9] [9]

DINOv3

URLhttps://arxiv.org/abs/2508.10104. Song, Y ., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. InInternational Conference on Learning Representations (ICLR),

work page internal anchor Pith review Pith/arXiv arXiv

[10] [10]

Understanding Neural Networks Through Deep Visualization

Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., and Lipson, H. Understanding neural networks through deep visual- ization.arXiv preprint arXiv:1506.06579,

work page internal anchor Pith review Pith/arXiv arXiv

[11] [11]

Related Work A.1

11 A Distributional View for Visual Mechanistic Interpretability A. Related Work A.1. Mechanistic Interpretability in Vision Models To understand the internal mechanisms of the model, we need to study its internal computational units. Neurons are the most direct internal units of a model, and abundant researches have focused on them to reveal the internal...

work page 2024

[12] [12]

To move beyond attribution, concept-based methods (Poeta et al., 2023; Kim et al.,

trace the models in under specific stimuli, but they do not reveal “what” internal features are being computed (Fel et al., 2025). To move beyond attribution, concept-based methods (Poeta et al., 2023; Kim et al.,

work page 2025

[13] [13]

shift the focus to interpreting the latent representations within the model (e.g., neurons). However, this neuron-centric methods faces fundamental theoretical limitations: individual neurons frequently exhibit polysemanticity, lack a privileged basis representation, and implicitly premise that the semantic feature space is dimensionally bounded by the nu...

work page 2024

[14] [14]

Under the LRH, we can learn this polysemantic concepts through the dictionary learning methods

crystallized intoLinear Representation Hypothesis(LRH), which holds that models contain many more features than neurons, arranged as sparse, quasi-orthogonal directions (Fel et al., 2025). Under the LRH, we can learn this polysemantic concepts through the dictionary learning methods. SAE has been proven to be an effective research method. The polysemantic...

work page 2025

[15] [15]

The vast image distribution may contain “fooling” inputs that excite the internal units without resembling natural images (Pennisi et al., 2025)

demonstrate that unconstrained optimization may lead to uninterpretable patterns. The vast image distribution may contain “fooling” inputs that excite the internal units without resembling natural images (Pennisi et al., 2025). To address this issue, previous activation maximization paradigms design specific regularization to constraint the image distribu...

work page 2025

[16] [16]

These methods rely on prompt-guided diffusion models to obtain the natural image prior, but this process from image to prompt inevitably results in information loss

introduce adiffusion-based proxy paradigms, which optimize a soft or hard prompt to generate the preferred stimuli for specific internal units. These methods rely on prompt-guided diffusion models to obtain the natural image prior, but this process from image to prompt inevitably results in information loss. This information loss is not severe in visualiz...

work page 2008

[17] [17]

The Detailed Formulation of SAE We first restate the sparse autoencoder (SAE) formulation, which serves as the base model throughout this paper

C. The Detailed Formulation of SAE We first restate the sparse autoencoder (SAE) formulation, which serves as the base model throughout this paper. Let x∈RB×S×ddenote a batch of intermediate representations (e.g., patch-/token-wise activations), whereB is the batch size, S the number of spatial positions/tokens, andd the channel dimension. A standard SAE ...

work page 2003

[18] [18]

and applies guidance via∇xts(ˆx0). This guidance can be viewed as a practical 30 A Distributional View for Visual Mechanistic Interpretability surrogate for the ideal correction in (193): it injects the intrinsic signal directly into the reverse dynamics while retaining the diffusion prior. A key advantage is that this construction isscore-agnostic: given...

work page 2023

[19] [19]

J. Implementation Details for DINOv3 Experiments This appendix specifies the exact experimental configurations used in Section 3, including (i) transcoder training on DINOv3 activations, (ii) intrinsic-score construction and stabilization, and (iii) method-specific settings for activation maximization (MACO (Fel et al., 2023a)), proxy-guided generation (D...

work page 2025

[20] [20]

We sample 1,000,000images from ImageNet-1K, resized to256×256

For an input imagex∈X, the hooked feature map has shaped×H×W= 768×16×16. We sample 1,000,000images from ImageNet-1K, resized to256×256. Training objective and hyperparameters.We train a transcoder (Appendix C) with top- 64 sparsification and reconstruc- tion loss only (no auxiliary losses). We use learning rate 1×10−4. Training is continued until the expl...

work page 2025