Pith. sign in

REVIEW 4 major objections 5 minor 55 references

Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse dictionary learning on FLUX.1's residual streams yields features that beat the model's own neurons on interpretability scores and steer image generation via activation addition.

desk verdict Solid scaling study of SAEs on FLUX.1 with a real whitening fix and honest limitations, but the headline interpretability claim leans on a metric the authors themselves concede is shallow. read the letter →

arxiv 2505.24360 v3 pith:QUK53CR6 submitted 2025-05-30 cs.LG

Stepan Shabalin , Ayush Panda , Dmitrii Kharlapenko , Abdur Raheem Ali , Yixiong Hao , Arthur Conmy This is my paper
classification cs.LG
keywords sparseautoencodersdictionarylearninginterpretabilitytext-to-imagediffusionFLUX.1activationsteeringPCAwhiteningautomatic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sparse dictionary learning can be scaled to a state-of-the-art text-to-image model: the paper trains sparse autoencoders on the residual streams of FLUX.1, a 12-billion-parameter diffusion transformer, and reports that the resulting features reconstruct activations accurately and score higher on an automated interpretability benchmark than the model's own MLP neurons. The same holds for a cheaper variant, Inference-Time Decomposition of Activations (ITDA), which reaches comparable interpretability scores. The paper also shows that adding an SAE feature's decoder direction to the residual stream during sampling steers the generated image toward the feature's semantic content, though only when the prompt is already related to that content. The significance is that internal states of large generative image models become inspectable and controllable at the feature level, much as they have become for language models.

What carries the argument

The load-bearing machinery is the TopK sparse autoencoder applied to the residual stream of a specific transformer layer, with a PCA-whitening pretransform that rescales the anisotropic variance of the stream before encoding and folds the affine transform back into the decoder. The whitening step is what makes training viable: without it the SAE develops more than 99% dead features and its latents collapse into one low-dimensional subspace. Running alongside is the visual autointerpretation pipeline—an off-the-shelf multimodal language model that explains maximum-activating images with activations painted in blue and then judges, image by image, whether a proposed explanation matches the feature's activation pattern—which supplies the interpretability scores that ground the central comparison. For the steering result, the operative mechanism is activation addition: a decoder direction $W_{\mathrm{dec}}[f]$ is added over a spatial patch of the residual stream at early sampling steps, biasing generation toward the feature's semantic content. The ITDA variant replaces the learned encoder with gradient pursuit and builds its dictionary from poorly reconstructed training points, achieving similar interpretability without encoder training.

What would settle it

Run the paper's autointerp pipeline on a randomly initialized (untrained) TopK SAE of identical architecture and compare its score distribution to the trained SAE's reported mean of 0.819 and MLP baseline of 0.654; if the random features score comparably, the metric is not measuring learned semantic content, and the SAE-over-MLP comparison collapses. A complementary check is to take features whose explanations describe only a color, steer with them, and test whether the image's color actually changes; absence of any effect would show the explanations are causally inert.

Watch

Extended reading notes

Core claim

On its own terms, the paper's finding is that residual-stream activations of FLUX.1—a multimodal diffusion transformer with 12 billion parameters—can be decomposed into interpretable sparse features by dictionary learning, and that these features are both faithful and usable. Two decomposition methods are compared: TopK sparse autoencoders and ITDA, which builds its dictionary directly from the training activations and encodes via gradient pursuit. A necessary enabling step is PCA whitening of the inputs: the residual stream's variance is so concentrated in a handful of basis-aligned principal components that naive training produces more than 99% dead features, whereas whitening brings the dead-feature rate below 30%. On interpretability, the paper introduces a visual autointerpretation pipeline in which a multimodal language model writes a description of a feature's maximum-activating images and then scores how accurately that description predicts the feature's activations on new images; by this measure SAEs and ITDAs both beat MLP neurons (mean scores 0.819 and 0.776 versus 0.654). Finally, the paper demonstrates feature-level control: adding a feature's decoder vector to a spatial region of the residual stream during the first denoising steps changes the output consistently with the feature's meaning.

Load-bearing premise

The comparison of SAEs and ITDAs against MLP neurons rests on the automated interpretability score measuring genuine semantic content; the paper itself notes the pipeline may instead reward simple visual statistics such as color.

Editorial extensions

If this is right

  • If the central claim holds, steering a text-to-image model becomes a dictionary lookup: find the internal feature for a concept and add its decoder direction during sampling, with no prompt engineering or fine-tuning.
  • The PCA-whitening requirement will likely transfer to other large diffusion transformers with similarly anisotropic residual streams; training SAEs on them without whitening will produce mostly dead features.
  • ITDA's comparable interpretability at dramatically lower training cost makes dictionary learning feasible for models too large to train encoders on, widening the scope of mechanistic interpretability of generative models.
  • The visual autointerpretation pipeline supplies a scalable replacement for human feature inspection, extending detection scoring from language activations to image activations.
  • The manual difference the paper reports—SAE features encode concrete objects while ITDA features encode abstract attributes—means downstream applications should choose the decomposition type according to whether they need to manipulate objects or textures and colors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the locality of steering—effects appear only when the prompt already relates to the feature—suggests features are not globally coherent across contexts, so a circuit-level account of these models would have to explain how prompts select feature subsets.
  • Beyond the paper: the admitted risk that autointerp rewards color-like statistics implies a direct test—features whose explanations mention only color should be steered with; if output color does not change, the high scores are not evidence of semantic interpretability.
  • Beyond the paper: the paper's own spatial-locality observation predicts that a convolutional encoder, which respects the structure of adjacent image patches, should improve reconstruction or interpretability at fixed sparsity.
  • Beyond the paper: the basis-aligned high-variance directions in the residual stream may be low-level encoding artifacts (image-embedding or noise-encoding content); isolating and subtracting them before training is a concrete next experiment that could sharpen feature semantics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper applies dictionary learning methods—TopK sparse autoencoders (SAEs) and Inference-Time Decomposition of Activations (ITDA)—to the residual streams of the large text-to-image diffusion model FLUX.1. The authors introduce PCA whitening to stabilize SAE training, evaluate reconstruction quality, compare SAE, ITDA, and MLP neuron features using a Gemini-based automated interpretability pipeline, and demonstrate steering of image generation via activation addition with SAE features. The paper claims that both SAEs and ITDAs outperform MLP neurons on interpretability metrics while maintaining high reconstruction quality, and that SAE features enable targeted steering of image generation.

Significance. If the claims hold, this would be a useful step toward applying dictionary-learning interpretability to state-of-the-art text-to-image diffusion models, with potential applications in model control and transparency. The paper also provides engineering contributions: a JAX implementation of FLUX.1, TPU-friendly SAE training infrastructure, and publicly released code for training and autointerpretation. The reconstruction results are plausible and the steering demonstrations are visually striking. However, the central interpretability comparison rests on an automated metric that the paper itself acknowledges may be superficial, and the steering evidence is anecdotal rather than systematic. These issues are load-bearing for the main claims, so the manuscript requires substantial revision before the conclusions can be accepted.

major comments (4)
  1. [§5.2, §6, §7] The automated interpretability metric is not validated, and the paper's own text provides direct evidence that it may not measure semantic interpretability. Section 6 states 'it is possible that our pipeline may pick up on simple characteristics of the image like color,' and Section 7 admits that autointerp metrics 'don't distinguish between abstract visual properties and manipulable semantic concepts.' Since the headline claim that 'both SAEs and ITDAs outperform MLP neurons on interpretability' rests entirely on this pipeline, the metric must be validated against human judgments on a subset of features, with inter-rater reliability reported, and the pipeline must be shown not to be driven by simple statistics such as color or texture. Without this, the comparison in Figure 7 is not sufficient to support the central interpretive claim.
  2. [§5.3, Figure 1] The steering claim is demonstrated only on a small set of cherry-picked examples, and the paper provides no systematic evaluation. To support 'general-purpose steering of image representations using sparse autoencoder latents,' the authors should report results across multiple features and prompts, include negative controls (e.g., random directions or non-activating features), measure intervention effects quantitatively (e.g., via CLIP similarity or detection metrics), and specify how often steering succeeds. The current evidence is anecdotal and does not support the strength of the claim as stated.
  3. [§3.4, §3.5, Figure 6] The comparison between SAEs and ITDAs is potentially confounded by different preprocessing: Section 3.4 states that all SAE training uses PCA whitening, while Section 3.5 states that ITDA training is normalized 'without PCA.' If the two methods are trained in different input spaces, the reconstruction comparison in Figure 6 and the interpretability comparison in Figure 7 may not be on equal footing. The authors should clarify whether the reconstruction targets are in the same space, whether whitening is applied to ITDA in any experiment, and whether this affects the results. Additionally, the figures report only point estimates with no confidence intervals or multiple seeds, making it unclear whether the observed ITDA reconstruction advantage is significant.
  4. [§5.2, Figure 7] The selection procedure for features included in the autointerpretation comparison is underspecified. The text says 'only consider activating features from each method,' but does not define an activation threshold, does not report the number of features per method, and does not specify which MLP neurons were used (e.g., which layer or whether they are from the same residual stream). Without this information, the histograms in Figure 7 could reflect different feature-selection biases across methods rather than differences in inherent interpretability. The authors should specify the exact inclusion criteria and report feature counts.
minor comments (5)
  1. [§5.1] The formula for variance explained in Section 5.1 is unusual and appears to compute an average of squared correlations rather than the standard 1 - MSE/variance definition. Please clarify the formula or replace it with the standard definition to avoid confusion.
  2. [Figure 6] The legend lists ITDA dictionary sizes up to 64k, but the text in Section 5.2 states that autointerpretation uses d=64000; please ensure the dictionary sizes are consistent and label the SAE in Figure 6 with its full size (d=59k vs. d=64k).
  3. [References] The reference list contains typos and formatting errors, e.g., 'UseNix Security' for 'USENIX Security,' and reference [17] has an anomalous trailing '1'. A careful proofreading pass is needed.
  4. [Appendix D] The prompts for the explainer and scorer are helpful, but it is unclear whether the scorer is given the same annotated images as the explainer or only the textual explanation. This detail matters for reproducibility.
  5. [§3.3] The paper states that SAEs are trained on single-step generations from FLUX.1 Schnell, but later discusses applicability to FLUX.1 Dev in future work; please clarify whether any results (reconstruction, autointerp, steering) use only Schnell, and whether the conclusions are intended to generalize to Dev.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation chain; empirical results are direct measurements. The autointerp metric is self-referential (same Gemini model writes and scores explanations) and the paper concedes it may track superficial color statistics, but this is a construct-validity caveat, not a step where an output equals an input by construction.

full rationale

The paper contains no derivation whose conclusion is an input in disguise. The headline results are direct empirical comparisons: reconstruction VE/FVU of SAEs and ITDAs on FLUX residual streams, autointerp score histograms against MLP neurons, and steering images via activation addition. None is generated by fitting a parameter to the target quantity and then 'predicting' it. The only self-referential element is Section 5.2's visual autointerpretation pipeline, where the same Gemini 2.0 Flash model both writes the explanation and judges whether images match it; the resulting 'AutoInterp Score' therefore measures Gemini's self-agreement rather than an externally validated construct. The paper itself flags this in Future Work ('it is possible that our pipeline may pick up on simple characteristics of the image like color') and the Conclusion concedes that the automated metrics do not distinguish abstract visual properties from manipulable semantic concepts. That is a genuine measurement-validity limitation and a reason to be cautious about the interpretability ranking, but it is not circular in the technical sense used here: the comparison is measured, not derived from the metric's definition. Self-citations to prior work by the senior author (e.g., [7] and [40]) are contextual, appearing for steering-vector context and a weight-folding trick, and they do not carry the load of the central claims. No uniqueness theorem is imported from the authors' own papers, and no ansatz is smuggled in via citation. Thus the score of 2 reflects a minor self-referential measurement choice, not a circular derivation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central results rest on standard dictionary learning assumptions, the choice of hyperparameters k and d, an ITDA threshold found by sweeps, the whitening transformation, and the unvalidated autointerp score. No new physical entities are introduced.

free parameters (3)
  • TopK sparsity k = 64 for main autointerp runs; swept in Figure 6
    Chosen by hand; controls reconstruction versus interpretability tradeoff.
  • Dictionary size d = 64,000 for layer 18; 16k to 64k in Figure 6
    Chosen by hand; larger d increases capacity and training cost.
  • ITDA reconstruction threshold = Not reported; found using sweeps
    Determines which datapoints are added to the dictionary; tuned empirically.
assumptions (5)
  • domain assumption Residual stream activations can be approximated as sparse linear combinations of dictionary features.
    Core SAE and ITDA premise; standard in dictionary learning literature.
  • domain assumption PCA whitening of residual stream preserves the information needed for downstream interpretability and steering.
    The paper assumes normalization does not destroy semantic content; no direct validation is provided.
  • domain assumption The autointerpretation pipeline produces valid interpretability scores.
    Used to compare methods; limited by explainer and scorer biases as the paper notes in Future Work.
  • domain assumption Max-activating examples are representative of a feature.
    Both explanation generation and human inspection rely on max-activating examples.
  • domain assumption FLUX.1 Schnell single-step activations are a sufficient proxy for FLUX.1 behavior.
    All training uses Schnell at 256x256 in one step; the paper does not study other timesteps or the Dev variant.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning." pith.science (2026). https://pith.science/paper/QUK53CR6

@misc{pith2026250524360,
  author       = {Pith},
  title        = {Pith review of: Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QUK53CR6}},
  note         = {Machine review of arXiv:2505.24360}
}
read the original abstract

Sparse autoencoders are a promising new approach for decomposing language model activations for interpretation and control. They have been applied successfully to vision transformer image encoders and to small-scale diffusion models. Inference-Time Decomposition of Activations (ITDA) is a recently proposed variant of dictionary learning that takes the dictionary to be a set of data points from the activation distribution and reconstructs them with gradient pursuit. We apply Sparse Autoencoders (SAEs) and ITDA to a large text-to-image diffusion model, Flux 1, and consider the interpretability of embeddings of both by introducing a visual automated interpretation pipeline. We find that SAEs accurately reconstruct residual stream embeddings and beat MLP neurons on interpretability. We are able to use SAE features to steer image generation through activation addition. We find that ITDA has comparable interpretability to SAEs.

Figures

Figures reproduced from arXiv: 2505.24360 by the authors.

Figure 1
Figure 1. Maximum activating examples and steering effects of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Residual SAEs for FLUX.1 a hyperparameter that controls the complexity of the prob￾lem, with higher k allowing better reconstruction, but also potentially decreasing interpretability. The vector e is then decoded into y ∈ R d , y ≈ x, through an affine decoder: y = Wdece + bdec. This SAE is trained with MSE recon￾struction loss against the original activations MSE P (x, y) = n i=1 1 n (yi − xi) 2 with no additional … view at source ↗
Figure 3
Figure 3. Residual stream norms for double blocks [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Residual stream norms for single blocks 4.2. Latent space spectrum When we trained SAEs without any normalization (Sec￾tion 3.3), we found adequate reconstruction quality with many dead features. The alive features seemed inter￾pretable, but there were less than 1000 o…
Figure 5
Figure 5. Figure 5: Variances explained by principal components [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Reconstruction quality of FLUX.1 double block 18 [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Histograms of autointerp accuracy scores for three meth [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Reconstruction quality for ITDAs and SAEs across lay [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Maximum activating examples of some interpretable [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Maximum activating examples of some interpretable [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 39 canonical work pages

  1. [1]

    Language mod- els can explain neurons in language models

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan 6 Leike, Jeff Wu, and William Saunders. Language mod- els can explain neurons in language models. URL https://openaipublic. blob. core. windows. net/neuron- explainer/paper/index. html.(Date accessed: 14.05. 2023), 2,

    work page 2023

  2. [2]

    Thomas Blumensath and Mike E. Davies. Gradient pursuits. IEEE Transactions on Signal Processing, 56(6):2370–2382,

    work page

  3. [3]

    Towards monosemanticity: Decomposing language mod- els with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yi- fan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Ka- rina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, an...

  4. [4]

    Thread: Circuits

    Nick Cammarata, Shan Carter, Gabriel Goh, Chris Olah, Michael Petrov, Ludwig Schubert, Chelsea V oss, Ben Egan, and Swee Kiat Lim. Thread: Circuits. Distill, 2020. https://distill.pub/2020/circuits. 2

    work page 2020

  5. [5]

    Extracting Training Data from Diffusion Models

    Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagiel- ski, Vikash Sehwag, Florian Tram `er, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting Training Data from Diffusion Models. UseNix Security, 2023. arXiv:2301.13188 [cs]. 1

    arXiv 2023

  6. [6]

    Activation atlas

    Shan Carter, Zan Armstrong, Ludwig Schubert, Ian John- son, and Chris Olah. Activation atlas. Distill, 2019. https://distill.pub/2019/activation-atlas. 6

    work page 2019

  7. [7]

    Im- proving Steering Vectors by Targeting Sparse Autoencoder Features, 2024

    Sviatoslav Chalnev, Matthew Siu, and Arthur Conmy. Im- proving Steering Vectors by Targeting Sparse Autoencoder Features, 2024. arXiv:2411.02193 [cs]. 2

    arXiv 2024

  8. [8]

    Be- yond surface statistics: Scene representations in a latent dif- fusion model, 2023

    Yida Chen, Fernanda Vi ´egas, and Martin Wattenberg. Be- yond surface statistics: Scene representations in a latent dif- fusion model, 2023. 1, 6

    work page 2023

Show all 55 references
  1. [9]

    Sparse autoencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2401.01345, 2024. 1

    2024 arXiv

  2. [10]

    Saeuron: Interpretable concept unlearning in diffusion models with sparse autoen- coders, 2025

    Bartosz Cywi ´nski and Kamil Deja. Saeuron: Interpretable concept unlearning in diffusion models with sparse autoen- coders, 2025. 2

    2025

  3. [11]

    Interpreting and steering features in images

    Gytis Daujotas. Interpreting and steering features in images. LessWrong, 2024. 2

    2024

  4. [12]

    Qlora: Efficient finetuning of quantized llms,

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms,

  5. [13]

    Toy models of superposition

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield- Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superpos...

    2022

  6. [14]

    Scaling rectified flow trans- formers for high-resolution image synthesis, 2024

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yan- nik Marek, and Robin Rombach. Scaling rectified flow tr...

    2024

  7. [15]

    Towards multimodal interpretability: Learning sparse interpretable features in vision transformers

    Hugo Fry. Towards multimodal interpretability: Learning sparse interpretable features in vision transformers. Less- Wrong, 2024. 2

    2024

  8. [16]

    Concept sliders: Lora adaptors for precise control in diffusion models, 2023

    Rohit Gandikota, Joanna Materzynska, Tingrui Zhou, Anto- nio Torralba, and David Bau. Concept sliders: Lora adaptors for precise control in diffusion models, 2023. 2

    2023

  9. [17]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupr ´e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093, 2024. 2, 3, 6, 1

    2024 arXiv

  10. [18]

    Sparse autoencoders can interpret randomly ini- tialized transformers, 2025

    Thomas Heap, Tim Lawson, Lucy Farnik, and Laurence Aitchison. Sparse autoencoders can interpret randomly ini- tialized transformers, 2025. 6

    2025

  11. [19]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. CoRR, abs/2006.11239, 2020. 2

    2006 arXiv

  12. [20]

    Arbitrary style transfer in real-time with adaptive instance normalization, 2017

    Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization, 2017. 4

    2017

  13. [21]

    Sparse autoencoders find highly interpretable features in language models

    Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representa- tions, 2024. 1, 2, 5

    2024

  14. [22]

    Ayodeji Ijishakin, Ming Liang Ang, Levente Baljer, Daniel Chee Hian Tan, Hugo Laurence Fry, Ahmed Abdulaal, Aen- gus Lynch, and James H. Cole. H-space sparse autoencoders. In Neurips Safe Generative AI Workshop 2024, 2024. 2

    2024

  15. [23]

    Understanding and steering Llama 3, 2024

    Caden Juang, Gonc ¸alo Paulo, Jacob Drori, and Belrosem Nora. Understanding and steering Llama 3, 2024. 2, 5

    2024

  16. [24]

    Concept steerers: Leveraging k-sparse autoencoders for controllable genera- tions, 2025

    Dahye Kim and Deepti Ghadiyaram. Concept steerers: Leveraging k-sparse autoencoders for controllable genera- tions, 2025. 2

    2025

  17. [25]

    Revelio: Interpreting and leveraging semantic information in diffusion models, 2024

    Dahye Kim, Xavier Thomas, and Deepti Ghadiyaram. Revelio: Interpreting and leveraging semantic information in diffusion models, 2024. 2

    2024

  18. [26]

    Bert busters: Outlier dimensions that dis- rupt transformers, 2021

    Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers, and Anna Rumshisky. Bert busters: Outlier dimensions that dis- rupt transformers, 2021. 4, 5

    2021

  19. [27]

    Instrumenting llm model internals in jax,

    J ´anos Kram ´ar. Instrumenting llm model internals in jax,

  20. [28]

    Diffusion models already have a semantic latent space, 2023

    Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. Diffusion models already have a semantic latent space, 2023. 2, 6

    2023

  21. [29]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 2

    2024

  22. [30]

    The remark- able robustness of llms: Stages of inference?, 2024

    Vedang Lad, Wes Gurnee, and Max Tegmark. The remark- able robustness of llms: Stages of inference?, 2024. 4

    2024

  23. [31]

    Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models

    Patrick Leask, Neel Nanda, and Noura Al Moubayed. Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models. ICML 2025, 2025. arXiv:2505.17769 [cs] version: 1. 1, 2 7

    2025 arXiv

  24. [32]

    Sparse crosscoders for cross-layer features and model diff- ing

    Jack Lindsey*, Adly Templeton*, Jonathan Marcus*, Thomas Conerly*, Joshua Batson, and Christopher Olah. Sparse crosscoders for cross-layer features and model diff- ing. Transformer Circuits, 2024. 6

    2024

  25. [33]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling, 2023. 2

    2023

  26. [34]

    Flow straight and fast: Learning to generate and transfer data with rectified flow, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow, 2022. 2

    2022

  27. [35]

    Sparse autoencoder

    Andrew Ng. Sparse autoencoder. CS294A Lecture Notes ,

  28. [36]

    Shortform comment on sae locality

    nostalgebraist. Shortform comment on sae locality. Less- Wrong, 2024. 6

    2024

  29. [37]

    Steering Language Model Refusal with Sparse Autoencoders

    Kyle O’Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangde. Steering Language Model Refusal with Sparse Autoencoders. 2024. Publisher: arXiv Version Number: 1. 2

    2024

  30. [38]

    Unsupervised discovery of semantic latent directions in diffusion models, 2023

    Yong-Hyun Park, Mingi Kwon, Junghyo Jo, and Youngjung Uh. Unsupervised discovery of semantic latent directions in diffusion models, 2023. 2

    2023

  31. [39]

    Scalable diffusion models with transformers, 2023

    William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. 2

    2023

  32. [40]

    Jumping ahead: Improving reconstruction fi- delity with jumprelu sparse autoencoders, 2024

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Son- nerat, Arthur Conmy, Vikrant Varma, J ´anos Kram ´ar, and Neel Nanda. Jumping ahead: Improving reconstruction fi- delity with jumprelu sparse autoencoders, 2024. 3

    2024

  33. [41]

    U-net: Convolutional networks for biomedical image segmentation,

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation,

  34. [42]

    Adversarial diffusion distillation, 2023

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation, 2023. 2

    2023

  35. [43]

    A multimodal automated interpretability agent, 2025

    Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, and An- tonio Torralba. A multimodal automated interpretability agent, 2025. 2

    2025

  36. [44]

    Shawn Shan, Wenxin Ding, Josephine Passananti, Stan- ley Wu, Haitao Zheng, and Ben Y . Zhao. Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Gen- erative Models. 2024. arXiv:2310.13828 [cs]. 1

    2024 arXiv

  37. [45]

    Taking fea- tures out of superposition with sparse autoencoders

    Lee Sharkey, Dan Braun, and Beren Millidge. Taking fea- tures out of superposition with sparse autoencoders. AI Alignment Forum, 2022. 1

    2022

  38. [46]

    Replacing sae encoders with inference-time optimisation, 2024

    Lewis Smith. Replacing sae encoders with inference-time optimisation, 2024. 2, 3

    2024

  39. [47]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In 9th International Con- ference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. 2

    2021

  40. [48]

    Unpacking sdxl turbo: Interpreting text-to-image models with sparse au- toencoders, 2024

    Viacheslav Surkov, Chris Wendler, Mikhail Terekhov, Justin Deschenaux, Robert West, and Caglar Gulcehre. Unpacking sdxl turbo: Interpreting text-to-image models with sparse au- toencoders, 2024. 2

    2024

  41. [49]

    Vazquez, Ulisse Mini, and Monte MacDi- armid

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDi- armid. Steering Language Models With Activation Engineer- ing, 2024. arXiv:2308.10248 [cs]. 2

    2024 arXiv

  42. [50]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. 5

    2023

  43. [51]

    A general protocol to probe large vision models for 3d physical understanding, 2024

    Guanqi Zhan, Chuanxia Zheng, Weidi Xie, and Andrew Zis- serman. A general protocol to probe large vision models for 3d physical understanding, 2024. 1

    2024

  44. [52]

    Why are adaptive methods good for attention models?,

    Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J Reddi, Sanjiv Kumar, and Suvrit Sra. Why are adaptive methods good for attention models?,

  45. [53]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 6 8 Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning Supplementary Material 0 10 20 30 Layer 0.0 0.2 0.4 0.6 0.8 1.0Variance explained Var...

    2023

  46. [2011]

    Unpublished lecture notes. 1

  47. [2023]

    https://transformer-circuits.pub/2023/monosemantic- features/index.html. 1, 2

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.