Pith. sign in

REVIEW 3 major objections 4 minor 58 references

Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a single steering vector, computed from the difference between unwanted and safe text representations in the middle block of a multimodal diffusion transformer, can erase a target concept from generated images…

desk verdict Plausible and cheap tuning-free erasure for MM-DiTs, but the timestep-transfer assumption is unverified and the evaluation is tuned on the same metrics. read the letter →

arxiv 2608.12829 v1 pith:AT2BVXXO submitted 2026-08-13 cs.CV

Qiao Li , Xiaomeng Fu , Yuanshu Zhao , Qipeng Wang , Jiao Dai , Jizhong Han This is my paper · ORCID
classification cs.CV
keywords concepterasuremultimodaldiffusiontransformerssteeringvectorsinference-timeinterventioncontrollablegenerationrectifiedflowtext-to-imagemodelsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal diffusion transformers, the architecture behind many recent text-to-image models, can be made to stop generating unwanted content—celebrities, art styles, nudity—without any fine-tuning. The paper claims that the target concept's semantics are concentrated in the middle transformer blocks, and that a single vector computed from the difference between an unwanted prompt and a safe prompt in the text branch can push generation away from the concept when added to early and middle blocks at every denoising step. Because the vector lives on the sparse text tokens and rectified flow sampling follows a straight trajectory, one vector constructed at one timestep can steer all steps with negligible overhead. If correct, this gives deployed models an inexpensive safety switch that also permits controlled redirection, such as turning one art style into another.

What carries the argument

The load-bearing object is the middle-block text-branch steering vector, defined in Eq. (4): $v_b = \frac{1}{n}\sum_i (x^+_{b,i} - x^-_{b,i})$, the average difference of summed text-token outputs for safe versus unwanted paired prompts, then normalized and scaled. It works because the text branch is sparse (154 or 333 tokens versus 4096 image tokens), because middle-block text activations are where concept semantics are most salient, and because rectified flow gives a straight, deterministic sampling path so a direction measured at one intermediate timestep remains coherent when added at all timesteps. Injection into consecutive early and middle blocks (in the paper, blocks 3 through 10) lets the bias accumulate without the distortion that early-only injection causes or the ineffectiveness of middle- or late-only injection.

What would settle it

Take paired prompts with the target concept placed in different sentence positions and syntactic roles, construct the steering vector from one subset, then measure erasure on held-out prompts; if erasure collapses when the context changes, the vector is encoding context-specific artifacts rather than concept semantics.

Watch

Extended reading notes

Core claim

The central claim is that target-concept semantics in MM-DiTs are localized: early blocks settle global layout, late blocks add fine detail, and middle blocks carry the primary conceptual content, as supported by block-wise noise injection and attention-map visualization. Taking this as a premise, the paper constructs a steering vector $v^{\mathrm{steer}}_b$ at a middle block $b$ by feeding paired prompts that differ only in the target phrase, summing the text-branch output tokens that differ, averaging the difference over $n$ pairs, normalizing, and scaling by a strength $l$. Injecting this single vector into the output text tokens of several consecutive early and middle blocks at every denoising timestep shifts the unwanted concept toward the chosen safe one. The paper reports state-of-the-art erasure of celebrities, art styles, and nudity on Stable Diffusion-v3.5-medium and FLUX.1 while keeping aesthetic quality and unrelated-image fidelity, with controllable multi-style steering and strong robustness to adversarial prompts.

Load-bearing premise

The method assumes that summing the text-branch token differences between an unwanted and a safe prompt isolates the target concept's semantic direction, and that one scaled version of that direction remains correct for every prompt context and every denoising step.

Editorial extensions

If this is right

  • A single forward pass per concept builds the steering vector; erasure then costs only vector additions at inference, making it practical for deployed models.
  • Erasure is controllable: the user chooses the safe concept, so the same target can be redirected to different outputs, such as turning Van Gogh style into pixel art, watercolor, or ink wash.
  • Because steering acts on internal representations rather than prompt keywords, it resists adversarial prompts that rename or paraphrase the target concept.
  • The same vector applied across all denoising steps keeps steering temporally coherent, so image layout and quality are largely preserved.
  • The construction is conceptually transferable across MM-DiT models, as demonstrated on two different architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same construction could be repurposed beyond safety: choosing contrastive pairs for any attribute, such as hairstyle, lighting, era, or object, would yield a single-vector editor, provided that attribute is semantically localized in middle blocks as the paper shows for the tested concepts.
  • Because the steering vector is built in the text branch and injected additively, multiple concept vectors might be composable or portable across MM-DiTs that share the text encoder; the paper does not test either possibility.
  • A practical limit the paper does not address: an additive, publicly known steering direction can in principle be inverted and subtracted, so the method is a content filter rather than an unlearning guarantee against a determined adversary.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a training-free concept erasure method for multimodal diffusion transformers (MM-DiTs). The method constructs a steering vector from the difference between text-branch activations of an unwanted-concept prompt and a safe-concept prompt, extracted at an intermediate block and an intermediate denoising timestep, and injects this single vector into a consecutive set of early-to-middle blocks at every denoising timestep. Experiments on Stable Diffusion v3.5 and FLUX.1 cover three concept categories (celebrity, art style, nudity), compare against tuning-free and fine-tuning baselines, and include ablations on construction block, construction timestep, injection block combinations, steering strength, multi-style steering, and adversarial robustness.

Significance. If the claims hold, the paper offers a practical, low-overhead intervention for safety-oriented concept erasure in modern MM-DiTs, with a useful capability of steering toward a specified safe concept rather than merely suppressing the unwanted one. The authors provide extensive qualitative ablations, a multi-style steering demonstration, and adversarial robustness results, and they explicitly target components of the architecture (text-branch sparsity, middle-block semantics, rectified-flow sampling) that are plausible and well motivated. The work is a reasonable fit for the multimedia community and could be a useful baseline for future activation-steering research. However, the central temporal-transfer mechanism is not directly verified, and the quantitative comparisons rest on single-run numbers with hyperparameters selected on the same metrics.

major comments (3)
  1. [§3.3, Table 2] The claim that a single steering vector constructed at t=0.5 can be injected at all denoising timesteps is justified only by the straightness of rectified flow latent trajectories, but that property concerns latent sample paths, not text-branch activation differences. Table 2 varies the construction timestep while still injecting the resulting vector at all timesteps, so it does not establish that the t=0.5 direction is appropriate at t=0.1 or t=0.9. I request either a per-timestep injection ablation (e.g., injecting only into early, middle, or late timestep windows) or a direct measurement of the consistency of the steering direction across timesteps (e.g., cosine similarity between vectors computed at different t), because the method's stated efficiency and coherence depend on this transfer.
  2. [§4.2, Table 1] Table 1 reports each metric for a single run or a single prompt set, with no error bars or significance tests. Since the steering strength l, construction block b, injection block range, and construction timestep t are selected using ablations that report the same erasure and aesthetic metrics as Table 1, the headline comparisons to baselines are at risk of selection bias. Please report results over multiple seeds, state how the test prompts and hyperparameter-selection data were separated, and confirm that the final Table 1 numbers are obtained on a held-out set.
  3. [Eq. (4), §3.2] The number n of paired prompts used to construct the steering vector is never stated, and no analysis is provided of the variability of the per-pair differences (x+_{b,i} - x-_{b,i}). If the summed token differences are dominated by syntactic or contextual artifacts of the paired prompts rather than by the target concept's semantics, the resulting vector may not transfer across prompt contexts. Please report n, the per-pair direction consistency (e.g., average pairwise cosine similarity), and an experiment varying n to show that the vector saturates as n grows.
minor comments (4)
  1. [§4.3] The sensitivity analysis heading contains a typo: 'ł' should be 'l' in both the heading and the surrounding text, and the variable should be consistently typeset as l or \ell.
  2. [§4.1] The experimental setup does not state how many images are generated per prompt, how many random seeds are used, or what sampling steps (e.g., 50, 28) are employed; these details are needed to interpret the FID and CLIP numbers and to reproduce the results.
  3. [Figure 3] The caption of Figure 3 would be clearer if it specified the scale of the injected Gaussian noise and which block outputs are modified, since those choices affect the qualitative conclusions about block roles.
  4. [References] Some references are incomplete or inconsistent with archival style: [22] points to a general URL without a version, and [9] mixes ICML formatting with a different style; please check all entries for completeness.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the steering vector construction and injection are self-contained, and the reported erasure is verified by external detectors rather than by the construction equations.

full rationale

The derivation is self-contained: the steering vector in Eq. (3)-(4) is defined as the average activation difference between contrastive prompt pairs, and injection is an additive operation on text-branch activations. The paper's erasure claims are then verified by external detectors (GIPHY, LLaVA, Gram, NudeNet), not by the construction equations themselves, so there is no step where the target result is equivalent to an input by definition. The sensitivity-based selection of the scalar strength l, the construction block b, and the injection block range is disclosed in Section 4.3 as ordinary hyperparameter tuning; it may affect generalization but does not make the derivation circular. The rectified-flow-based claim that a vector computed at one timestep transfers to all timesteps is an empirical assumption that would need further validation, but an unsupported assumption is a correctness risk rather than a circular step. No load-bearing self-citation, imported uniqueness theorem, or renamed known result appears in the derivation chain.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or ontological entities. Its load-bearing components are the steering vector and the hand-chosen hyperparameters (l, b, injection range, t). All of these are derived from or tuned on the model and evaluation metrics, which is captured in the free-parameter list. The domain assumptions listed here are the qualitative architectural and sampling priors that the method relies on without formal proof.

free parameters (5)
  • Steering strength l = l=70 for celebrity, l=40 for art style; nudity value not reported
    Chosen empirically to balance erasure effectiveness and aesthetic score in Section 4.3 and Figure 7. It is a scalar multiplier on the normalized difference vector and directly controls the magnitude of intervention.
  • Construction block b = b=10 for SDv3.5 (stable interval b in [9,13])
    Selected via ablation comparing early, middle, and late blocks in Table 2 and Figure 5. The middle-block choice is justified by noise-injection and attention-map analysis, but the exact index is model-specific and hand-picked from the ablation results.
  • Injection block range = b=3..10 for SDv3.5
    Determined through ablation of single- and multi-block injection strategies in Table 3. The range is a hand-chosen hyperparameter that balances structural preservation and semantic steering.
  • Construction timestep t = t=0.5
    Selected via ablation comparing early, intermediate, and late timesteps in Table 2 and Figure 5. The choice is justified by the intuition that intermediate timesteps balance structure and detail, but it is an empirical hyperparameter.
  • Number of paired prompts n = not reported
    Section 3.2 states a dataset of n carefully curated paired prompts but never specifies n or the curation procedure. This affects the variance and generality of the steering vector.
assumptions (4)
  • domain assumption MM-DiT block-wise semantic concentration: middle blocks encode primary concept semantics, early blocks encode global structure, late blocks encode fine details.
    Supported qualitatively by noise-injection experiments (Figure 3a,b) and attention-map visualization (Figure 3c), but not quantified or theoretically derived. The whole method depends on this architectural prior.
  • domain assumption Rectified flow's straight sampling trajectory makes a steering vector computed at an intermediate timestep valid for all denoising timesteps.
    Stated in Section 3.3 without formal derivation. The claim that features evolve coherently and predictably along a straight path is load-bearing for the all-timestep injection strategy.
  • domain assumption Summing the text-branch output vectors of the tokens that differ between paired prompts yields a semantically meaningful concept direction.
    In Eq. (4), the steering direction is the average over pairs of the summed token vectors. The paper provides no evidence that token-summed differences are semantically aligned in the contextual T5-XXL embedding space.
  • domain assumption The external detectors (GIPHY, LLaVA, NudeNet, Gram matrix, LPIPS) provide valid measures of concept presence.
    Evaluation of erasure effectiveness relies entirely on these detectors, whose false-positive and false-negative rates are not reported for the test distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers." pith.science (2026). https://pith.science/paper/AT2BVXXO

@misc{pith2026260812829,
  author       = {Pith},
  title        = {Pith review of: Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AT2BVXXO}},
  note         = {Machine review of arXiv:2608.12829}
}
read the original abstract

Multimodal Diffusion Transformers (MM-DiTs) have demonstrated remarkable text-to-image generation performance, surpassing traditional U-Net-based diffusion models. Nevertheless, their powerful generative capabilities also raise significant safety concerns, as they may generate sensitive or inappropriate content. While existing concept erasure methods aim to mitigate such risks, most require modifying model parameters, which are often architecture-specific and impractical for deployed larger models. Several tuning-free approaches face challenges when applied to advanced large-scale MM-DiTs due to their deeply embedded knowledge, broad semantic space, and context-dependent text encoders. To address these challenges, we propose to erase concepts by directly manipulating the model's internal representations. Our key insight, derived from an in-depth analysis of MM-DiT's block-wise generative roles, is that text-conditioned semantic representations are most salient in the middle blocks of MM-DiTs. Based on this, we extract representations of an unwanted concept and a desirable safe one from the middle block, construct a steering vector from their difference, and inject this single vector into consecutive early and middle blocks. By operating exclusively on the sparse text-branch tokens and leveraging the straight sampling trajectory of rectified flow, our method achieves effective concept erasure with negligible overhead and without any training. Extensive experiments across MM-DiT models demonstrate that our method achieves state-of-the-art performance in erasing diverse concepts, enables effective control over the final output, and remains robust to adversarial attacks.

Figures

Figures reproduced from arXiv: 2608.12829 by the authors.

Figure 1
Figure 1. During the generation process with the multimodal diffusion transformer, our method applies steering vectors to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall pipeline of our proposed method. We [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. (a) and (b) are images generated by SDv3.5, where [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between our method and baselines on Stable Diffusion-v3.5-medium. Our method effectively [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Images generated using steering vectors constructed [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Sensitivity analysis of the steering strength [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Visual examples of nudity erasure against adversar [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 19 canonical work pages

  1. [1]

    Nick Hasty, Ihor Kroosh, Dmitry Voitekh, and Dmytro Korduban. 2019. Giphy celebrity detector. https://github.com/Giphy/celeb-detection-oss

    work page 2019

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  3. [3]

    Fazl Barez, Tingchen Fu, Ameya Prabhu, Stephen Casper, Amartya Sanyal, Adel Bibi, Aidan O’Gara, Robert Kirk, Ben Bucknall, Tim Fist, et al . 2025. Open problems in machine unlearning for ai safety.arXiv preprint arXiv:2501.04952 (2025)

    arXiv 2025

  4. [4]

    P Bedapudi. 2019. Nudenet: Neural nets for nudity classification, detection and selective censoring. (2019)

    work page 2019

  5. [5]

    Anh Bui, Long Vuong, Khanh Doan, Trung Le, Paul Montague, Tamas Abraham, and Dinh Phung. 2024. Erasing undesirable concepts in diffusion models with adversarial preservation.arXiv preprint arXiv:2410.15618(2024)

    arXiv 2024

  6. [6]

    Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin-Yu Chen, and Wei-Chen Chiu. 2023. Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts.arXiv preprint arXiv:2309.06135(2023)

    arXiv 2023

  7. [7]

    Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language models: A simple approach to controlled text generation.arXiv preprint arXiv:1912.02164 (2019)

    arXiv 2019

  8. [8]

    Kamil Deja et al. 2025. Saeuron: Interpretable concept unlearning in diffusion models with sparse autoencoders.arXiv preprint arXiv:2501.18052(2025)

    arXiv 2025

Show all 58 references
  1. [9]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. InForty- first international conference on ma...

    2024

  2. [10]

    Masane Fuchi and Tomohiro Takagi. 2024. Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearning.. InBMVC

    2024

  3. [11]

    Slabaugh, Jiankang Deng, and Ismail Elezi

    Tatiana Gaintseva, Andreea-Maria Oncescu, Chengcheng Ma, Ziquan Liu, Mar- tin Benning, Gregory G. Slabaugh, Jiankang Deng, and Ismail Elezi. 2025. CASteer: Cross-Attention Steering for Controllable Concept Erasure. https: //api.semanticscholar.org/CorpusID:276961220

    2025

  4. [12]

    Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau

  5. [13]

    Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyńska, and David Bau. 2024. Unified concept editing in diffusion models. InProceedings of the IEEE/CVF winter conference on applications of computer vision. 5111–5120

    2024

  6. [14]

    Daiheng Gao, Shilin Lu, Wenbo Zhou, Jiaming Chu, Jie Zhang, Mengxi Jia, Bang Zhang, Zhaoxin Fan, and Weiming Zhang. 2025. Eraseanything: Enabling concept erasure in rectified flow transformers. InForty-second International Conference on Machine Learning

    2025

  7. [15]

    Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2015. A neural algorithm of artistic style.arXiv preprint arXiv:1508.06576(2015)

    2015 arXiv

  8. [16]

    Chao Gong, Kai Chen, Zhipeng Wei, Jingjing Chen, and Yu-Gang Jiang. 2024. Re- liable and efficient concept erasure of text-to-image diffusion models. InEuropean Conference on Computer Vision. Springer, 73–88

    2024

  9. [17]

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems30 (2017)

    2017

  10. [18]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models.Advances in neural information processing systems33 (2020), 6840–6851

    2020

  11. [19]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.Iclr1, 2 (2022), 3

    2022

  12. [20]

    Anubhav Jain, Yuya Kobayashi, Takashi Shibuya, Yuhta Takida, Nasir Memon, Julian Togelius, and Yuki Mitsufuji. 2024. Trasce: Trajectory steering for concept erasure.arXiv preprint arXiv:2412.07658(2024)

    2024 arXiv

  13. [21]

    Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. 2023. Ablating concepts in text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision. 22691– 22702

    2023

  14. [22]

    Black Forest Labs. 2024. FLUX. https://blackforestlabs.ai/announcing-black- forest-labs/. Accessed: [19.11.2025]

    2024

  15. [23]

    LAION-AI. 2022. aesthetic-predictor. https://github.com/LAION-AI/aesthetic- predictor

    2022

  16. [24]

    Chak Tou Leong, Yi Cheng, Jiashuo Wang, Jian Wang, and Wenjie Li. 2023. Self- detoxifying language models via toxification reversal. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 4433–4449

    2023

  17. [25]

    Leyang Li, Shilin Lu, Yan Ren, and Adams Wai-Kin Kong. 2025. Set you straight: Auto-steering denoising trajectories to sidestep unwanted concepts. InProceed- ings of the 33rd ACM International Conference on Multimedia. 9257–9266

    2025

  18. [26]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. InEuropean conference on computer vision. Springer, 740–755

    2014

  19. [27]

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. 2022. Flow matching for generative modeling. InThe eleventh international conference on learning representations

    2022

  20. [28]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 26296–26306

    2024

  21. [29]

    Xingchao Liu, Chengyue Gong, and Qiang Liu. 2023. Flow straight and fast: Learn- ing to generate and transfer data with rectified flow. InInternational conference on learning representations (ICLR)

    2023

  22. [30]

    Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. 2024. Mace: Mass concept erasure in diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6430–6440

    2024

  23. [31]

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. 2021. Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073(2021)

    2021 arXiv

  24. [32]

    Byeonghu Na, Mina Kang, Jiseok Kwak, Minsang Park, Jiwoo Shin, SeJoon Jun, Gayoung Lee, Jin-Hwa Kim, and Il-Chul Moon. 2026. Training-free safe text embedding guidance for text-to-image diffusion models.Advances in Neural Information Processing Systems38 (2026), 85984–86014

    2026

  25. [33]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transform- ers. InProceedings of the IEEE/CVF international conference on computer vision. 4195–4205

    2023

  26. [34]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learnin...

    2021

  27. [35]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research21, 140 (2020), 1–67

    2020

  28. [36]

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexan- der Turner. 2024. Steering llama 2 via contrastive activation addition. InProceed- ings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 15504–15522

    2024

  29. [37]

    Pau Rodriguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, Xavier Suau, et al. 2025. Controlling language and diffusion models by transport- ing activations. InInternational Conference on Learning Representations, Vol. 2025. 89812–89855

    2025

  30. [38]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695

    2022

  31. [39]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. InInternational Conference on Medical image computing and computer-assisted intervention. Springer, 234–241

    2015

  32. [40]

    Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting

  33. [41]

    Yang Song and Stefano Ermon. 2019. Generative modeling by estimating gradients of the data distribution.Advances in neural information processing systems32 (2019)

    2019

  34. [42]

    InProceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 22522–22531

  35. [43]

    Nishant Subramani, Nivedita Suresh, and Matthew E Peters. 2022. Extracting latent steering vectors from pretrained language models. InFindings of the Asso- ciation for Computational Linguistics: ACL 2022. 566–581

    2022

  36. [44]

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2020. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456(2020)

    2020 arXiv

  37. [45]

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. Steering language models with activation engineering.arXiv preprint arXiv:2308.10248(2023)

    2023 arXiv

  38. [46]

    Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. 2023. Ring-a-bell! how reliable are concept removal methods for diffusion models?. InThe Twelfth International Conference on Learning Representations

    2023

  39. [47]

    Kang Wei, Xin Yuan, Fushuo Huo, Chuan Ma, Long Yuan, Songze Li, Ming Ding, and Dacheng Tao. 2025. Responsible Diffusion: A Comprehensive Survey on MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Qiao Li et al. Safety, Ethics, and Trust in Diffusion Models.arXiv preprint a...

    2025

  40. [48]

    Haoran Wang and Kai Shu. 2024. Trojan activation attack: Red-teaming large language models using steering vectors for safety-alignment. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management. 2347–2357

    2024

  41. [49]

    Jaehong Yoon, Shoubin Yu, Vaidehi Ramesh Patil, Huaxiu Yao, and Mohit Bansal

  42. [50]

    Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung-Yi Ho, Nan Xu, and Qiang Xu

  43. [51]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang

  44. [52]

    Hongjue Zhao, Haosen Sun, Jiangtao Kong, Xiaochang Li, Qineng Wang, Liwei Jiang, Qi Zhu, Tarek Abdelzaher, Yejin Choi, Manling Li, et al. 2026. Odesteer: A unified ode-based steering framework for llm alignment.arXiv preprint arXiv:2602.17560(2026)

    2026

  45. [54]

    Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi

  46. [55]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Forget-me-not: Learning to forget in text-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1755–1764

  47. [2018]

    InProceedings of the IEEE conference on computer vision and pattern recognition

    The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition. 586–595

  48. [2023]

    InProceedings of the IEEE/CVF international conference on computer vision

    Erasing concepts from diffusion models. InProceedings of the IEEE/CVF international conference on computer vision. 2426–2436

  49. [2024]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Mma-diffusion: Multimodal attack on diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7737–7746

  50. [2025]

    InInternational Conference on Learning Representations, Vol

    Safree: Training-free and adaptive guard for safe text-to-image and video generation. InInternational Conference on Learning Representations, Vol. 2025. 56439–56465

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.