Pith. sign in

REVIEW 5 major objections 5 minor 80 references

A text-to-image model can be made safer at deployment time by reading and rebalancing its own attention maps, with no retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:37 UTC pith:KXWXVLCO

load-bearing objection Plausible inference-time attention modulation for FLUX safety, but the unseen-adapter generalization claim is untested and the paper's main control parameter is defined inconsistently. the 5 major comments →

arxiv 2607.14945 v1 pith:KXWXVLCO submitted 2026-07-16 cs.CV

Introspective Attention Modulation for Safe Text-to-Image Generation

Basim Azam , Hossein Rahmani , Naveed Akhtar This is my paper
classification cs.CV
keywords text-to-image safetyattention modulationinference-time interventiondiffusion transformersflow matchingLoRA adaptationconcept erasuresafe generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that modern text-to-image diffusion models can police their own unsafe outputs by observing and adjusting their attention maps during generation, with no retraining or weight modification. At each denoising step the method extracts attention activations, scores them with pre-trained per-head safety classifiers, and runs a small constrained optimization that pulls suspect attention toward a safe reference while leaving most structure intact. On a deliberately unsafe LoRA-adapted FLUX.1-dev model, reported detector-visible nudity drops from 56.3% to 39.6% on the I2P benchmark and vision-language-model-flagged unsafe images fall from 8.1% to 1.7%, while CLIP text-image alignment holds steady or improves. A sympathetic reader would care because lightweight adapters easily bypass prompt filters and concept-erasure fixes, so an inference-time mechanism that generalizes across model variants and preserves quality would give deployed systems a practical safety lever.

Core claim

The central claim is that unsafe content in rectified-flow diffusion transformers leaves a detectable trace in the model's own attention states, and that rebalancing those states at inference time can suppress that content without degrading generation. The mechanism: at each sampling step, extract the attention maps from dual-stream and single-stream transformer blocks, detect unsafe patterns with sparse per-head logistic regressors W_unsafe, and solve a small constrained optimization—coherence energy toward the safe reference plus a safety regularizer plus a KL proximity term—using projected gradient descent. The corrected attention maps are then injected back into the transformer to recomp

What carries the argument

The load-bearing object is a collection of sparse logistic regressors, W_unsafe^(ℓ,h) per attention head, learned offline on labeled attention maps. Given a current attention set, the method forms the loss L_int = coherence term (Frobenius distance to safe reference attention) + λ safety term (inner product with W_unsafe) + (1/η) KL proximity to the original attention, minimizes it by projected gradient descent for a few steps, and re-injects the result into the transformer via an attention-conditioned velocity evaluation. The introspection ratio α_intro = 1 − ||φ*−Φ_safe||_F / ||Φ−Φ_safe||_F measures how far the modulation moves toward the safe reference and acts as a runtime dial for the s

Load-bearing premise

The load-bearing premise is that unsafe content is linearly separable in the model's attention maps: if a sparse logistic regressor trained on a modest labeled set cannot recognize an unseen unsafe pattern, the modulation has nothing to suppress regardless of the optimization.

What would settle it

Train the per-head regressors on one set of unsafe categories (say, nudity) and test on disjoint categories (say, violence or self-harm) under the same optimization; if the method's safety gains on held-out categories collapse while in-distribution gains persist, the mechanism would be shown to depend on category-specific linear signatures rather than a general unsafe signal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, a base model with an unsafe LoRA can be made safer purely by wrapping inference, which addresses the adapter-bypass problem without retraining the adapter away.
  • The introspection ratio gives a knob for context-dependent moderation: one deployed model can be run at different safety strengths rather than maintaining multiple fine-tuned checkpoints.
  • Because the recipe is attention-based rather than tied to one architecture, the same safety mechanism works across FLUX variants and on SDXL, suggesting it may persist across next-generation diffusion transformers.
  • Safety gains here do not come at the cost of alignment; the paper's CLIP scores stay flat or improve, which is a concrete advantage over concept-erasure methods that can degrade quality or even backfire.
  • The same learned-concept framework can be pointed at other harmful categories—violence, self-harm, illegal activity—by swapping the concept vector, as the paper explores on an attack benchmark.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own adversarial numbers leave headroom; a natural next step would be training W_unsafe on a much wider and more diverse corpus and measuring whether SneakyPrompts-style obfuscations shrink further. This is our inference, not the paper's claim.
  • Because the method re-routes attention without retraining, it could plausibly be used to steer other semantic properties—style, bias, composition—by learning different per-head regressors; the paper does not claim this.
  • The dependence on linear separability in attention space suggests a stress test: evaluate the modulation on prompts whose unsafe content is encoded in unusual compositional forms; if gains vanish, the bottleneck is the linear safety probe rather than the modulation itself.
  • An adaptive adversary targeting the safety signal could search for prompts whose attention maps look safe to W_unsafe but still yield explicit output; the paper acknowledges this risk, and quantifying that attack surface would clarify the method's durability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes an inference-time safety intervention for rectified-flow transformer text-to-image models (FLUX-based and SDXL). At each denoising step, the method extracts the model's self- and cross-attention maps, scores them with per-head sparse logistic regressors W_unsafe learned offline on labeled attention examples, optimizes a modulated attention set under a coherence loss, a safety regularizer, and a proximity constraint, and re-injects the modulated attention into the transformer to produce a corrected velocity. The authors evaluate on I2P, SneakyPrompts, MMA-Diffusion, and UnlearnDiffAtk, comparing against concept-erasure baselines (UCE, ESD, EraseAnything, FlowEdit, MCE). The central claim is that, without retraining or modifying the base model, this attention-space regulation suppresses unsafe content even when the model has been adapted with a LoRA adapter, while preserving or slightly improving CLIP alignment.

Significance. If the central claim holds, the method is a meaningful contribution: it is a training-free, deployment-time safety mechanism for modern DiT/flow-matching T2I architectures, an area where the paper convincingly shows concept-erasure baselines degrade. The use of attention statistics as a safety signal and the per-head modulation mechanism are novel and plausible. The paper also ships a project page and commits to releasing code, which aids reproducibility. However, the evaluation currently supports the main empirical pattern on a single LoRA adapter and one architecture family; the paper's broader claims of model-agnostic and adapter-agnostic robustness are not yet supported by the evidence presented. The internal inconsistency around the introspection ratio and the undisclosed construction of W_unsafe are load-bearing concerns that require careful revision.

major comments (5)
  1. [§3.3, Eq. (12); §4.3, Table 3] The introspection ratio as defined is an outcome of the optimization: α_intro(t) = 1 − ||φ*_t − Φ_safe|| / ||Φ_t − Φ_safe||, with values near 1 meaning strong correction. Yet the experiments set α_intro to 0.25, 0.40, and 0.50 as configurations, and label Strong as α_intro=0.25, which Eq. (12) would interpret as near-minimal intervention. Either the quantity in Table 3 is a different hyperparameter (e.g., a target λ schedule) or the definition/interpretation in Eq. (12) is wrong. This undermines the claimed 'introspection ratio control' and the safety-quality tradeoff analysis, since the reader cannot tell what is being varied.
  2. [§3.3, Eq. (8); §4, Experimental Setup] W_unsafe is the load-bearing safety signal. The paper does not state the training set composition, size, label source, split from evaluation benchmarks, or—critically—whether the labeled attention maps come from the same Heartsync/Flux-NSFW-uncensored LoRA used in all §4 experiments. If W_unsafe is fitted on that adapter's attention signatures, the method may be memorizing one adapter's patterns rather than detecting unsafe content intrinsically. The limitation paragraph acknowledges coverage restrictions, but the manuscript needs to disclose the training data and provide unseen-adapter evaluations (different LoRAs, ranks, targets, DreamBooth, Textual Inversion) before the core 'intrinsic robustness to adaptation' claim can be assessed.
  3. [§4.3, 'Model-Agnostic Safety'; Table 5] The model-agnostic claim is underspecified for SDXL. FLUX and SDXL have different architectures, layer counts, head counts, and attention map sizes; the paper does not explain how a FLUX-trained W_unsafe is applied to SDXL or whether W_unsafe is retrained. If retraining is required, the 'same introspective attention modulation without any retraining' claim is contradicted. If the SDXL results use a separately trained W_unsafe, the table should say so. Fig. 5 is also only qualitative, so the cross-variant generalization claim lacks quantitative support.
  4. [Table 1; Table 2; Table 3] There are inconsistencies in the safety and quality metrics that affect reproducibility. Table 2's 'Overall' for the baseline is stated in the text as 75, but Table 1 reports baseline VLM rate on I2P as 8.1%; Table 2 values are said to be counts, but the magnitudes are inconsistent with a 4-sample-per-prompt evaluation over the full I2P set. Table 3's column layout appears shifted (five numbers under headers for six columns), and the CLIP value for the baseline differs from Table 1. The authors should give the exact evaluation protocol for the VLM counts (subset size, prompt template, threshold) and align all tables.
  5. [§4, 'Experimental Setup'; §4.3, 'Introspection Ratio Control'] All experiments use the single LoRA adapter Heartsync/Flux-NSFW-uncensored. The paper's motivation is that parameter-efficient adaptations easily bypass guardrails, so the central promise is robustness to malicious adapters. With only one adapter in all quantitative evaluations, the paper does not currently demonstrate that the method generalizes to unseen adapters, which is exactly the scenario that distinguishes this approach from methods that know the adapter a priori. Including at least one or two additional LoRA adapters (and ideally DreamBooth/Textual Inversion) is necessary to support the central claim.
minor comments (5)
  1. [Abstract and elsewhere] Typos and formatting: 'Ourcodecanbeaccessed' in the abstract, 'measurs' near Eq. (12), 'principled' in §4.3, 'strenght' in §4.3, and inconsistent spacing in 'cf.Eq.' throughout.
  2. [§3.3, Eq. (12)] The introspection ratio is defined per timestep t, but the experimental section treats it as a scalar configuration. Please clarify how per-timestep/head values are aggregated and how λ is scheduled from α_intro.
  3. [§4.2, Table 1] The CLIP-score differences between the proposed method and the baseline are very small (e.g., 30.98 vs 31.01 on COCO). Given the strong claim of 'maintaining or even improving alignment,' report confidence intervals or significance tests across samples.
  4. [§4.1 / Fig. 3] The qualitative figure would be more informative if the prompts were listed or referenced in the caption, and if the same random seed / latent initialization was used across methods for a fairer visual comparison.
  5. [References] The LoRA adapter is cited to a Hugging Face repository with no version or hash. Since the experimental setup depends on this artifact, a pinned version or more detailed description is needed for reproducibility.

Circularity Check

0 steps flagged

No significant circularity: the learned safety regressor is evaluated against external benchmarks disjoint from its training labels.

full rationale

The claimed derivation is an inference-time optimization over attention maps. The only learned input is W_unsafe (Eq. 8), fitted by sparse logistic regression on labeled examples. This parameter enters the objective only through the linear regularizer R_safe(φ)=⟨φ,W_unsafe⟩ (Eq. 9). If safety were measured by this inner product, the improvement would be by construction. It is not: the paper's headline safety metrics are NudeNet NR, Qwen-VL VLM rate, and CLIP score, evaluated on I2P, SneakyPrompts, MMA-Diffusion and UnlearnDiffAtk. The paper explicitly states W_unsafe is trained on labeled examples 'disjoint from evaluation benchmarks', so suppressing attention in the direction of W_unsafe does not logically force the reported NudeNet/VLM reductions; that is an empirical generalization claim. The optimization steps (Eqs. 5-14) implement a trust-region projection toward lower W_unsafe response while staying near the original attention and a safe reference; no equation equates the external safety metrics to the optimization objective. The only self-citation (ref. [1]) appears in a list of prior training-based suppression methods and is not load-bearing for the central claim. Undisclosed details about W_unsafe's training data, the possibility that it was trained on the same LoRA used in evaluation, and the dimensional transfer of the FLUX-trained regressor to SDXL are reproducibility/evidence gaps, not circularity. Accordingly, no circular step is identified.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The paper's central mechanism is a constrained optimization over attention maps. What it 'pays for' upfront is a learned concept classifier and several hand-set hyperparameters. The characterization of unsafe content is entirely delegated to W_unsafe; no first-principles safety signal is derived.

free parameters (6)
  • λ (safety weight) = 1.5 default; 0.5, 1.0 in ablations
    Controls strength of safety regularizer in Eq. (5); chosen by hand, central to safety-quality tradeoff.
  • η (proximity/learning rate) = 0.1
    Appears as 1/η KL weight in Eq. (5) and as learning rate in §4; chosen by hand.
  • α_step (PGD step size) = 0.1
    Step size in projected gradient update Eq. (11), chosen by hand.
  • PGD iterations = 5
    Number of optimization iterations per denoising step; chosen by hand.
  • NudeNet threshold = 0.6
    Detection threshold for the NR safety metric; determines reported nudity rates.
  • W_unsafe (learned per-head logistic regressors) = Not reported
    Learned on labeled attention examples; the core classifier that defines unsafe attention. Dataset, size, and split undisclosed.
axioms (4)
  • domain assumption Rectified-flow ODE and Euler integration accurately model FLUX generation
    Used in §3.1 as the setting; if the flow/ODE approximation is wrong, the attention-intervention interface is wrong.
  • domain assumption Unsafe content is detectable as a linear pattern in per-head attention maps
    Central premise of Eq. (8): a sparse logistic regressor over attention maps can predict unsafe generation. If unsafe signals are non-linear or not attention-localized, R_safe is blind.
  • domain assumption Injecting modified attention maps φ* into the transformer yields a valid velocity field
    Eq. (13) assumes the rest of the transformer layers can consume replaced attention maps without distributional collapse; only KL proximity protects this.
  • domain assumption W_unsafe trained on an undisclosed disjoint set generalizes to I2P, SneakyPrompts, MMA-Diffusion, UnlearnDiffAtk, and other architectures
    The paper's only evidence is §4 results; no analysis of coverage or domain shift is given.

pith-pipeline@v1.3.0-alltime-deepseek · 14968 in / 15745 out tokens · 145765 ms · 2026-08-02T00:37:00.918219+00:00 · methodology

0 comments
read the original abstract

State-of-the-art flow based text-to-image (T2I) models exhibit remarkable generative abilities but remain vulnerable to producing unsafe content. Prior safety efforts range from concept erasure and prompt filtering to classifier-based gating. However, simple techniques like parameter efficient adaptations of the models easily bypass such guardrails. We introduce a unique principled approach that achieves safety by regulating the model's attention dynamics through inference-time introspection, exhibiting intrinsic robustness. Our method analyzes and rebalances attention activations throughout image synthesis, steering generations away from unsafe concepts while preserving semantic alignment. This introspective control ensures safety of deployed models. Across standard and adversarial safety benchmarks, our approach achieves remarkable safety scores while maintaining or even improving alignment and perceptual quality. Our results reveal that attention-space regulation offers a considerably more promising path to safer diffusion transformer based image generation than the existing concept erasing mechanism.Our code can be accessed at https://basim-azam.github.io/iam/

Figures

Figures reproduced from arXiv: 2607.14945 by Basim Azam, Hossein Rahmani, Naveed Akhtar.

Figure 1
Figure 1. Figure 1: Proposed method retains high image quality and semantic alignment while successfully suppressing unsafe content. Representative examples of flux.1-dev outputs are shown. For each image pair: Left is baseline model output and Right is the result with our intervention. Blur is manually added by the authors for discretion. NSFW classifiers [8, 44] to screen inputs or outputs. Yet, they prove brittle to advers… view at source ↗
Figure 2
Figure 2. Figure 2: (Left) Schematics: At each time step t, we ○1 extract attention traces Φt from both dual-stream and single-stream blocks of the transformer and ○2 detect unsafe patterns via logistic regression based pre-learned concept representations Wunsafe. ○3 Modulated attention map φ ∗ t is computed from the identified unsafe weights and ○4 injected to produce new velocity field. ○5 The updated velocity vˆθ steers sa… view at source ↗
Figure 3
Figure 3. Figure 3: Representative examples from I2P [60] using FLUX.1-dev adapted for unsafe generation. Each row shows a different prompt; columns compare methods. Our tech￾nique performs better in both safety and quality. Blur added manually for discretion [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (a) Attention Evolution: Cross-attention maps from semantic (Block 5), mid￾level (Block 11), and output (Block 18) layers. Warmer colors denote stronger attention. The baseline (left) progressively concentrates on unsafe tokens, whereas our method (right) redirects attention toward contextual elements while preserving compositional structure. (b) Safety Control: Generated samples show alignment between att… view at source ↗
Figure 5
Figure 5. Figure 5: Our method generalizes across models with different ar￾chitectures and training objec￾tives. All FLUX variants produce explicit unsafe content under the baseline, whereas our interven￾tion consistently improves safety without retraining or model￾specific tuning. with intermediate settings producing expected gradations. The same pattern holds under adversarial conditions. On SneakyPrompts, nudity rates foll… view at source ↗
Figure 6
Figure 6. Figure 6: Safety-quality Pareto frontier and cross-benchmark consistency. (a) On I2P, our method uniquely achieves both the highest safety score and quality. (b) Consistent performance across I2P and SneakyPrompts. generates explicit unsafe content (top row), while our intervention eliminates inappropriate elements across all variants (bottom row) while preserving their specialized capabilities. This model-agnostic … view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparisons on (a) MMA-Diffusion [73] and (b) UnlearnDif￾fAtk [78]. Safety-quality Analysis [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

80 extracted references · 17 linked inside Pith

  1. [1]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Azam, B., Akhtar, N.: Plug-and-play interpretable responsible text-to-image gener- ation via dual-space multi-facet concept control. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2976–2985 (June 2025) 2

    2025

  2. [2]

    arXiv preprint arXiv:2308.12966 (2023) 8

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J.: Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 (2023) 8

    Pith/arXiv arXiv 2023

  3. [3]

    arXiv preprint arXiv:2211.01324 (2022) 3

    Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Zhang, Q., Kreis, K., Ait- tala, M., Aila, T., Laine, S., et al.: ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324 (2022) 3

    Pith/arXiv arXiv 2022

  4. [4]

    GitHub Repository (2019),https://github.com/notAI-tech/NudeNet8

    Bedapudi, P.: NudeNet: Neural networks for nudity detection and censoring. GitHub Repository (2019),https://github.com/notAI-tech/NudeNet8

    2019

  5. [5]

    Chang,H.,Zhang,H.,Jiang,L.,Liu,C.,Freeman,W.T.:MaskGIT:Maskedgenera- tiveimagetransformer.In:ProceedingsoftheIEEE/CVFConferenceonComputer Vision and Pattern Recognition (CVPR). pp. 11315–11325 (2022) 3

    2022

  6. [6]

    ACM trans- actions on Graphics (TOG)42(4), 1–10 (2023) 3

    Chefer, H., Alaluf, Y., Vinker, Y., Wolf, L., Cohen-Or, D.: Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM trans- actions on Graphics (TOG)42(4), 1–10 (2023) 3

    2023

  7. [7]

    In: ICML

    Chen, M., Radford, A., Child, R., Wu, J., Jun, H., Luan, D., Sutskever, I.: Gener- ative pretraining from pixels. In: ICML. pp. 1691–1703. PMLR (2020) 3

    2020

  8. [8]

    Hugging Face Model Card (2022) 2, 4

    CompVis and LAION: stable-diffusion-safety-checker. Hugging Face Model Card (2022) 2, 4

    2022

  9. [9]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2023) 2

    Conmy, A., Nanda, N., Gurnee, T., DasSarma, N.P., Lieberum, T., Hewitt, J.: Towards automated circuit discovery for mechanistic interpretability. In: Advances in Neural Information Processing Systems (NeurIPS) (2023) 2

    2023

  10. [10]

    In: Ad- vances in Neural Information Processing Systems

    Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. In: Ad- vances in Neural Information Processing Systems. vol. 34, pp. 8780–8794 (2021) 3

    2021

  11. [11]

    arXiv preprint arXiv:2302.09712 (2023) 3

    Du, Z., Lin, H., Zhao, X.: Reduce and focus: Improving text-to-image generation with cross-attention modifications. arXiv preprint arXiv:2302.09712 (2023) 3

    arXiv 2023

  12. [12]

    arXiv preprint arXiv:2209.10652 (2022) 2

    Elhage, N., Nanda, N., Olsson, C., Joseph, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., DasSarma, N., Ringer, S., Mossing, D., Rahtz, M., Olah, C., Ka- plan, J., Amodei, D.: Toy models of superposition. arXiv preprint arXiv:2209.10652 (2022) 2

    Pith/arXiv arXiv 2022

  13. [13]

    In: Proceedings of the 41st International Confer- ence on Machine Learning (ICML)

    Esser, P., Kulal, S., Blattmann, A., et al.: Scaling rectified flow transformers for high-resolution image synthesis. In: Proceedings of the 41st International Confer- ence on Machine Learning (ICML). PMLR (2024) 1, 3, 4, 5, 8

    2024

  14. [14]

    In: CVPR

    Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: CVPR. pp. 12873–12883 (2021) 3 16 B. Azam et al

    2021

  15. [15]

    Advances in Neural Information Processing Systems36, 18225–18250 (2023) 3

    Feng, W., Zhu, W., Fu, T.j., Jampani, V., Akula, A., He, X., Basu, S., Wang, X.E., Wang, W.Y.: Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems36, 18225–18250 (2023) 3

    2023

  16. [16]

    In: Burges, C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K

    Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Ranzato, M.A., Mikolov, T.: Devise: A deep visual-semantic embedding model. In: Burges, C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K. (eds.) Advances in Neural Informa- tion Processing Systems. vol. 26. Curran Associates, Inc. (2013) 3

    2013

  17. [17]

    arXiv preprint arXiv:2208.01618 (2022) 4

    Gal,R.,Alaluf,Y.,Atzmon,Y.,Patashnik,O.,Bermano,A.H.,Chechik,G.,Cohen- Or, D.: An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618 (2022) 4

    Pith/arXiv arXiv 2022

  18. [18]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Gandikota, R., Materzynska, J., Fiotto-Kaufman, J., Bau, D.: Erasing concepts from diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 2426–2436 (2023) 2, 3, 9

    2023

  19. [19]

    Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., Bau, D.: Unified concept editingindiffusionmodels.In:2024IEEE/CVFWinterConferenceonApplications of Computer Vision (WACV). pp. 5099–5108 (2024) 2, 3, 9

    2024

  20. [20]

    ICML 2025 (2024) 2, 3, 9

    Gao, D., Lu, S., Walters, S., Zhou, W., Chu, J., Zhang, J., Zhang, B., Jia, M., Zhao, J., Fan, Z., et al.: Eraseanything: Enabling concept erasure in rectified flow transformers. ICML 2025 (2024) 2, 3, 9

    2025

  21. [21]

    In: Advances in Neural Information Processing Systems

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems. pp. 2672–2680 (2014) 3

    2014

  22. [22]

    Hugging Face Model Repository (2024),https: //huggingface.co/Heartsync/Flux-NSFW-uncensored, nSFW LoRA adapter for FLUX.1 models 8

    Heartsync: Flux-nsfw-uncensored. Hugging Face Model Repository (2024),https: //huggingface.co/Heartsync/Flux-NSFW-uncensored, nSFW LoRA adapter for FLUX.1 models 8

    2024

  23. [23]

    Advances in Neural Information Processing Systems (NeurIPS)33, 6840–6851 (2020) 3

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS)33, 6840–6851 (2020) 3

    2020

  24. [24]

    arXiv preprint arXiv:2207.12598 (2021) 3

    Ho, J., Salimans, T.: Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 (2021) 3

    Pith/arXiv arXiv 2021

  25. [25]

    arXiv preprint arXiv:2106.09685 (2021),https://arxiv.org/abs/2106.096851, 4, 5, 8

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021),https://arxiv.org/abs/2106.096851, 4, 5, 8

    Pith/arXiv arXiv 2021

  26. [26]

    arXiv preprint arXiv:2302.09778 (2023) 3

    Huang, L., Chen, D., Liu, Y., Shen, Y., Zhao, D., Zhou, J.: Composer: Cre- ative and controllable image synthesis with composable conditions. arXiv preprint arXiv:2302.09778 (2023) 3

    Pith/arXiv arXiv 2023

  27. [27]

    In: Proceedings of the IEEE Conference on Computer Vision (ICCV)

    Jia, Y., Salzmann, M., Darrell, T.: Learning cross-modal similarities between text and images. In: Proceedings of the IEEE Conference on Computer Vision (ICCV). pp. 3592–3599 (2011) 3

    2011

  28. [28]

    In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C

    Jiang, D., Song, G., Wu, X., Zhang, R., Shen, D., Zong, Z., Liu, Y., Li, H.: Comat: Aligning text-to-image diffusion model with image-to-text concept matching. In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural Information Processing Systems. vol. 37, pp. 76177– 76209. Curran Associates, Inc....

    2024

  29. [29]

    arXiv preprint arXiv:2304.05012 (2023) 3

    Jiang, J., Wu, Z., Li, J.: Avatarcraft: Zero-shot text-to-avatar generation via diffu- sion models. arXiv preprint arXiv:2304.05012 (2023) 3

    Pith/arXiv arXiv 2023

  30. [30]

    arXiv preprint arXiv:2304.01578 (2023) 3

    Jiang, W., Zhang, H., Zhao, J.: Object-aware editing with diffusion models. arXiv preprint arXiv:2304.01578 (2023) 3

    Pith/arXiv arXiv 2023

  31. [31]

    In: Proceedings Introspective Attention Modulation for Safe T2I Generation 17 of the IEEE/CVF International Conference on Computer Vision

    Kulikov, V., Kleiner, M., Huberman-Spiegelglas, I., Michaeli, T.: Flowedit: Inversion-free text-based editing using pre-trained flow models. In: Proceedings Introspective Attention Modulation for Safe T2I Generation 17 of the IEEE/CVF International Conference on Computer Vision. pp. 19721–19730 (2025) 3, 9

    2025

  32. [32]

    arXiv preprint arXiv:2503.08250 (2025) 3

    Lee, J.Y., Cha, B., Kim, J., Ye, J.C.: Aligning text to image in diffusion models is easier than you think. arXiv preprint arXiv:2503.08250 (2025) 3

    arXiv 2025

  33. [33]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Li, H., Wang, C., Wang, C., Tao, D.: Self-discovering interpretable diffusion la- tent directions for responsible text-to-image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8525–8535 (2024) 2

    2024

  34. [34]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Li, H., Wang, C., Wang, C., Tao, D.: Self-discovering interpretable diffusion la- tent directions for responsible text-to-image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8525–8535 (2024) 4

    2024

  35. [35]

    arXiv preprint arXiv:2305.13655 (2023) 3

    Lian, L., Li, B., Yala, A., Darrell, T.: Llm-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models. arXiv preprint arXiv:2305.13655 (2023) 3

    Pith/arXiv arXiv 2023

  36. [36]

    In: European Conference on Computer Vision (ECCV)

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: European Conference on Computer Vision (ECCV). pp. 740–755 (2014) 9

    2014

  37. [37]

    In: International Conference on Learning Representations (ICLR) (2023) 1

    Lipman, Y., Chen, R.T.Q., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. In: International Conference on Learning Representations (ICLR) (2023) 1

    2023

  38. [38]

    Liu, X., et al.: To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now. In: European Conference on Computer Vision (ECCV) (2024) 9

    2024

  39. [39]

    arXiv preprint arXiv:2209.03003 (2022) 1, 3

    Liu, X., Gong, C., Liu, Q.: Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 (2022) 1, 3

    Pith/arXiv arXiv 2022

  40. [40]

    In: Euro- pean Conference on Computer Vision (ECCV) Workshops

    Liu, Z., Liu, R., He, P., Chen, Z., Yu, L., Li, S., He, X., Cai, D.: Latent guard: A safety framework for text-to-image generation in text embedding space. In: Euro- pean Conference on Computer Vision (ECCV) Workshops. pp. 389–406. Springer (2024) 2, 4

    2024

  41. [41]

    In: International Conference on Machine Learning (ICML) (2025) 3, 9

    Lu, R., et al.: Minimalist concept erasure in generative models. In: International Conference on Machine Learning (ICML) (2025) 3, 9

    2025

  42. [42]

    Lv, Z., Pan, T., Si, C., Chen, Z., Zuo, W., Liu, Z., Wong, K.Y.K.: Rethinking cross-modal interaction in multimodal diffusion transformers (2025) 3

    2025

  43. [43]

    arXiv preprint arXiv:2112.10741 (2021) 3

    Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., Chen, M.: Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 (2021) 3

    Pith/arXiv arXiv 2021

  44. [44]

    OpenAI: Dall·e 3 system card (2023),https://openai.com/research/dall-e-3- system-card2, 4

    2023

  45. [45]

    In: IEEE/CVF Winter Conference on Applica- tions of Computer Vision (WACV)

    Park, S., Oh, S.J., Han, B.: Localization and manipulation of immoral visual cues for safe Text-to-Image generation. In: IEEE/CVF Winter Conference on Applica- tions of Computer Vision (WACV). pp. 3132–3141 (2024) 4

    2024

  46. [46]

    Under review (2024) 2

    Park, Y., Park, J., Kim, J., Hong, S.: Dissecting and mitigating diffusion bias via mechanistic interpretability. Under review (2024) 2

    2024

  47. [47]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 4195– 4205 (2023) 1, 3, 5

    2023

  48. [48]

    In: ICLR (2024) 14

    Podell, D., English, Z., Lacey, K., et al.: SDXL: Improving latent diffusion models for high-resolution image synthesis. In: ICLR (2024) 14

    2024

  49. [49]

    In: CVPR

    Qiao, T., Zhang, J., Xu, D., Tao, D.: Mirrorgan: Learning text-to-image generation by redescription. In: CVPR. pp. 1505–1514 (2019) 3 18 B. Azam et al

    2019

  50. [50]

    In: International Conference on Machine Learning (ICML)

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning (ICML). pp. 8748–8763 (2021) 8

    2021

  51. [51]

    arXiv preprint arXiv:2204.06125 (2022) 1, 3

    Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125 (2022) 1, 3

    Pith/arXiv arXiv 2022

  52. [52]

    In: ICML

    Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML. pp. 8821–8831. PMLR (2021) 3

    2021

  53. [53]

    arXiv preprint arXiv:2210.04610 (2022) 1, 3

    Rando, J., Paleka, D., Lindner, D., Heim, L., Tramèr, F.: Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610 (2022) 1, 3

    Pith/arXiv arXiv 2022

  54. [54]

    NeurIPS32(2019) 3

    Razavi, A., Van den Oord, A., Vinyals, O.: Generating diverse high-fidelity images with vq-vae-2. NeurIPS32(2019) 3

    2019

  55. [55]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Reed, S., Akata, Z., Lee, H., Schiele, B.: Learning deep representations of fine- grained visual descriptions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 49–58 (2016) 3

    2016

  56. [56]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10684– 10695 (2022) 1, 3, 5, 10

    2022

  57. [57]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Aberman, K., Rubinstein, M.: Dream- Booth: Fine tuning text-to-image diffusion models for subject-driven generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 22500–22510 (2023) 1, 4

    2023

  58. [58]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2022),https://arxiv.org/abs/2205.114871, 3

    Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S.K.S., Ayan, B.K., Mahdavi, S.S., Lopes, R.G., Salimans, T., Ho, J., Fleet, D.J., Norouzi, M.: Photorealistic text-to-image diffusion models with deep language un- derstanding. In: Advances in Neural Information Processing Systems (NeurIPS) (2022),https://arxiv.org/abs/2205.114871, 3

    Pith/arXiv arXiv 2022

  59. [59]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Schramowski, P., Brack, M., Deiseroth, B., Friedrich, F., Hintersdorf, D., Ommer, B., Kersting, K.: Safe latent diffusion: Mitigating inappropriate content genera- tion in text-to-image models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 22522–22531 (2023) 2, 3, 9

    2023

  60. [60]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Schramowski, P., Brack, M., Deiseroth, B., Friedrich, F., Hintersdorf, D., Ommer, B., Kersting, K.: Safe latent diffusion: Mitigating inappropriate content genera- tion in text-to-image models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 22522–22531 (2023) 2, 4, 10

    2023

  61. [61]

    In: Ad- vances in Neural Information Processing Systems (NeurIPS) Datasets and Bench- marks (2022) 1, 3

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C.W., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kun- durthy, S.R., Crowson, K., Schmidt, L., Kaczmarczyk, R., Jitsev, J.: LAION-5b: An open large-scale dataset for training next generation image-text models. In: Ad- vances in Neural Information Processing Sy...

    2022

  62. [62]

    In: International Conference on Machine Learning (ICML)

    Sohl-Dickstein, J., Weiss, E.A., Maheswaranathan, N., Ganguli, S.: Deep unsuper- vised learning using nonequilibrium thermodynamics. In: International Conference on Machine Learning (ICML). pp. 2256–2265 (2015) 3 Introspective Attention Modulation for Safe T2I Generation 19

    2015

  63. [63]

    Advances in Neural Information Processing Systems (NeurIPS)32 (2019) 3

    Song, Y., Ermon, S.: Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems (NeurIPS)32 (2019) 3

    2019

  64. [64]

    In: Interna- tional Conference on Learning Representations (ICLR) (2021) 1

    Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. In: Interna- tional Conference on Learning Representations (ICLR) (2021) 1

    2021

  65. [65]

    In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL)

    Tang, R., Liu, L., Pandey, A., Jiang, Z., Yang, G., Kumar, K., Stenetorp, P., Lin, J., Ture, F.: What the DAAM: Interpreting stable diffusion using cross attention. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL). pp. 5624–5643 (2023) 2, 5

    2023

  66. [66]

    Tsai, Y.H.H., Su, J.W., Yeh, S.Y., Ravikumar, P., Salakhutdinov, R.: Ring-a-bell! how reliable are concept removal methods for diffusion models? In: International Conference on Learning Representations (ICLR) (2024) 2

    2024

  67. [67]

    Tsai, Y.H., Su, J., Yeh, S., Ravikumar, P., Salakhutdinov, R.: Ring-a-bell! how reliable are concept removal methods for diffusion models? In: International Con- ference on Learning Representations (ICLR) (2024) 4

    2024

  68. [68]

    arXiv preprint arXiv:2204.01697 (2022) 3

    Tu, Z., Talebi, H., Zhang, H., Yang, F., Milanfar, P., Bovik, A., Li, Y.: MaxViT: Multi-axis vision transformer. arXiv preprint arXiv:2204.01697 (2022) 3

    Pith/arXiv arXiv 2022

  69. [69]

    In: Advances in Neural Information Processing Systems

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems. pp. 5998–6008 (2017) 3

    2017

  70. [70]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Wang, R., Chen, Z., Chen, C., Ma, J., Lu, H., Lin, X.: Compositional text-to-image synthesis with attention map control of diffusion models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 5544–5552 (2024) 3

    2024

  71. [71]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., Shou, M.Z.: Boxdiff: Text- to-image synthesis with training-free box-constrained diffusion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 7452–7461 (2023) 3

    2023

  72. [72]

    arXiv preprint arXiv:2508.03006 (2025), https://arxiv.org/abs/2508.0300610

    Yang, S., Tian, Y., Song, G., Liu, Y., Luo, P.: In-generation multi-modal con- tent moderation for text-to-image models. arXiv preprint arXiv:2508.03006 (2025), https://arxiv.org/abs/2508.0300610

    Pith/arXiv arXiv 2025

  73. [73]

    In: CVPR (2024) 3, 9, 13

    Yang,Y.,Gao,R.,Wang,Y.,etal.:MMA-diffusion:MultiModalattackondiffusion models. In: CVPR (2024) 3, 9, 13

    2024

  74. [74]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2024) 1, 2, 4

    Yang, Y., Ye, X., Zheng, M., Wu, B., Wang, Z., Qin, Z., Xu, Q.: GuardT2I: Safety red teaming and model alignment for text-to-image generation. In: Advances in Neural Information Processing Systems (NeurIPS) (2024) 1, 2, 4

    2024

  75. [75]

    In: Proceedings of the IEEE Symposium on Security and Privacy (2024) 1, 2, 3, 4, 9

    Yang, Y., Hui, B., Yuan, H., Gong, N., Cao, Y.: Sneakyprompt: Jailbreaking text- to-image generative models. In: Proceedings of the IEEE Symposium on Security and Privacy (2024) 1, 2, 3, 4, 9

    2024

  76. [76]

    In: ICCV

    Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., Metaxas, D.N.: Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In: ICCV. pp. 5907–5915 (2017) 3

    2017

  77. [77]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 3836–3847 (2023) 4

    2023

  78. [78]

    Zhang, Y., Jia, J., Chen, X., et al.: To generate or not? Safety-Driven unlearned diffusion models are still easy to generate unsafe images... for now. In: ECCV (2024) 3, 13

    2024

  79. [79]

    Springer (2024) 1, 2, 4 20 B

    Zheng, A.Y., Yeh, R.A.: IMMA: Immunizing text-to-image models against mali- ciousadaptation.In:EuropeanConferenceonComputerVision(ECCV).pp.19–36. Springer (2024) 1, 2, 4 20 B. Azam et al

    2024

  80. [80]

    In: CVPR

    Zhu, M., Pan, P., Chen, W., Yang, Y.: Dm-gan: Dynamic memory generative ad- versarial networks for text-to-image synthesis. In: CVPR. pp. 5802–5810 (2019) 3

    2019