Pith. sign in

REVIEW 4 major objections 7 minor 4 cited by

Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Benign fine-tuning can undo the safety alignment of text-to-image diffusion models, and Modular LoRA prevents this by keeping the safety module detached during fine-tuning and reattaching it only at inference.

desk verdict The paper demonstrates a real, practically important failure mode and a simple, likely effective fix, but the negative-transfer mechanism it leans on is supported more by intuition than by measurement. read the letter →

arxiv 2412.00357 v1 pith:H52C72UG submitted 2024-11-30 cs.AI cs.CV

Sanghyun Kim , Moonseok Choi , Jinwoo Shin , Juho Lee This is my paper · ORCID
classification cs.AIcs.CV
keywords safetyalignmentfine-tuningjailbreakingtext-to-imagediffusionmodelsconceptremovalLoRAModularNSFWcontentsuppressionmodelmerging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fine-tuning a safety-aligned text-to-image diffusion model on an innocuous dataset can undo its safety alignment: nudity, artist styles, or signatures that the model was trained to suppress re-emerge in generated images, and larger or more conceptually similar datasets make the effect stronger. The paper argues this is not merely forgetting or damage to the filter; fine-tuning actively re-learns the suppressed concept from the model's own weights, and in some setups the effect exceeds the unaligned base model. The proposed remedy, Modular LoRA, isolates the safety behavior in a separate low-rank adapter, removes that adapter while the task-specific fine-tuning runs, and reattaches it at inference. The paper reports that this keeps exposed-body-part rates close to the pre-fine-tuning aligned levels across three anime-style datasets while preserving downstream task quality.

What carries the argument

The load-bearing object is the additive weight decomposition $W = W_0 + \Delta W_{\mathrm{safe}} + \Delta W_{\mathrm{ft}}$, where LoRA (low-rank adaptation) represents each weight update as a product of two small matrices. $\Delta W_{\mathrm{safe}}$ is the safety adapter trained by a concept-removal method such as ESD or SDD; $\Delta W_{\mathrm{ft}}$ is the adapter learned during downstream fine-tuning. The paper shows that with standard fine-tuning, $\Delta W_{\mathrm{ft}}$ re-encodes the inverse of $\Delta W_{\mathrm{safe}}$, and Modular LoRA trains the fine-tuning adapter as $\Delta W^*_{\mathrm{ft}}$ with $\Delta W_{\mathrm{safe}}$ detached, then merges at inference as $W^* = W_0 + \Delta W_{\mathrm{safe}} + \Delta W^*_{\mathrm{ft}}$.

What would settle it

Fine-tune the same safety-aligned model on the Danbooru dataset twice: once with the safety LoRA attached as in standard fine-tuning, and once with it detached as in Modular LoRA, then compare the fraction of images flagged by an automated NSFW detector over training steps. If the detached run shows the same early spike in unsafe images as the standard run, or if reattaching the safety module at inference no longer suppresses nudity, the proposed mechanism is not what stops re-emergence.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuning a safety-aligned text-to-image diffusion model on benign data reactivates the exact concepts the alignment had suppressed, rather than simply damaging or forgetting the safety behavior. The evidence includes FLUX.1 producing artist signatures and explicit content after a few thousand fine-tuning steps on a clean Pokémon dataset, and ESD- and SDD-aligned Stable Diffusion models showing a sharp rise in unsafe images in the first hundreds of fine-tuning steps, before the benign target style has been learned. Weight-negation experiments support the proposed mechanism: removing the safety adapter makes a safe model generate nudity from an innocuous prompt, while removing the fine-tuning adapter suppresses nudity even under explicitly NSFW prompts. The solution is Modular LoRA, which trains the fine-tuning adapter with the safety adapter detached and merges them only at inference as $W^* = W_0 + \Delta W_{\mathrm{safe}} + \Delta W^*_{\mathrm{ft}}$.

Load-bearing premise

The whole fix rests on the assumption that a safety alignment lives in a detachable additive weight module, and that training the task module without that safety module attached prevents the model from re-learning the suppressed concept.

Editorial extensions

If this is right

  • Fine-tuning APIs that let users adapt a safety-aligned image model on their own data can silently reintroduce NSFW output even when the uploaded data is clean; Modular LoRA gives such services a simple way to keep the safety component intact.
  • The early-stage spike in unsafe images means safety evaluations should look at early checkpoints of fine-tuning, not only the final model.
  • Concept-removal methods implemented as LoRA modules become reusable components: the same safety module can stay attached during inference across many downstream fine-tunes.
  • The method preserves downstream task quality: DreamBooth image and text alignment scores with Modular LoRA are comparable to full fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same ‘detach the invariant module’ principle applies to any attribute a deployer wants to keep fixed during fine-tuning, such as a watermark, a face identity, or a licensed style, not only to safety concepts.
  • A testable extension is to measure the principal subspaces of $\Delta W_{\mathrm{safe}}$ and $\Delta W^*_{\mathrm{ft}}$; if the modules occupy nearly orthogonal low-rank subspaces, then the method’s success is a subspace-separation phenomenon and could be predicted from the spectra of the two adapters.
  • The finding that suppressed concepts can be amplified by fine-tuning suggests that alignment should be treated as a persistent constraint in the fine-tuning objective, not as a one-time preprocessing stage.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper studies fine-tuning jailbreaking in text-to-image diffusion models: fine-tuning a safety-aligned model (ESD, SDD, or a commercially aligned model such as FLUX.1) on benign anime-style datasets can cause previously suppressed unsafe content, such as nudity or artist signatures, to reappear. The authors attribute this to a negative-transfer mechanism in which the fine-tuning LoRA ΔW_ft learns the inverse of the safety LoRA ΔW_safe, and they propose Modular LoRA: train a safety LoRA separately, detach it during downstream fine-tuning, and re-attach it at inference time (Eq. 3). The main experiments compare full fine-tuning, standard LoRA fine-tuning, and Modular LoRA on Pokemon, Naruto, and Danbooru datasets, reporting that Modular LoRA greatly reduces the percentage of images flagged as unsafe while maintaining downstream generation quality.

Significance. If the empirical result holds, the paper documents a practically important vulnerability: fine-tuning safety-aligned text-to-image models on benign data can degrade safety, and a simple modular training procedure can substantially mitigate this degradation. The main quantitative gaps are large and consistent, e.g., in Table 2 the average exposed-body-part rate for SDD is 1.8% with Modular LoRA versus 29.1% with standard LoRA fine-tuning, and the effect replicates across three datasets, two safety methods, and multiple prompt types. The paper also provides useful complementary results, including FLUX.1 jailbreaking examples, a curated Danbooru subset, and detailed experimental appendices. However, the paper's central explanatory mechanism—that ΔW_ft actively re-learns +nudity via negative transfer from ΔW_safe—is supported only by qualitative single-prompt weight-negation visualizations, and several quantitative claims are stated without uncertainty estimates. The empirical contribution is therefore stronger than the mechanistic story, and the revision should either substantiate the mechanism or carefully delimit the claims.

major comments (4)
  1. [Sec. 3.4 and Figs. 8-9] The claim that the fine-tuning LoRA ΔW_ft acquires the inverse of ΔW_safe through negative transfer is the stated cause of jailbreaking and the motivation for Modular LoRA in Sec. 4, yet the only evidence is two qualitative, single-prompt comparisons for "a sexy cute girl with a cat" (Figs. 8 and 9). No quantitative measurement of what ΔW_ft encodes is reported, there are no confidence intervals, and there is no control for the mere presence of a frozen LoRA during fine-tuning. An alternative explanation, namely that fine-tuning W0 directly suppresses the base model's unsafe tendencies while fine-tuning W0+ΔW_safe drifts because the benign data does not reinforce the already-safe behavior, would also explain the data without requiring negative transfer. Please add quantitative support, for example by evaluating W0+ΔW_ft versus W0+ΔW*_ft on a multi-prompt NSFW benchmark, measuring the projection or cosine similarity between ΔW_ft and ΔW_safe, and running a control experiment with an unrelated frozen LoRA attached during fine-tuning. If such measurements are not available, the mechanism claim should be softened to a hypothesis.
  2. [Tables 2 and 3] The text says Modular LoRA "prevents the re-learning of harmful content" (Sec. 4) and that it "successfully mitigates jailbreaking" (Sec. 5.2), but the before/after numbers show residual safety degradation on several datasets. In Table 2, ESD Modular goes from 4.5% before fine-tuning to 7.3% on Naruto and 8.5% on Danbooru, with an average of 6.1% that is above the 4.5% before value. In Table 3, ESD Modular increases from 41.4% before to 48.7% on average after fine-tuning. The claims should be calibrated to these increases, and the discussion should address why Modular LoRA reduces but does not eliminate the re-learning effect, especially on datasets with more human and nuanced content.
  3. [Tables 2-4 and Appendix C] No error bars, confidence intervals, or significance tests are reported for any of the main safety tables. This matters because Appendix C.2 and C.3 explicitly acknowledge that the NudeNet and Q16 classifiers have high false-positive rates and that small percentage differences are not considered significant. The headline gaps in Tables 2 and 3 are large and probably robust, but the smaller comparisons, such as ESD Modular 6.1% versus its 4.5% before value, need uncertainty quantification. Please report means and standard errors over multiple fine-tuning seeds or bootstrap over generated images, and state whether the same random seeds and prompt sets were used across compared methods.
  4. [Eq. (2) and Table 1] Equation (2) writes the standard fine-tuning pipeline as W' = W0 + ΔW_safe + ΔW_ft, which presupposes that the initial safety alignment is itself represented as a LoRA module. However, Table 1 includes Full -> Full and Full -> LoRA configurations where the safety alignment is a full-model update and the additive decomposition in Eq. (2) does not literally hold. This makes it unclear whether the proposed negative-transfer analysis applies to safety alignments that are not LoRA-based, and it complicates the comparison between the Full and LoRA rows in Table 1. Please clarify the notation and state explicitly that the mechanism analysis and Modular LoRA apply to LoRA-based safety alignment, with full-model safety alignment handled only empirically.
minor comments (7)
  1. [Abstract] The phrase "novel but immediate solution" is awkward; consider "novel and immediately applicable solution" or similar.
  2. [Table 4] The table header says "average CLIP similarity" and the caption says lower is better, but CLIP score is conventionally higher-is-better. Please add an explicit note in the table or caption that a lower value indicates greater style-removal effectiveness, not lower text-image alignment in general.
  3. [Sec. 3.1 and Fig. 1] The statement that fine-tuning "actively reactivates latent concepts within the model's weights" is stronger than what Fig. 1 alone demonstrates; the increase in signature generation could also arise from a style shift that makes the model more likely to render text-like artifacts. Please present this as an interpretation or provide a control that directly tests the reactivation claim.
  4. [Appendix D.1] The hyperparameter narrative would be easier to follow as a table: ESD uses a LoRA learning rate of 1e-4 for NSFW and 1e-5 for artist removal, while SDD uses 1e-5 in both cases, and the fine-tuning step counts also differ. A compact table would reduce the risk of readers misapplying these settings.
  5. [Fig. 13 and Appendix E.1] The Dreambooth example in Fig. 13 uses an explicit evaluation prompt ("gorgeous, naked, without clothes") but the caption describes it only as Dreambooth training on five sneakers images. Please state in the caption that the evaluation prompt is deliberately explicit, since this is important for interpreting the reported harmful-image percentages.
  6. [Table 6] For DINO feature consistency, the direction of improvement is not stated. Please specify whether higher DINO values are better, as is implicitly suggested by the comparison.
  7. [Fig. 5 caption] The caption uses shorthand like "Full -> Full in Tab. 1" without defining the arrow notation in the caption itself; a one-sentence definition would help readers who are not looking at Table 1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the direct safety floor of Modular LoRA is by construction, but the load-bearing task-LoRA comparison is empirical and externally evaluated.

full rationale

The derivation chain does not reduce to its inputs. Modular LoRA's inference model (Eq. 3) retains the frozen safety module ΔWsafe, so preserving the safety module's own suppression behavior is definitional; the paper presents this as the method's mechanism, not as a fitted prediction. The central empirical content is the comparison between a fine-tuning LoRA trained with the safety module attached (ΔWft) and one trained without it (ΔW*_ft): Fig. 13 reports 62% vs 32% harmful images on W0, and Tables 2-4 compare LoRA vs Modular rows using external classifiers (NudeNet, Q16, CLIP). Those differences are not forced by Eq. 3 and could have gone the other way. The Sec. 3.4 causal claim that ΔWsafe drives ΔWft to learn +nudity is inferred from single-prompt weight-negation visuals and is vulnerable to a base-model confound, but that is an evidentiary weakness rather than a circular reduction. The paper's self-citations (SDD [32,33], evaluation thresholds from Kim et al. [32]) are non-load-bearing because the jailbreaking result and the method are also demonstrated with external methods ESD and MACE and external benchmark prompts. No equation is equal to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or architectural entities. The central claim is empirical and depends primarily on the validity of automatic safety classifiers, the safety of the fine-tuning datasets, and the additivity of safety LoRA weights. Several hand-chosen hyperparameters affect the magnitude of the measured effect but not the existence of the phenomenon.

free parameters (3)
  • LoRA rank = 16
    Rank used for all LoRA fine-tuning in the paper, including FLUX.1 and ESD/SDD experiments. Chosen by hand following HuggingFace defaults, not fitted to data, but it affects the capacity of both safety and task adapters.
  • LoRA fine-tuning learning rate = 1e-4 for NSFW/harmful removal, 1e-5 for artist removal
    The authors adjusted the learning rate per task because 1e-4 caused degenerate outputs for artist style removal (Appendix D.1). This hand-chosen knob affects measured jailbreaking severity and downstream quality.
  • Fine-tuning steps = 5000 steps for downstream fine-tuning; 1500 for ESD/SDD LoRA removal; 1000 for artist removal
    Training durations were selected to show consistent jailbreaking effects and to avoid overfitting or instability, as described in Appendix D. Results early in training differ from later stages.
assumptions (4)
  • domain assumption NudeNet v3 and Q16 classifiers are valid proxies for image safety, and the reported percentages reflect meaningful safety differences.
    Used as the primary evaluation metrics in Sec. 5.1 and Tabs. 2-3. The paper itself notes in Appendix C.2 that these classifiers have high false-positive rates, so small differences are not meaningful; large differences are assumed to be real.
  • domain assumption The fine-tuning datasets (Pokemon, Naruto, and the curated Danbooru subset) contain no harmful images or captions.
    Relied on in Sec. 3.2 and Appendix C.1 to support the claim that jailbreaking is not caused by learning from harmful data. The paper manually reviewed captions and shows word clouds, but Danbooru is acknowledged to contain sexually nuanced imagery, and word clouds are weak evidence of absence.
  • ad hoc to paper Safety alignment from ESD and SDD can be represented as an additive LoRA weight that can be detached and re-merged without changing its behavior.
    This is the core assumption of Modular LoRA in Sec. 4, Eq. 3. It is supported only by a few weight-negation visualizations (Figs. 8-9) on single prompts, not by a systematic study of module independence.
  • domain assumption FLUX.1 [dev] is safety-aligned, and increases in NSFW or signature outputs after fine-tuning reflect breakdown of alignment rather than transfer from the fine-tuning data.
    Used in Sec. 3.1 and Appendix B.1. FLUX.1's training data and alignment procedures are not public, so this premise cannot be independently verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models." pith.science (2026). https://pith.science/paper/H52C72UG

@misc{pith2026241200357,
  author       = {Pith},
  title        = {Pith review of: Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H52C72UG}},
  note         = {Machine review of arXiv:2412.00357}
}
read the original abstract

Fine-tuning text-to-image diffusion models is widely used for personalization and adaptation for new domains. In this paper, we identify a critical vulnerability of fine-tuning: safety alignment methods designed to filter harmful content (e.g., nudity) can break down during fine-tuning, allowing previously suppressed content to resurface, even when using benign datasets. While this "fine-tuning jailbreaking" issue is known in large language models, it remains largely unexplored in text-to-image diffusion models. Our investigation reveals that standard fine-tuning can inadvertently undo safety measures, causing models to relearn harmful concepts that were previously removed and even exacerbate harmful behaviors. To address this issue, we present a novel but immediate solution called Modular LoRA, which involves training Safety Low-Rank Adaptation (LoRA) modules separately from Fine-Tuning LoRA components and merging them during inference. This method effectively prevents the re-learning of harmful content without compromising the model's performance on new tasks. Our experiments demonstrate that Modular LoRA outperforms traditional fine-tuning methods in maintaining safety alignment, offering a practical approach for enhancing the security of text-to-image diffusion models against potential attacks.

Figures

Figures reproduced from arXiv: 2412.00357 by the authors.

Figure 1
Figure 1. After fine-tuning FLUX.1 for 2,000 steps on Pok [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Impact of fine-tuning ESD on safety performance with varying numbers of training images on Pokemon dataset. Results ´ display the percentage of unsafe images generated over fine-tuning steps, with darker lines representing larger training sets (from 5 to 848 images ). Models fine-tuned on larger datasets tend to produce more unsafe images over time, while those trained with fewer images exhibit early-stage safety de… view at source ↗
Figure 4
Figure 4. Impact of fine-tuning SDD on safety performance with varying numbers of training images on Pokemon dataset. Results ´ demonstrate that models trained on larger datasets (darker lines) exhibit a significant increase in the generation of unsafe images, with some reaching up to 70% unsafe content [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (11 more)
Figure 8
Figure 8. Figure 8: Negation of safety LoRA (∆Wsafe) leads to generating harmful images even with a nuanced prompt such as "a sexy cute girl with a cat". Exposed body parts are masked by the authors (marked ⋆). (a) SD v1.4 W0 (b) W0 − ∆Wft [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 6
Figure 6. Figure 6: Images generated at every 200 fine-tuning steps. We can [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 9
Figure 9. Figure 9: Negation of FT LoRA (∆Wft) somewhat miti￾gates exhibiting nudity concept even when a hash prompt is provided: "a sexy cute girl with a cat, nudity, naked body". Exposed body parts are masked by the authors (marked ⋆). Such conjecture aligns with the observed discrepanc…
Figure 11
Figure 11. Figure 11: Samples from anime-style datasets with three different [PITH_FULL_IMAGE:figures/full_fig_p006_11.png]
Figure 12
Figure 12. Figure 12: We fine-tune the ESD model (top) with the Pokemon ´ dataset, which recovers the removed Gogh concept (bottom). We prompted Gogh’s famous artwork titles to generate images. Tab. 2 shows the percentage of images with exposed body parts (where lower percentages indicate …
Figure 13
Figure 13. Figure 13: Dreambooth training on five sneakers images. Jointly [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]
Figure 14
Figure 14. Figure 14: After fine-tuning FLUX.1 for 1,500 steps on Danbooru [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: After fine-tuning FLUX.1 for 2,000 steps on Danbooru [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Visualization of jailbreaking under different safety [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: Wordclouds of each dataset’s captions. We can readily [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Danbooru website interface. • [PITH_FULL_IMAGE:figures/full_fig_p019_18.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safety

    cs.CY 2026-06 accept novelty 6.5 of 10

    Legal and ethical bans on CSAM access and generation break standard AI safety techniques, creating 15 open problems that demand new methods for dataset cleaning, concept fusion prevention, fine-tuning resilience, dete...

  2. How Robust is Model Editing after Fine-Tuning? An Empirical Study on Text-to-Image Diffusion Models

    cs.AI 2025-06 conditional novelty 6.0 of 10

    Fine-tuning text-to-image diffusion models generally undoes prior model edits, with DoRA causing the strongest reversal and UCE edits being most robust.

  3. Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling

    cs.LG 2025-05 conditional novelty 6.0 of 10

    RPG-RT iteratively fine-tunes an LLM with rule-based preferences from a decoupled CLIP scoring model, letting it rewrite prompts that bypass unknown safety defenses in black-box text-to-image systems.

  4. Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts

    cs.CV 2025-04 conditional novelty 6.0 of 10

    ANT erases unwanted concepts from text-to-image diffusion models by reversing the guidance direction only during mid-to-late denoising, preserving early structural quality.

Reference graph

Works this paper leans on

92 extracted references · 30 canonical work pages · cited by 4 Pith papers

  1. [1]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 2, 1

    2023

  2. [2]

    Demystifying mmd gans

    Mikołaj Bi ´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. arXiv preprint arXiv:1801.01401, 2018. 5, 6

    arXiv 2018

  3. [3]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Alt- man, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. 1

    arXiv 2021

  4. [4]

    Sega: Instructing text-to-image models using semantic guidance

    Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kristian Kersting. Sega: Instructing text-to-image models using semantic guidance. Advances in Neural Information Processing Systems , 36: 25365–25389, 2023. 2, 1

    2023

  5. [5]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2, 1

    2023

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 8

    2021

  7. [7]

    Naruto blip captions

    Eole Cervenka. Naruto blip captions. https : / / huggingface . co / datasets / lambdalabs / naruto-blip-captions/, 2022. 6, 7, 3, 4

    2022

  8. [8]

    Muse: Text-to-image generation via masked generative transform- ers

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Mur- phy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transform- ers. arXiv preprint arXiv:2301.00704, 2023. 1

    arXiv 2023

Show all 92 references
  1. [9]

    Acceler- ating large language model decoding with speculative sam- pling

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean- Baptiste Lespiau, Laurent Sifre, and John Jumper. Acceler- ating large language model decoding with speculative sam- pling. arXiv preprint arXiv:2302.01318, 2023. 2

    2023 arXiv

  2. [10]

    Prompting4debugging: Red- teaming text-to-image diffusion models by finding problem- atic prompts

    Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin- Yu Chen, and Wei-Chen Chiu. Prompting4debugging: Red- teaming text-to-image diffusion models by finding problem- atic prompts. arXiv preprint arXiv:2309.06135, 2023. 2, 1

    2023

  3. [11]

    Emu: Enhanc- ing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xi- aofang Wang, Abhimanyu Dubey, et al. Emu: Enhanc- ing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807, 2023. 2, 1

    2023 arXiv

  4. [12]

    Controlled text generation via language model arithmetic

    Jasper Dekoninck, Marc Fischer, Luca Beurer-Kellner, and Martin Vechev. Controlled text generation via language model arithmetic. arXiv preprint arXiv:2311.14479 , 2023. 3, 5, 2

    2023 arXiv

  5. [13]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 2, 1

    2021

  6. [14]

    Interpreting the weight space of customized dif- fusion models

    Amil Dravid, Yossi Gandelsman, Kuan-Chieh Wang, Rameen Abdal, Gordon Wetzstein, Alexei A Efros, and Kfir Aberman. Interpreting the weight space of customized dif- fusion models. arXiv preprint arXiv:2406.09413, 2024. 3, 2

    2024 arXiv

  7. [15]

    Diffusion in style

    Martin Nicolas Everaert, Marco Bocchio, Sami Arpa, Sabine S¨usstrunk, and Radhakrishna Achanta. Diffusion in style. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2251–2261, 2023. 7

    2023

  8. [16]

    Erasing concepts from diffusion models

    Rohit Gandikota, Joanna Materzynska, Jaden Fiotto- Kaufman, and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 2426–2436, 2023. 2, 3, 4, 6, 7, 8, 1, 5

    2023

  9. [17]

    Unified concept editing in diffusion models

    Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy´nska, and David Bau. Unified concept editing in diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 5111–5120, 2024. 2, 6, 1

    2024

  10. [18]

    Red teaming language models to reduce harms: Methods, scaling behav- iors, and lessons learned

    Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behav- iors, and lessons learned. arXiv preprint arXiv:2209.07858,

    arXiv

  11. [19]

    Gener- ative language models and automated influence operations: Emerging threats and potential mitigations

    Josh A Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. Gener- ative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246, 2023. 1

    2023 arXiv

  12. [20]

    Towards a unified view of parameter-efficient transfer learning

    Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg- Kirkpatrick, and Graham Neubig. Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366, 2021. 3, 2

    2021 arXiv

  13. [21]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 5, 6

    2017

  14. [22]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 1

    2022 arXiv

  15. [23]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  16. [24]

    Parameter-efficient transfer learning for NLP

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In International conference on machine learning, pages 2790–2799. PMLR, 2019. 3, 2

    2019

  17. [25]

    9 Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 9 Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 2, 3, 6

    2021 arXiv

  18. [26]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022. 3, 2

    2022 arXiv

  19. [27]

    Patching open-vocabulary models by interpolating weights

    Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre, Shuran Song, Hannaneh Hajishirzi, Simon Kornblith, Ali Farhadi, and Ludwig Schmidt. Patching open-vocabulary models by interpolating weights. Advances in Neural Infor- mation Processing Systems, 35:29262–29277, 2022. 3, 2

    2022

  20. [28]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer, 2022. 3, 2

    2022

  21. [29]

    Exploiting programmatic behavior of llms: Dual-use through standard security attacks

    Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. In 2024 IEEE Security and Privacy Workshops (SPW), pages 132–143. IEEE, 2024. 1

    2024

  22. [30]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 1

    2022

  23. [31]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19113–19122, 2023. 3, 2

    2023

  24. [32]

    Towards safe self- distillation of internet-scale text-to-image diffusion models

    Sanghyun Kim, Seohyeon Jung, Balhae Kim, Moonseok Choi, Jinwoo Shin, and Juho Lee. Towards safe self- distillation of internet-scale text-to-image diffusion models. arXiv preprint arXiv:2307.05977, 2023. 2, 3, 4, 6, 7, 1, 5

    2023 arXiv

  25. [33]

    Safeguard text-to-image diffusion models with human feedback inversion

    Sanghyun Kim, Seohyeon Jung, Balhae Kim, Moonseok Choi, Jinwoo Shin, and Juho Lee. Safeguard text-to-image diffusion models with human feedback inversion. arXiv preprint arXiv:2407.21032, 2024. 2, 3, 1, 6, 7

    2024 arXiv

  26. [34]

    Overcoming catastrophic forgetting in neu- ral networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sc...

    2017

  27. [35]

    Ablating con- cepts in text-to-image diffusion models

    Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating con- cepts in text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 22691–22702, 2023. 1, 2

    2023

  28. [36]

    Converting ’id photos’ into ’nude photos’

    Kyunghyang Shinmun. Converting ’id photos’ into ’nude photos’... the ’ai technology’ of soda snow app.Kyunghyang Shinmun, 2024. Original article in Korean. Accessed on [Your Access Date]. 1

    2024

  29. [37]

    A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity

    Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K Kummerfeld, and Rada Mihalcea. A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity. arXiv preprint arXiv:2401.01967, 2024. 2

    2024 arXiv

  30. [38]

    Direct consistency optimization for compositional text- to-image personalization

    Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, and Jinwoo Shin. Direct consistency optimization for compositional text- to-image personalization. arXiv preprint arXiv:2402.12004,

    arXiv

  31. [39]

    The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021. 3, 2, 5

    2021 arXiv

  32. [40]

    Multi-step jailbreaking pri- vacy attacks on chatgpt

    Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. Multi-step jailbreaking pri- vacy attacks on chatgpt. arXiv preprint arXiv:2304.05197 ,

    arXiv

  33. [41]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 7

    2023

  34. [42]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  35. [43]

    Flow matching for generative mod- eling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling. arXiv preprint arXiv:2210.02747, 2022. 2, 1, 6

    2022 arXiv

  36. [44]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021. 3, 2

    arXiv

  37. [45]

    Mace: Mass concept erasure in diffu- sion models

    Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6430– 6440, 2024. 2, 3, 6, 1

    2024

  38. [46]

    Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks.arXiv preprint arXiv:2106.04489, 2021

    Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa De- hghani, and James Henderson. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks.arXiv preprint arXiv:2106.04489, 2021. 3, 2

    2021 arXiv

  39. [47]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 14297–14306, 2023. 2, 1

    2023

  40. [48]

    Efficient estimation of word representa- tions in vector space

    Tomas Mikolov. Efficient estimation of word representa- tions in vector space. arXiv preprint arXiv:1301.3781, 3781,

    arXiv

  41. [49]

    Fine-tuning diffusion models with limited data

    Taehong Moon, Moonseok Choi, Gayoung Lee, Jung-Woo Ha, and Juho Lee. Fine-tuning diffusion models with limited data. In NeurIPS 2022 Workshop on Score-Based Methods,

    2022

  42. [50]

    notAI tech. NudeNet. https://github.com/notAI- tech/nudenet, 2024. 5, 7, 3

    2024

  43. [51]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- 10 vances in neural information processing systems, ...

    2022

  44. [52]

    On aliased resizing and surprising subtleties in gan evaluation

    Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11410–11420, 2022. 8, 6

    2022

  45. [53]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–11, 2023. 2, 1

    2023

  46. [54]

    Py- Torch: An Imperative Style, High-Performance Deep Learn- ing Library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zem- ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas Kopf, Edward Yang, Zachary DeVito, Mar- tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steine...

    2019

  47. [55]

    Adapterfusion: Non- destructive task composition for transfer learning

    Jonas Pfeiffer, Aishwarya Kamath, Andreas R ¨uckl´e, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non- destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247, 2020. 3, 2

    2005 arXiv

  48. [56]

    Justin N. M. Pinkney. Pokemon blip captions. https: / / huggingface . co / datasets / lambdalabs / pokemon-blip-captions/, 2022. 4, 6, 7, 3

    2022

  49. [57]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1

    2023 arXiv

  50. [58]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 2, 1

    2022 arXiv

  51. [59]

    Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023. 1, 2, 8

    2023 arXiv

  52. [60]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  53. [61]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 2, 1

    2024

  54. [62]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Confer- ence on Machine Learning, pages 8821–8831. PMLR, 2021. 1

    2021

  55. [63]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1

    2022 arXiv

  56. [64]

    Red-teaming the stable diffusion safety filter

    Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tram`er. Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610, 2022. 2, 1

    2022 arXiv

  57. [65]

    Six-cd: Benchmarking concept removals for benign text-to-image diffusion models

    Jie Ren, Kangrui Chen, Yingqian Cui, Shenglai Zeng, Hui Liu, Yue Xing, Jiliang Tang, and Lingjuan Lyu. Six-cd: Benchmarking concept removals for benign text-to-image diffusion models. arXiv preprint arXiv:2406.14855 , 2024. 1, 2, 7, 3, 6

    2024 arXiv

  58. [66]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 6, 7

    2022

  59. [67]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  60. [68]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  61. [69]

    Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and in turn reflecting on inappropriate content? In Proceedings of the ACM Conference on Fairness, Ac- countability, and Transparency (FAccT), 2022. 7, 5

    2022

  62. [70]

    Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models

    Patrick Schramowski, Manuel Brack, Bj ¨orn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22522–22531, 2023. 2, 6, 7, 1, 3

    2023

  63. [71]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:221...

    arXiv

  64. [72]

    Ziplora: Any subject in any style by effectively merging loras

    Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svet- lana Lazebnik, Yuanzhen Li, and Varun Jampani. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pages 422–438. Springer, 2025. 3, 8, 2

    2025

  65. [73]

    Unlearning or conceal- ment? a critical analysis and evaluation metrics for unlearn- ing in diffusion models

    Aakash Sen Sharma, Niladri Sarkar, Vikram Chundawat, Ankur A Mali, and Murari Mandal. Unlearning or conceal- ment? a critical analysis and evaluation metrics for unlearn- ing in diffusion models. arXiv preprint arXiv:2409.05668 ,

    arXiv

  66. [74]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 8, 1 11

    2015

  67. [75]

    Generative modeling by esti- mating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by esti- mating gradients of the data distribution. Advances in neural information processing systems, 32, 2019. 1

    2019

  68. [76]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 1

    2011 arXiv

  69. [77]

    Identifying and eliminating csam in genera- tive ml training data and models

    David Thiel. Identifying and eliminating csam in genera- tive ml training data and models. Technical report, Techni- cal Report. Stanford University, Palo Alto, CA. https://purl. stanford . . . , 2023. 2, 1

    2023

  70. [78]

    Ring-a-bell! how reliable are concept removal meth- ods for diffusion models? arXiv preprint arXiv:2310.10012,

    Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia- You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. Ring-a-bell! how reliable are concept removal meth- ods for diffusion models? arXiv preprint arXiv:2310.10012,

    arXiv

  71. [79]

    Diffusers: State-of-the-art diffu- sion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffu- sion models. https://github.com/huggingface/ diffusers...

    2022

  72. [80]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision a...

    2024

  73. [81]

    Jail- broken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jail- broken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024. 1

    2024

  74. [82]

    Assessing the brittleness of safety alignment via pruning and low-rank modifications

    Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruning and low-rank modifications. arXiv preprint arXiv:2402.05162, 2024. 2

    2024 arXiv

  75. [83]

    Challenges in detoxifying language models

    Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty An- derson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxifying language models. arXiv preprint arXiv:2109.07445, 2021. 1

    2021 arXiv

  76. [84]

    Fundamental limitations of alignment in large language models

    Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. Fundamental limitations of alignment in large language models. arXiv preprint arXiv:2304.11082 ,

    arXiv

  77. [85]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  78. [86]

    On the proactive generation of unsafe images from text-to-image models using benign prompts

    Yixin Wu, Ning Yu, Michael Backes, Yun Shen, and Yang Zhang. On the proactive generation of unsafe images from text-to-image models using benign prompts. arXiv preprint arXiv:2310.16613, 2023. 2, 1

    2023 arXiv

  79. [87]

    Removing rlhf protections in gpt-4 via fine-tuning

    Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang. Removing rlhf protections in gpt-4 via fine-tuning. arXiv preprint arXiv:2311.05553, 2023. 2

    2023 arXiv

  80. [88]

    Forget-me-not: Learning to forget in text-to-image diffusion models

    Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1755–1764, 2024. 1, 2

    2024

  81. [89]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 8, 5

    2018

  82. [90]

    Llama-adapter: Efficient fine-tuning of language models with zero-init attention

    Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Ao- jun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023. 3, 2

    2023 arXiv

  83. [91]

    forgetting

    Ming Zhong, Yelong Shen, Shuohang Wang, Yadong Lu, Yizhu Jiao, Siru Ouyang, Donghan Yu, Jiawei Han, and Weizhu Chen. Multi-lora composition for image generation. arXiv preprint arXiv:2402.16843, 2024. 3, 2 12 Safety Alignment Backfires: Preventing the Re-emergence of Suppresse...

    2024 arXiv

  84. [92]

    country body

    proposed switching and compositing LoRAs to avoid such issues, and Dravid et al. [14] explored the possibility of encoding semantics with LoRAs and merging them in the parameter space. Inspired upon arithmetic merging meth- ods, we propose learning a separate safety module, wh...

    2000

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.