{"id":"934c551e-7355-44e1-833c-837c80a1ad92","arxiv_id":"2412.00357","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Fine-tuning text-to-image diffusion models on benign data can reactivate suppressed unsafe concepts, and training the task adapter separately from a frozen safety LoRA prevents this.","lead":"This paper shows that fine-tuning a safety-aligned image generation model on ordinary, non-harmful data can make previously suppressed nude or signed images reappear, and proposes a simple fix: keep the safety adjustment as a separate LoRA module and detach it while fine-tuning. The result matters because fine-tuning APIs and everyday users can unknowingly undo built-in safety filters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Negative-transfer mechanism underpinning Modular LoRA is supported only by single-prompt weight-negation visuals; a confound could explain the safety gains, so the 'backfire' claim is not yet established.","rationale":"The reader's weakest assumption correctly flagged the unverified additivity and independence of the safety LoRA. My concern is adjacent but more specific: the paper's mechanistic story is that ΔWsafe's presence during fine-tuning causes ΔWft to encode the suppressed concept, and the only direct evidence is the visual anecdote in Figs. 8–9. This matters because the title and abstract claim that fine-tuning 'actively reactivates' suppressed concepts and that the safety training component itself 'drives the relearning.' If the negative-transfer mechanism is wrong, the modular approach might still work empirically (the numbers in Tables 2–4 show large, consistent gaps), but the central explanatory claim would be overstated. I credit the paper for broad empirical coverage across multiple datasets, safety methods (ESD, SDD), and evaluation prompts, and for including downstream task quality metrics (Table 6). The main quantitative gaps in Tables 2–4 are large enough that the practical phenomenon is likely real. The concern is about the mechanism, not the existence of the phenomenon, so the reader's CONDITIONAL verdict remains appropriate. The proposed test directly isolates the mechanism: it compares standard versus modular training, and adds a random-LoRA control to show whether the effect is specific to the semantic content of ΔWsafe or to any frozen LoRA. This would settle whether Modular LoRA works because it prevents negative transfer, or for a simpler reason (e.g., fine-tuning an unsafe base on safe data is itself a safety-inducing operation). Until such a control is reported, the 'safety alignment backfires' framing should be treated as a hypothesis rather than an established causal explanation.","tokens_in":24592,"tokens_out":8422,"duration_ms":84290,"concrete_test":"Run ESD+LoRA safety alignment on SD v1.4, then fine-tune on Pokemon under three conditions: (i) standard: train ΔWft on W0+ΔWsafe; (ii) modular: train ΔWft* on W0; (iii) control: train ΔWft_rand on W0+ΔWrandom with ΔWrandom a random LoRA norm-matched to ΔWsafe. After training, evaluate N=100 images per prompt over Six-CD/I2P nuanced sets for: (a) the full merged models (safety + FT), and (b) the FT LoRA alone applied to W0 (W0+ΔWft). Report NudeNet %exposed with 95% CIs. If (i)(b) >> (ii)(b) but (iii)(b) ≈ (ii)(b), the negative-transfer mechanism is confirmed and is specific to safety content; if (iii)(b) ≈ (i)(b), the effect is due to any frozen LoRA, not to ΔWsafe's semantics; if (i)(b) ≈ (ii)(b), the mechanism is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central explanatory claim (Sec. 3.4) is that when the safety LoRA ΔWsafe is attached during fine-tuning, the fine-tuning LoRA ΔWft learns the inverse concept (+nudity) via 'negative transfer' from ΔWsafe, and this is what causes jailbreaking. The only evidence adduced is Figs. 8–9, which show a handful of images for one prompt ('a sexy cute girl with a cat') where negating ΔWsafe or ΔWft visually changes nudity. No quantitative measurement is reported of what ΔWft encodes, no confidence intervals, and no control for the mere presence of a frozen LoRA. The entire Modular LoRA method—and the title's 'backfires' thesis—depends on this mechanism being real. An alternative explanation consistent with the data: fine-tuning W0 (an unsafe base) on benign anime data directly suppresses the base's unsafe tendencies, while fine-tuning W0+ΔWsafe (already safe) drifts because the benign data is less 'protective'; the resulting ΔWft difference has nothing to do with ΔWsafe's semantic content. If the latter is true, the safety gains of Modular LoRA may survive but the paper's causal story and the claim that safety alignment 'actively reactivates' suppressed concepts would be unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies fine-tuning jailbreaking in text-to-image diffusion models: fine-tuning a safety-aligned model (ESD, SDD, or a commercially aligned model such as FLUX.1) on benign anime-style datasets can cause previously suppressed unsafe content, such as nudity or artist signatures, to reappear. The authors attribute this to a negative-transfer mechanism in which the fine-tuning LoRA ΔW_ft learns the inverse of the safety LoRA ΔW_safe, and they propose Modular LoRA: train a safety LoRA separately, detach it during downstream fine-tuning, and re-attach it at inference time (Eq. 3). The main experiments compare full fine-tuning, standard LoRA fine-tuning, and Modular LoRA on Pokemon, Naruto, and Danbooru datasets, reporting that Modular LoRA greatly reduces the percentage of images flagged as unsafe while maintaining downstream generation quality.","tokens_in":24847,"tokens_out":5532,"duration_ms":53978,"significance":"If the empirical result holds, the paper documents a practically important vulnerability: fine-tuning safety-aligned text-to-image models on benign data can degrade safety, and a simple modular training procedure can substantially mitigate this degradation. The main quantitative gaps are large and consistent, e.g., in Table 2 the average exposed-body-part rate for SDD is 1.8% with Modular LoRA versus 29.1% with standard LoRA fine-tuning, and the effect replicates across three datasets, two safety methods, and multiple prompt types. The paper also provides useful complementary results, including FLUX.1 jailbreaking examples, a curated Danbooru subset, and detailed experimental appendices. However, the paper's central explanatory mechanism—that ΔW_ft actively re-learns +nudity via negative transfer from ΔW_safe—is supported only by qualitative single-prompt weight-negation visualizations, and several quantitative claims are stated without uncertainty estimates. The empirical contribution is therefore stronger than the mechanistic story, and the revision should either substantiate the mechanism or carefully delimit the claims.","major_comments":[{"comment":"The claim that the fine-tuning LoRA ΔW_ft acquires the inverse of ΔW_safe through negative transfer is the stated cause of jailbreaking and the motivation for Modular LoRA in Sec. 4, yet the only evidence is two qualitative, single-prompt comparisons for \"a sexy cute girl with a cat\" (Figs. 8 and 9). No quantitative measurement of what ΔW_ft encodes is reported, there are no confidence intervals, and there is no control for the mere presence of a frozen LoRA during fine-tuning. An alternative explanation, namely that fine-tuning W0 directly suppresses the base model's unsafe tendencies while fine-tuning W0+ΔW_safe drifts because the benign data does not reinforce the already-safe behavior, would also explain the data without requiring negative transfer. Please add quantitative support, for example by evaluating W0+ΔW_ft versus W0+ΔW*_ft on a multi-prompt NSFW benchmark, measuring the projection or cosine similarity between ΔW_ft and ΔW_safe, and running a control experiment with an unrelated frozen LoRA attached during fine-tuning. If such measurements are not available, the mechanism claim should be softened to a hypothesis.","section":"Sec. 3.4 and Figs. 8-9"},{"comment":"The text says Modular LoRA \"prevents the re-learning of harmful content\" (Sec. 4) and that it \"successfully mitigates jailbreaking\" (Sec. 5.2), but the before/after numbers show residual safety degradation on several datasets. In Table 2, ESD Modular goes from 4.5% before fine-tuning to 7.3% on Naruto and 8.5% on Danbooru, with an average of 6.1% that is above the 4.5% before value. In Table 3, ESD Modular increases from 41.4% before to 48.7% on average after fine-tuning. The claims should be calibrated to these increases, and the discussion should address why Modular LoRA reduces but does not eliminate the re-learning effect, especially on datasets with more human and nuanced content.","section":"Tables 2 and 3"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any of the main safety tables. This matters because Appendix C.2 and C.3 explicitly acknowledge that the NudeNet and Q16 classifiers have high false-positive rates and that small percentage differences are not considered significant. The headline gaps in Tables 2 and 3 are large and probably robust, but the smaller comparisons, such as ESD Modular 6.1% versus its 4.5% before value, need uncertainty quantification. Please report means and standard errors over multiple fine-tuning seeds or bootstrap over generated images, and state whether the same random seeds and prompt sets were used across compared methods.","section":"Tables 2-4 and Appendix C"},{"comment":"Equation (2) writes the standard fine-tuning pipeline as W' = W0 + ΔW_safe + ΔW_ft, which presupposes that the initial safety alignment is itself represented as a LoRA module. However, Table 1 includes Full -> Full and Full -> LoRA configurations where the safety alignment is a full-model update and the additive decomposition in Eq. (2) does not literally hold. This makes it unclear whether the proposed negative-transfer analysis applies to safety alignments that are not LoRA-based, and it complicates the comparison between the Full and LoRA rows in Table 1. Please clarify the notation and state explicitly that the mechanism analysis and Modular LoRA apply to LoRA-based safety alignment, with full-model safety alignment handled only empirically.","section":"Eq. (2) and Table 1"}],"minor_comments":[{"comment":"The phrase \"novel but immediate solution\" is awkward; consider \"novel and immediately applicable solution\" or similar.","section":"Abstract"},{"comment":"The table header says \"average CLIP similarity\" and the caption says lower is better, but CLIP score is conventionally higher-is-better. Please add an explicit note in the table or caption that a lower value indicates greater style-removal effectiveness, not lower text-image alignment in general.","section":"Table 4"},{"comment":"The statement that fine-tuning \"actively reactivates latent concepts within the model's weights\" is stronger than what Fig. 1 alone demonstrates; the increase in signature generation could also arise from a style shift that makes the model more likely to render text-like artifacts. Please present this as an interpretation or provide a control that directly tests the reactivation claim.","section":"Sec. 3.1 and Fig. 1"},{"comment":"The hyperparameter narrative would be easier to follow as a table: ESD uses a LoRA learning rate of 1e-4 for NSFW and 1e-5 for artist removal, while SDD uses 1e-5 in both cases, and the fine-tuning step counts also differ. A compact table would reduce the risk of readers misapplying these settings.","section":"Appendix D.1"},{"comment":"The Dreambooth example in Fig. 13 uses an explicit evaluation prompt (\"gorgeous, naked, without clothes\") but the caption describes it only as Dreambooth training on five sneakers images. Please state in the caption that the evaluation prompt is deliberately explicit, since this is important for interpreting the reported harmful-image percentages.","section":"Fig. 13 and Appendix E.1"},{"comment":"For DINO feature consistency, the direction of improvement is not stated. Please specify whether higher DINO values are better, as is implicitly suggested by the comparison.","section":"Table 6"},{"comment":"The caption uses shorthand like \"Full -> Full in Tab. 1\" without defining the arrow notation in the caption itself; a one-sentence definition would help readers who are not looking at Table 1.","section":"Fig. 5 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical finding is likely correct and valuable, and the proposed Modular LoRA is simple enough to be a useful baseline for future work. My main concern is that the negative-transfer mechanism, which the title and Sec. 3.4 elevate to a central claim, is not quantitatively supported. I would recommend requiring the additional experiments described in major comment 1 before publication, or a substantial rewriting that downgrades the mechanism to a conjecture. I also suggest the editors ask the authors to make their evaluation code and generated-image seeds available, since the safety numbers are highly sensitive to classifier thresholds and prompt sampling."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the main empirical claim is solid: fine-tuning safety-aligned text-to-image models on benign anime data does bring suppressed nudity back, and the gap is large and consistent across ESD and SDD, three datasets, and several prompt sets (e.g., SDD Modular 1.8% vs LoRA 29.1% on exposed body parts). The paper also ships a genuinely useful countermeasure: train the fine-tuning LoRA on the unsafe base model, then re-attach the safety LoRA at inference. That is cheap, simple, and works in their tables. This is a real contribution to the safety-alignment subfield, and it transfers a known LLM failure mode to diffusion models in a way that is not just a rehash.\n\nWhat is soft, in proportion: there are no error bars or significance tests anywhere, so we are trusting that the large gaps are not driven by a few outlier seeds. The authors acknowledge their NudeNet and Q16 classifiers have high false-positive rates, which is honest but means the absolute percentages are not precise. The CLIP-score metric for artist-style reappearance is weak, and the paper does not release code or the curated Danbooru subset, only a promise. None of this kills the main result, but it does cap how much you can trust the fine granularity.\n\nThe mechanism story is the weakest section. Section 3.4 claims the safety LoRA transfers a negative concept into the fine-tuning LoRA, causing the jailbreak, and the only direct evidence is two single-prompt weight-negation visuals. The stress-test confound—that fine-tuning the already-safe model drifts because the benign data is less protective, while fine-tuning the unsafe base suppresses unsafe tendencies—is not controlled for. That said, the method itself does not depend on the mechanism being correct: the Dreambooth experiment (Fig. 13) empirically shows that the fine-tuning LoRA trained without the safety LoRA is less harmful when re-attached, and that difference is what Modular LoRA exploits. So the core result is probably robust even if the causal story is overreaching.\n\nFor a referee: this deserves a serious look, not a desk reject. The demands should be reasonable: release the code and dataset, add error bars or at least multiple seeds, and run a proper control for the negative-transfer claim—e.g., a random frozen LoRA instead of the safety LoRA, or a direct measurement of what the fine-tuning LoRA encodes. With those, the paper would be significantly stronger. I would bring it to a reading group and would likely cite it if I worked on diffusion safety.","headline":"The paper demonstrates a real, practically important failure mode and a simple, likely effective fix, but the negative-transfer mechanism it leans on is supported more by intuition than by measurement.","tokens_in":25382,"tokens_out":2431,"would_cite":true,"duration_ms":25555,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Benign fine-tuning can undo the safety alignment of text-to-image diffusion models, and Modular LoRA prevents this by keeping the safety module detached during fine-tuning and reattaching it only at inference.","keywords":["safety alignment","fine-tuning jailbreaking","text-to-image diffusion models","concept removal","LoRA","Modular LoRA","NSFW content suppression","model merging"],"falsifier":"Fine-tune the same safety-aligned model on the Danbooru dataset twice: once with the safety LoRA attached as in standard fine-tuning, and once with it detached as in Modular LoRA, then compare the fraction of images flagged by an automated NSFW detector over training steps. If the detached run shows the same early spike in unsafe images as the standard run, or if reattaching the safety module at inference no longer suppresses nudity, the proposed mechanism is not what stops re-emergence.","tokens_in":24355,"feed_emoji":"🛡️","tokens_out":10049,"duration_ms":83298,"temperature":0.7,"pith_summary":"Fine-tuning a safety-aligned text-to-image diffusion model on an innocuous dataset can undo its safety alignment: nudity, artist styles, or signatures that the model was trained to suppress re-emerge in generated images, and larger or more conceptually similar datasets make the effect stronger. The paper argues this is not merely forgetting or damage to the filter; fine-tuning actively re-learns the suppressed concept from the model's own weights, and in some setups the effect exceeds the unaligned base model. The proposed remedy, Modular LoRA, isolates the safety behavior in a separate low-rank adapter, removes that adapter while the task-specific fine-tuning runs, and reattaches it at inference. The paper reports that this keeps exposed-body-part rates close to the pre-fine-tuning aligned levels across three anime-style datasets while preserving downstream task quality.","feed_headline":"Fine-tuning revives censored concepts in image models","feed_subtitle":"Clean-data fine-tuning revives removed concepts; training the task LoRA without the safety LoRA stops the leak.","key_machinery":"The load-bearing object is the additive weight decomposition $W = W_0 + \\Delta W_{\\mathrm{safe}} + \\Delta W_{\\mathrm{ft}}$, where LoRA (low-rank adaptation) represents each weight update as a product of two small matrices. $\\Delta W_{\\mathrm{safe}}$ is the safety adapter trained by a concept-removal method such as ESD or SDD; $\\Delta W_{\\mathrm{ft}}$ is the adapter learned during downstream fine-tuning. The paper shows that with standard fine-tuning, $\\Delta W_{\\mathrm{ft}}$ re-encodes the inverse of $\\Delta W_{\\mathrm{safe}}$, and Modular LoRA trains the fine-tuning adapter as $\\Delta W^*_{\\mathrm{ft}}$ with $\\Delta W_{\\mathrm{safe}}$ detached, then merges at inference as $W^* = W_0 + \\Delta W_{\\mathrm{safe}} + \\Delta W^*_{\\mathrm{ft}}$.","core_discovery":"The central claim is that fine-tuning a safety-aligned text-to-image diffusion model on benign data reactivates the exact concepts the alignment had suppressed, rather than simply damaging or forgetting the safety behavior. The evidence includes FLUX.1 producing artist signatures and explicit content after a few thousand fine-tuning steps on a clean Pokémon dataset, and ESD- and SDD-aligned Stable Diffusion models showing a sharp rise in unsafe images in the first hundreds of fine-tuning steps, before the benign target style has been learned. Weight-negation experiments support the proposed mechanism: removing the safety adapter makes a safe model generate nudity from an innocuous prompt, while removing the fine-tuning adapter suppresses nudity even under explicitly NSFW prompts. The solution is Modular LoRA, which trains the fine-tuning adapter with the safety adapter detached and merges them only at inference as $W^* = W_0 + \\Delta W_{\\mathrm{safe}} + \\Delta W^*_{\\mathrm{ft}}$.","pith_inferences":["The paper leaves implicit that the same ‘detach the invariant module’ principle applies to any attribute a deployer wants to keep fixed during fine-tuning, such as a watermark, a face identity, or a licensed style, not only to safety concepts.","A testable extension is to measure the principal subspaces of $\\Delta W_{\\mathrm{safe}}$ and $\\Delta W^*_{\\mathrm{ft}}$; if the modules occupy nearly orthogonal low-rank subspaces, then the method’s success is a subspace-separation phenomenon and could be predicted from the spectra of the two adapters.","The finding that suppressed concepts can be amplified by fine-tuning suggests that alignment should be treated as a persistent constraint in the fine-tuning objective, not as a one-time preprocessing stage."],"forward_implications":["Fine-tuning APIs that let users adapt a safety-aligned image model on their own data can silently reintroduce NSFW output even when the uploaded data is clean; Modular LoRA gives such services a simple way to keep the safety component intact.","The early-stage spike in unsafe images means safety evaluations should look at early checkpoints of fine-tuning, not only the final model.","Concept-removal methods implemented as LoRA modules become reusable components: the same safety module can stay attached during inference across many downstream fine-tunes.","The method preserves downstream task quality: DreamBooth image and text alignment scores with Modular LoRA are comparable to full fine-tuning."],"supporting_citations":[{"why":"Defines LoRA, the low-rank adaptation decomposition that both the safety and fine-tuning modules use.","marker":"[25]"},{"why":"ESD, the concept-removal method whose safety alignment is the main test bed, reimplemented with LoRA in the experiments.","marker":"[16]"},{"why":"SDD, the second concept-removal baseline, whose more aggressive removal shows stronger fine-tuning jailbreaking.","marker":"[32]"},{"why":"Establishes that benign fine-tuning compromises safety alignment in language models, the phenomenon this paper extends to image models.","marker":"[59]"},{"why":"Model arithmetic and weight negation, used to show that the safety LoRA encodes the suppressed concepts and the fine-tuning LoRA re-encodes them.","marker":"[12]"},{"why":"DreamBooth, the downstream personalization protocol used to test whether Modular LoRA preserves task performance.","marker":"[67]"},{"why":"Six-CD benchmark prompts used to measure NSFW and harmful-concept re-emergence under harsh, malicious, and clean prompts.","marker":"[65]"},{"why":"I2P prompts and the ‘country body’ prompts used for nuanced-prompt evaluation and early-stage jailbreaking observations.","marker":"[70]"}],"fun_headline_variants":["Safety alignment backfires in fine-tuned image models","Benign fine-tuning leaks censored concepts in image AI","Modular LoRA stops fine-tuning from reviving suppressed content","Fine-tuning jailbreaks safety filters in text-to-image models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole fix rests on the assumption that a safety alignment lives in a detachable additive weight module, and that training the task module without that safety module attached prevents the model from re-learning the suppressed concept.","fun_headline_variants_meta":{"raw":{"variants":["Safety alignment backfires in fine-tuned image models","Benign fine-tuning leaks censored concepts in image AI","Modular LoRA stops fine-tuning from reviving suppressed content","Fine-tuning jailbreaks safety filters in text-to-image models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000414,"raw_usage":{"total_tokens":2140,"prompt_tokens":946,"completion_tokens":1194,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":1127}},"tokens_in":562,"tokens_out":1194,"duration_ms":10117,"temperature":1.0,"reasoning_tokens":1127,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:27:24.829554+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune the same safety-aligned model on the Danbooru dataset twice: once with the safety LoRA attached as in standard fine-tuning, and once with it detached as in Modular LoRA, then compare the fraction of images flagged by an automated NSFW detector over training steps. If the detached run shows the same early spike in unsafe images as the standard run, or if reattaching the safety module at inference no longer suppresses nudity, the proposed mechanism is not what stops re-emergence.","supporting_citations":[{"cited_title":"Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation","cited_arxiv_id":null,"evidence_quote":"DreamBooth, the downstream personalization protocol used to test whether Modular LoRA preserves task performance."},{"cited_title":"Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models","cited_arxiv_id":"2406.14855","evidence_quote":"Six-CD benchmark prompts used to measure NSFW and harmful-concept re-emergence under harsh, malicious, and clean prompts."},{"cited_title":"Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models","cited_arxiv_id":null,"evidence_quote":"I2P prompts and the ‘country body’ prompts used for nuanced-prompt evaluation and early-stage jailbreaking observations."}],"review_version":1}