REVIEW 8 cited by
Direct Unlearning Optimization for Robust and Safe Text-to-Image Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advancements in text-to-image (T2I) models have unlocked a wide range of applications but also present significant risks, particularly in their potential to generate unsafe content. To mitigate this issue, researchers have developed unlearning techniques to remove the model's ability to generate potentially harmful content. However, these methods are easily bypassed by adversarial attacks, making them unreliable for ensuring the safety of generated images. In this paper, we propose Direct Unlearning Optimization (DUO), a novel framework for removing Not Safe For Work (NSFW) content from T2I models while preserving their performance on unrelated topics. DUO employs a preference optimization approach using curated paired image data, ensuring that the model learns to remove unsafe visual concepts while retaining unrelated features. Furthermore, we introduce an output-preserving regularization term to maintain the model's generative capabilities on safe content. Extensive experiments demonstrate that DUO can robustly defend against various state-of-the-art red teaming methods without significant performance degradation on unrelated topics, as measured by FID and CLIP scores. Our work contributes to the development of safer and more reliable T2I models, paving the way for their responsible deployment in both closed-source and open-source scenarios.
Forward citations
Cited by 8 Pith papers
-
You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models
A per-prompt cross-attention spike detector plus repulsive-attractive guidance (GUARD) substantially reduces verbatim and template memorization in Stable Diffusion at inference time.
-
ZIUM: Zero-Shot Intent-Aware Adversarial Attack on Unlearned Models
ZIUM attacks unlearned diffusion models by optimizing an image-captioning module that turns a target image into a text embedding, then reuses that module zero-shot on unseen images of the same unlearned concept.
-
Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts
ANT erases unwanted concepts from text-to-image diffusion models by reversing the guidance direction only during mid-to-late denoising, preserving early structural quality.
-
ACE: Anti-Editing Concept Erasure in Text-to-Image Models
ACE trains a LoRA adapter on both conditional and unconditional noise predictions so that erased concepts are suppressed during both generation and text-guided editing.
-
Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization
Prompt-Noise Optimization jointly tunes the prompt embedding and diffusion noise at inference time to suppress unsafe images while keeping outputs close to the prompt.
-
Opt-In Art: Learning Art Styles Only from Few Examples
A diffusion model pretrained exclusively on photographs can learn a painter's style from just a handful of examples, matching the style fidelity of models pretrained on large art-containing datasets.
-
SecureT2I: No More Unauthorized Manipulation on AI Generated Images from Prompts
A diffusion editing model is fine-tuned with a blur target for forbidden images and the original output for permitted images, claiming selective suppression of unauthorized edits.
-
Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models
A position paper urging a shift from data-tracing to knowledge-tracing machine unlearning for foundation models, supported by a CLIP case study that shows current methods struggle to generalize.
Discussion (0). Continue with ORCID to comment.