REVIEW 46 cited by
Red-Teaming the Stable Diffusion Safety Filter
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Stable Diffusion is a recent open-source image generation model comparable to proprietary models such as DALLE, Imagen, or Parti. Stable Diffusion comes with a safety filter that aims to prevent generating explicit images. Unfortunately, the filter is obfuscated and poorly documented. This makes it hard for users to prevent misuse in their applications, and to understand the filter's limitations and improve it. We first show that it is easy to generate disturbing content that bypasses the safety filter. We then reverse-engineer the filter and find that while it aims to prevent sexual content, it ignores violence, gore, and other similarly disturbing content. Based on our analysis, we argue safety measures in future model releases should strive to be fully open and properly documented to stimulate security contributions from the community.
Forward citations
Cited by 46 Pith papers
-
MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning
MOON applies spectral-nuclear-norm geometry to multi-objective gradient manipulation and uses polar-factor updates, with O(T^-1/2) deterministic and O(T^-1/4) stochastic convergence to Pareto stationarity.
-
Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency
Shuffling words and image patches in harmful prompts bypasses safety mechanisms of several commercial and open-source multimodal models, and a black-box search over shuffles raises attack success rates substantially.
-
TYPO: Instruction-Dense Visual Jailbreaks against Commercial Closed-Source Image-Generation Models
Safety alignment fails to transfer from text replies to text-in-image, and TYPO’s dual-channel combinatorial search jailbreaks four commercial image models at >90% ASR for ~$0.04.
-
PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoders
PEAK erases concepts from diffusion models by training a k-sparse autoencoder to localize target features, then fine-tuning the model to suppress those features while preserving all others.
-
Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models
MIND learns a 'defense profile' of a T2I model from fine-grained feedback, then uses it to guide an evolutionary search, achieving 95.62% ASR across six defenses and 91.58% on Wan-2.5.
-
Evaluating Intellectual Property Guardrails of Generative Image Models: A Technical Report
All 14 tested text-to-image models readily generate recognizable IP; private models refuse at highly uneven rates, with commercial logos refused least and generated most.
-
UnHype: CLIP-Guided Hypernetworks for Dynamic LoRA Unlearning
UnHype generates concept-specific LoRA unlearning weights on the fly from CLIP text embeddings by training a hypernetwork to follow the gradient of an unlearning loss, enabling single- and multi-concept erasure in dif...
-
Video Deepfake Abuse: How Company Choices Predictably Shape Misuse Patterns
A few open-weight video models and distribution platforms dominate the creation and spread of NSFW AI video, making developer and platform choices the main intervention points for reducing non-consensual deepfake abuse.
-
A Unified Framework for Diffusion Model Unlearning with f-Divergence
Diffusion model unlearning is generalized from KL/MSE to any f-divergence, with closed-form Hellinger and chi-square losses and a variational min-max form.
-
LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearning
LoReUn, a plug-in loss-based reweighting strategy, improves approximate machine unlearning by focusing updates on hard-to-forget low-loss data points.
-
Customize Multi-modal RAI Guardrails with Precedent-based predictions
Conditioning a multimodal guardrail on retrieved 'precedent' reasoning traces, rather than static policy definitions, improves few-shot and novel-policy content-moderation F1 scores on UnsafeBench.
-
PLA: Prompt Learning Attack against Text-to-Image Generative Models
PLA trains adversarial prompts with a zero-order gradient method and multimodal CLIP losses to bypass safety filters and post-hoc checkers in black-box text-to-image models, outperforming earlier word-substitution attacks.
-
"I Cannot Write This Because It Violates Our Content Policy": Understanding Content Moderation Policies and User Experiences in Generative AI Products
GAI tools' content moderation policies are comprehensive in scope but thin on user reporting and appeals, and Reddit users report frequent frustration with opaque moderation decisions.
-
GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models
A reinforcement-learning red-team LLM generates stealthy prompts that bypass text-to-image safety filters and produce toxic images, with reported transfer success against commercial APIs.
-
SAGE: Exploring the Boundaries of Unsafe Concept Domain with Semantic-Augment Erasing
SAGE erases concepts from diffusion models by optimizing attack prompts against the model's own text encoder and then fine-tuning that encoder with a global-local retention loss.
-
CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems
CuRe scores text-to-image systems by how much their output changes as prompts add cultural details, and reports better agreement with human ratings than existing proxies.
-
Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling
RPG-RT iteratively fine-tunes an LLM with rule-based preferences from a decoupled CLIP scoring model, letting it rewrite prompts that bypass unknown safety defenses in black-box text-to-image systems.
-
TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysis
TokenProber bypasses five NSFW safety checkers on three text-to-image models by separately preserving dirty words and reducing the influence of non-dirty discrepant words.
-
Erased but Not Forgotten: How Backdoors Compromise Concept Erasure
Backdoor triggers injected into text-to-image diffusion models survive state-of-the-art concept erasure, restoring erased identities and explicit content with high success rates.
-
Predictive Red Teaming: Breaking Policies Without Breaking Robots
A generative image editing plus anomaly detection pipeline predicts a visuomotor policy's success-rate degradation across off-nominal environmental factors, with an average prediction error below 0.19 in hardware trials.
-
SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders
SAeUron removes concepts from text-to-image diffusion models by ablating concept-specific sparse autoencoder features during inference, achieving state-of-the-art unlearning on UnlearnCanvas and I2P without weight updates.
-
T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation
T2ISafety is a large annotated benchmark plus a fine-tuned MLLM evaluator (ImageGuard) for measuring toxicity, privacy, and fairness in text-to-image models.
-
ACE: Anti-Editing Concept Erasure in Text-to-Image Models
ACE trains a LoRA adapter on both conditional and unconditional noise predictions so that erased concepts are suppressed during both generation and text-guided editing.
-
SafeCFG: Controlling Harmful Features with Dynamic Safe Guidance for Safe Generation
SafeCFG adapts classifier-free guidance with a learned feature controller so that clean prompts generate normally while harmful prompts are pushed away from unsafe content.
-
Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models
A bilevel training procedure that simultaneously restores a pruned diffusion model's quality and suppresses targeted concepts beats sequential fine-tuning followed by unlearning.
-
Bridging the Data Provenance Gap Across Text, Speech and Video
A manual audit of nearly 4,000 text, speech, and video datasets finds AI training data increasingly comes from web and social media sources, carries hidden non-commercial restrictions, and remains Western-centric with...
-
Finding a Wolf in Sheep's Clothing: Combating Adversarial Text-To-Image Prompts with Text Summarization
Summarizing LLM-obfuscated text-to-image prompts before classification improves content-moderation F1 scores on the new ATTIP dataset.
-
Moderating the Generalization of Score-based Generative Model
MSGM is a score-adjustment unlearning method for score-based generative models that suppresses targeted content generation without full retraining.
-
Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters
AdaVD removes target concepts from diffusion models by soft-projecting value vectors away from the target token direction, with a sigmoid threshold that preserves unrelated prompts.
-
Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models
Typography inserted into input images can manipulate CLIP-guided image generation models to produce harmful or biased content, and existing text-focused defenses do not catch it.
-
Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization
Prompt-Noise Optimization jointly tunes the prompt embedding and diffusion noise at inference time to suppress unsafe images while keeping outputs close to the prompt.
-
Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models
Fine-tuning text-to-image diffusion models on benign data can reactivate suppressed unsafe concepts, and training the task adapter separately from a frozen safety LoRA prevents this.
-
ContrastiveCFG: Guiding Diffusion Sampling by Contrasting Positive and Negative Concepts
A new guidance reweighting, derived from a contrastive loss, makes negative prompting in diffusion models remove unwanted concepts with less quality loss than standard negated CFG.
-
DECAF: De-Clustering for Adaptive Representational Unlearning
DECAF is a forget-only unlearning method that adds input noise, suppresses the forget-class probability, and diversifies outputs, achieving 0.10% forget accuracy and 79.4% retain accuracy on CIFAR-10/ResNet-18 while d...
-
Introspective Attention Modulation for Safe Text-to-Image Generation
Inference-time attention modulation suppresses unsafe content in diffusion-transformer T2I models without retraining and beats concept-erasure baselines in the paper's benchmarks.
-
GIFT: Gradient-aware Immunization of diffusion models against malicious Fine-Tuning with safe concepts retention
GIFT immunizes diffusion models against malicious fine-tuning by combining loss maximization and representation noising, preserving safe concept generation.
-
Automating Evaluation of Diffusion Model Unlearning with (Vision-) Language Model World Knowledge
A new evaluation tool uses (vision-)language model world knowledge to rank nearby concepts and craft adversarial prompts, showing that diffusion unlearning is incomplete and that semantic similarity correlates with co...
-
The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI
Security's hard-won transparency practices, from Kerckhoffs' principle to vulnerability disclosure, form three transferable themes for AI transparency efforts.
-
DuMo: Dual Encoder Modulation Network for Precise Concept Erasure
DuMo erases target concepts from text-to-image models by adding a frozen-backbone skip-connection eraser with learned timestep and layer modulation, reporting the best trade-off on three concept erasure benchmarks.
-
Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization
A method to precisely localize and replace concepts in generated images by combining a few-shot attention-based concept localizer with masked dual-prompt cross-attention during denoising.
-
Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks
A step-by-step multimodal 'chain of attack' improves the transferability of targeted adversarial images against open vision-language models, with a new LLM-judged success metric.
-
MapRoute++: Surrogate-Guided Semantic Routing for Visual Concept Unlearning
A concept-erasure method built on MapRoute raises the official ERR score from 0.600 to 0.721 on the Genµ2.0 benchmark, but it does not beat MapRoute on object, animal, or action categories.
-
A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination
A new jailbreak framework, HACA, combines atomic text and image attack strategies selected by a cross-modal planner and generates attacks with LLMs and text-to-image models, reaching 95.48% average attack success acro...
-
Zero-Residual Concept Erasure via Progressive Alignment in Text-to-Image Model
A closed-form concept erasure method that enforces zero alignment residual in the optimization objective and applies updates progressively across layers to better preserve generation quality.
-
Memory Enhanced Fractional-Order Dung Beetle Optimization for Photovoltaic Parameter Identification
The claimed MFO-DBO algorithm and its CEC2017/PV results are absent from the manuscript, which instead contains an unrelated prompt-stealing attack paper.
-
Text-to-Image Synthesis: A Decade Survey
A decade-spanning survey categorizes over 440 text-to-image papers by architecture, research problem, dataset, and evaluation metric.
Discussion (0). Continue with ORCID to comment.