REVIEW 9 cited by
Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Data poisoning attacks manipulate training data to introduce unexpected behaviors into machine learning models at training time. For text-to-image generative models with massive training datasets, current understanding of poisoning attacks suggests that a successful attack would require injecting millions of poison samples into their training pipeline. In this paper, we show that poisoning attacks can be successful on generative models. We observe that training data per concept can be quite limited in these models, making them vulnerable to prompt-specific poisoning attacks, which target a model's ability to respond to individual prompts. We introduce Nightshade, an optimized prompt-specific poisoning attack where poison samples look visually identical to benign images with matching text prompts. Nightshade poison samples are also optimized for potency and can corrupt an Stable Diffusion SDXL prompt in <100 poison samples. Nightshade poison effects "bleed through" to related concepts, and multiple attacks can composed together in a single prompt. Surprisingly, we show that a moderate number of Nightshade attacks can destabilize general features in a text-to-image generative model, effectively disabling its ability to generate meaningful images. Finally, we propose the use of Nightshade and similar tools as a last defense for content creators against web scrapers that ignore opt-out/do-not-crawl directives, and discuss possible implications for model trainers and content creators.
Forward citations
Cited by 9 Pith papers
-
Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing
FaceLock perturbs portraits so diffusion-based edits destroy face-recognition similarity, and it evaluates success with the same face model that it attacks.
-
Imago Obscura: An Image Privacy AI Co-pilot to Enable Identification and Mitigation of Risks
An AI copilot that identifies privacy risks in photos and applies obfuscation edits increased users' self-reported awareness and ability to manage those risks in a 15-person lab study.
-
MixBridge: Heterogeneous Image-to-Image Backdoor Attack through Mixture of Schr\"odinger Bridges
MixBridge injects multiple backdoor triggers into image-to-image Schrödinger bridge models by training on poisoned pairs and merging task-specific experts, achieving high attack success and stealthy weights.
-
MagicNaming: Consistent Identity Generation by Finding a "Name Space" in T2I Diffusion Models
An image encoder maps any face to a 'name embedding' that, when prepended to a text prompt, makes an SDXL model generate consistent identities for arbitrary people without fine-tuning.
-
Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation
World-model-based embodied AI creates a predictive security boundary where attacks on data, sensors, imagination, ranking, and feedback can turn into unsafe physical action and false safety certificates.
-
MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes
MaRVL-QA is a new benchmark that tests multimodal AI systems on mathematical reasoning over surface plots, with two tasks: counting topological features and recognizing geometric transformations.
-
Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning
Sparse autoencoders and ITDA produce comparably interpretable and steerable features in FLUX.1, outperforming MLP neurons on an automated visual interpretability metric.
-
Unveiling Unicode's Unseen Underpinnings in Undermining Authorship Attribution
The paper proposes integrating Unicode steganography into adversarial stylometry to undermine authorship attribution.
-
The Generative AI Ethics Playbook
A structured playbook that collects existing guidance, checklists, and case studies to help generative AI practitioners identify and mitigate ethical harms across six lifecycle stages.
Discussion (0). Continue with ORCID to comment.