Use of model-generated content in training causes irreversible loss of distribution tails, termed model collapse, in VAEs, GMMs, and LLMs.
Poisoning and backdooring contrastive learning.arXiv preprint arXiv:2106.09667
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01% of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of 0.0001% of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
The paper presents a roadmap that identifies four unsolved problems in ML safety: robustness against hazards, monitoring for hazards, alignment of model goals with human intent, and systemic safety.
A system-first taxonomy and literature synthesis of multimodal unlearning across vision, language, video, and audio, with datasets, benchmarks, metrics, applications, and open challenges.
AI systems lack verifiability, versioning, observability, and traceability in their software supply chains, shown by dependency analysis of 48 projects yielding 4,664 direct and 11,508 transitive dependencies totaling 392M lines of code.
citing papers explorer
-
The Curse of Recursion: Training on Generated Data Makes Models Forget
Use of model-generated content in training causes irreversible loss of distribution tails, termed model collapse, in VAEs, GMMs, and LLMs.
-
Unsolved Problems in ML Safety
The paper presents a roadmap that identifies four unsolved problems in ML safety: robustness against hazards, monitoring for hazards, alignment of model goals with human intent, and systemic safety.
-
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
A system-first taxonomy and literature synthesis of multimodal unlearning across vision, language, video, and audio, with datasets, benchmarks, metrics, applications, and open challenges.
-
The Grand Software Supply Chain of AI Systems
AI systems lack verifiability, versioning, observability, and traceability in their software supply chains, shown by dependency analysis of 48 projects yielding 4,664 direct and 11,508 transitive dependencies totaling 392M lines of code.
- Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models