Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that adversarially mislabeled image–caption pairs, produced by perturbing benign images so that VLM captioners output attacker-chosen captions, act as highly potent dirty-label poison samples that redirect text-to-image…

desk verdict A real new attack surface and a thorough feasibility study, but the headline PSR metric overstates prompt-specific substitution; still deserves a serious referee. read the letter →

arxiv 2506.21874 v1 pith:4JHYGR5B submitted 2025-06-27 cs.CR cs.AI

Stanley Wu , Ronik Bhaskar , Anna Yoo Jeong Ha , Shawn Shan , Haitao Zheng , Ben Y. Zhao This is my paper
classification cs.CRcs.AI
keywords Text-to-imagediffusionmodelsDatapoisoningattacksAdversarialperturbationsVisionlanguageDirty-labelmislabelingVLMcaptioningPrompt-specific
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the widespread practice of using vision-language models (VLMs) to caption scraped training images creates a new, low-cost poisoning vector for text-to-image models. The authors propose that attackers can take benign images, add imperceptible perturbations, and make the VLM generate captions that describe an attacker-chosen target concept instead of the image's true content. These adversarially mislabeled pairs function as dirty-label poison samples: a diffusion model trained on them learns to output the reference concept's imagery whenever it is prompted with the poisoned target concept, so a prompt for 'cow' yields sunflowers. The authors measure 94%–99% poison success rates across Stable Diffusion 2.1, SDXL, and FLUX with 125 samples per concept, and show the attack also transfers to black-box commercial captioners. If correct, this is a practical way to manipulate the behavior of future image generators through the data preparation stage alone.

What carries the argument

The central mechanism is the AMP perturbation optimization, $\min_{\delta} \mathrm{Dist}(\phi(x_r + \delta), \phi(x_t))$ subject to $|\delta| < \epsilon$, where $\phi$ is the VLM's image feature extractor, $x_r$ a reference-concept image, and $x_t$ a target-concept image. Minimizing this feature-space distance makes the captioner describe the perturbed reference image as if it depicted the target, so the VLM's own text output becomes the wrong label. The paper argues this 'captioner writes the label' step is what makes AMP stealthy: the image still looks benign, the caption reads naturally, and the misalignment hides from quality filters that check caption length or aesthetic scores. The authors attribute AMP's higher potency relative to dirty-label pairs to the larger training loss that perturbed images induce.

What would settle it

Fine-tune SD2.1 on a dataset where injected perturbed images pass through the actual VLM captioner and only the measured ~70–90% fraction receive the attacker's target caption (the rest receive their true or unrelated captions), then measure poison success for the target prompts; if PSR falls to near the clean-model baseline (≈0.09), the practical potency claim is unsupported. A cheaper check is to rerun the 25-image low-dose condition while randomly discarding 10–30% of the poison samples to mimic realistic mislabel rates.

Watch

Extended reading notes

Core claim

The core discovery is that adversarial mislabeling attacks on VLMs can be weaponized as a data-preparation attack on text-to-image diffusion models. The attack, which the authors call AMP (Adversarial Mislabeling Poison), starts with a benign reference image (e.g., a statue) and perturbs it within a small $\ell_\infty$ budget (default $16/255$) so that its representation in the VLM's image-feature space moves close to a target image (e.g., a cat). The captioner then outputs a caption describing the target, and the perturbed image plus that caption becomes a poison training pair — a dirty-label sample produced by the captioning system itself. In fine-tuning experiments on SD2.1, SDXL, and FLUX, 125 such samples per concept raise the poison success rate (the probability that a prompt for the target concept produces an image a CLIP classifier does not label as the target) to 0.88–0.99, consistently matching or exceeding manually misaligned dirty-label pairs; a from-scratch SD1.5 training shows the same effect. An ensemble-ViT variant of the attack transfers to black-box commercial VLMs (Azure and Vertex AI) with per-image mislabel success above 42%, and the resulting pairs still poison downstream models.

Load-bearing premise

The headline poison-success numbers assume every injected image is successfully mislabeled by the VLM, whereas the measured per-image mislabel success rate is only 70–90%; in a real attack the effective poison dose would be smaller, and the reported 94–99% figures are conditional on perfect mislabeling.

Editorial extensions

If this is right

  • Any text-to-image training pipeline that uses a public VLM for captioning can be poisoned without the attacker touching the diffusion model or its training code.
  • Poison effectiveness scales with dose but remains nontrivial at low doses: 25 mislabeled samples per concept already push poison success to roughly 0.6.
  • The same poisoned caption set transfers across model architectures (SD2.1, SDXL, and FLUX), so a single batch of perturbed images can affect multiple downstream models.
  • Basic defenses — JPEG compression, blur, noise, and image-caption alignment filtering — can be bypassed by an adaptive attacker that incorporates them into the perturbation optimization, leaving only costly purification (e.g., DiffPure) as an effective mitigation.
  • Adversarially mislabeled pairs are more potent than ordinary dirty-label pairs, suggesting that the perturbation itself, not just the mismatched caption, contributes to the poison effect.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The attack's dependence on a shared ViT visual backbone suggests that captioners built on radically different vision encoders may be substantially more resistant; comparing AMP success across CNN-based captioners would be a direct test of this inference.
  • Because AMP potency tracks the training loss of the perturbed images, an attacker could tune the perturbation budget to sit just below a trainer's loss-based filter threshold, preserving poison effect while escaping detection — a trade-off the paper does not explore.
  • If the same adversarial mislabeling trick is applied to other VLM-mediated labeling tasks, such as video captioning or automatic alt-text generation, the poisoning mechanism could generalize beyond still-image text-to-image models.
  • The paper's finding that captioning prompts barely affect mislabel robustness hints that the vulnerability lives in the visual feature space rather than the language decoding stage; retraining or replacing the vision encoder may be a more promising defense than changing prompt design.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Adversarial Mislabeling Poison (AMP) attacks, in which an attacker adds imperceptible perturbations to benign images so that Vision-Language Model (VLM) captioners produce attacker-chosen captions, thereby creating mislabeled image-caption pairs that act as dirty-label poison samples for text-to-image diffusion models. The authors evaluate mislabeling success on three open-source VLMs (LLaVA, BLIP-3, CogVLM), poison effectiveness on four diffusion models (SD1.5, SD2.1, SDXL, FLUX) in both fine-tuning and from-scratch settings, several countermeasures and adaptive variants, and a black-box transfer attack against commercial VLMs (Microsoft Azure and Google Vertex AI). The headline results include 70-93% mislabel success rates, 94-99% poison success rates (PSR) under a perfect-mislabeling assumption, effectiveness at low doses (25 images per concept gives PSR 0.58), and >73% downstream poison success in black-box scenarios. The paper ships open-source white-box attack implementations and includes a responsible-disclosure note.

Significance. If the qualitative conclusions hold, this paper identifies a new and plausible poisoning vector that exploits the VLM captioning stage of modern text-to-image training pipelines, with the potential to affect multiple downstream models through a single captioner. The experimental scope is broad: multiple VLMs, multiple diffusion architectures, training from scratch and fine-tuning, a range of defenses, and tests against commercial APIs. The low-dose result and the black-box transferability results are particularly valuable for grounding the threat model. The paper also provides open-source code for its white-box attacks, which supports reproducibility. However, the headline poison success metric is defined as 'not the target concept' rather than 'matches the attacker's reference concept', so the paper's central claim of prompt-specific substitution is not quantitatively established by the reported PSR values. This issue, together with the reliance on perfectly mislabeled poison samples in the main experiments, means the quantitative strength of the central claims is currently overstated.

major comments (3)
  1. [Section 5.2, PSR definition] PSR(C) is defined as 1 minus the fraction of generated images classified as the target concept C by a zero-shot CLIP model over the top-100 concept set. This metric treats any output outside C, including unrelated or degenerate images, as a success. The 94-99% PSR values in Tables 4-5 therefore demonstrate suppression of the target concept but do not measure whether the model outputs the attacker's reference concept (e.g., sunflowers for 'cow'). The abstract's claim that prompts for 'cow' produce images of sunflowers is supported only by the qualitative examples in Figure 8 and the unquantified sentence in Section 5.4 that 'generated images often directly match the destination concept'. Since prompt-specific substitution is the central contribution, the authors should report a reference-match rate (the fraction of generated images classified as the reference concept) or an equivalent metric, either from the existing generated images or from new experiments, before the headline success rates can support the stated claim.
  2. [Section 5.4] The end-to-end poisoning experiments train on 125 poison samples per concept pair that are all guaranteed to be successfully mislabeled, even though Table 2 reports per-image mislabel success rates (MSR) of only 0.70-0.93. The paper acknowledges this assumption in the text, but the headline PSR values in Tables 4-5 therefore represent the outcome of successful mislabeling, not of a raw injection of 125 images. Using Table 6 as a guide, an attacker who injects 125 images with an 80% MSR would obtain roughly 100 effective poison samples and a PSR of about 0.89 rather than 0.95. The authors should report PSR as a function of injected images (not just successfully mislabeled samples), or factor the measured MSR into the reported attack success, so that the real-world cost of the attack is accurately represented.
  3. [Section 5.1 and Section 4.1] The fine-tuning dataset consists of 12,500 image-caption pairs, of which 125 x 25 = 3,125 pairs (25%) are poison samples. The abstract and Section 4.1 frame the attack as requiring only a 'small number' of poison samples per concept, but the paper never reports the per-concept frequency of benign samples in the fine-tuning or from-scratch training subsets. Without this, the reader cannot judge whether 25 or 125 images per concept is small relative to the concept's total training volume. The low-dose experiment in Table 6 partially mitigates this concern, but the authors should report the average (or distribution) of benign images per concept and the resulting poison-to-benign ratio per target concept to substantiate the 'small number' claim and to help readers assess attack feasibility in realistic web-scale datasets.
minor comments (6)
  1. [Table 5] Table 5 reports no error bars or number of independent runs for SD2.1, SDXL, and FLUX; the claimed ordering (FLUX 0.99 vs SD2.1 0.95) may reflect a single fine-tuning run and should be labeled as such.
  2. [Section 5.3, Table 3] The claim that adversarial images remain 'effectively mislabeled' across different captioning prompts is supported only by AAR/BAR values; the mislabel success rate (MSR) reported for the default prompt is not given for the alternative prompts, so the reader cannot assess whether MSR changes with the prompt.
  3. [Section 7.1, Eq. (2)] The adaptive attack's CLIP-similarity weight alpha in Eq. (2) is not specified in the text or the appendix; please report the value used and, if possible, a brief sensitivity analysis, since the behavior of the adaptive attack may depend on this hyperparameter.
  4. [Section 6 and Section 8.2] The black-box experiments in Section 8.2 use the same 'top 4 most potent target-reference concept pairs' selected in Section 6; results on these pairs may be optimistic relative to average concept pairs, and this selection should be acknowledged when interpreting the black-box MSR numbers.
  5. [Conclusion] The conclusion states 'achieving on average 95% poison success rate with only 125 samples'; the reported per-setting values in Tables 4-5 average around 0.93 rather than 0.95, and the relevant qualifier (PCB dataset, CogVLM, SD2.1) should be stated.
  6. [Section 5.4 and Abstract] The paper honestly discloses in Section 5.4 that every poison sample is successfully mislabeled in the main experiments, but the abstract's headline '94%-99% success rate' and the conclusion's '95% poison success rate' do not carry this qualifier; adding the qualifier would improve accuracy without weakening the qualitative contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's mislabeling and poisoning metrics are measured against external models and a clean baseline, not derived from the attack's own definitions.

full rationale

This is an empirical attack study in which every central quantity is measured against external, pre-trained models and compared to a clean baseline. The AMP perturbation objective (Eq. 1) minimizes VLM feature-space distance to a target image, and the mislabel metrics (MSR, AAR, BAR) and the downstream poisoning metric (PSR) are evaluated using held-out captioning, CLIP similarity, and zero-shot CLIP classification; no evaluation quantity is defined in terms of the quantity it is claimed to predict. The 'not target concept' PSR definition is a standard operationalization taken from prior poison-attack work and is reported alongside a clean-model baseline of about 0.09, so the high post-poison values are measured effects rather than analytical identities. The only self-citations (e.g., Nightshade as a baseline and as the source of the PSR convention) are background and measurement conventions, not load-bearing premises. The paper's stated assumption in Section 5.4 that every injected poison sample is successfully mislabeled is an experimental idealization, partially addressed by the dose-response experiment in Table 6; it is a limitation, not a circular derivation. No step was found where a prediction reduces to its input by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central results are empirical and rely on standard ML components: VLMs, diffusion models, CLIP, and adversarial optimization. There are no invented physical entities. The free parameters listed are experimental design choices that affect the reported success rates.

free parameters (6)
  • Perturbation budget epsilon (L-infinity) = 16/255
    Chosen as the default in Section 5.1. MSR and AAR trade off with this budget, and Figure 4 shows success plateaus beyond 16/255, so the default is a hand-selected operating point.
  • MSR difference threshold Delta = 0 (CLIP similarity difference)
    Set post hoc in Section 5.2 and Appendix A.2 based on observed distributions of Delta for benign and adversarial images. The authors argue thresholds 0-5 give the same TPR/FPR, but the metric itself is data-dependent.
  • Poison samples per concept = 125 for fine-tuning; 500 for from-scratch
    Chosen in Section 5.4. PSR scales with this count (Table 6), so the reported 0.88-0.99 PSR depends on a deliberately high dose and the assumption that all injected samples are mislabeled.
  • Adaptive attack weight alpha = not reported
    Equation (2) in Section 7.1 includes a term -alpha * CLIP similarity, but the value of alpha is never stated. This is a reproducibility gap in the adaptive attack.
  • Black-box ViT ensemble size = 8 ViTs
    Chosen in Section 8.1 from OpenCLIP. Transferability varies with the ensemble, as shown by the low 19% MSR transfer to CogVLM.
  • Concept grouping confidence threshold = 0.99
    Used in Section 5.1 to select target and reference images strongly aligned with a concept. This threshold affects which images are eligible for the attack.
assumptions (4)
  • domain assumption The VLM image feature extractor can be optimized with gradient descent to reduce the distance to a target image (Eq. 1).
    This is the core attack mechanism, taken from the cited algorithm in [92]. The paper does not verify the assumption independently.
  • domain assumption Modern text-to-image training pipelines use VLM captioners without human verification of each caption.
    Supported by the survey in Section 3.1, but the paper assumes no human-in-the-loop review that would catch mislabeled pairs.
  • domain assumption CLIP similarity is a valid measure of image-caption alignment for filtering and for success metrics.
    Used throughout for AAR, BAR, PSR classification, and alignment filtering. CLIP has known biases, but it is the standard tool in this literature.
  • domain assumption The attacker can inject images into the trainer's scraped training dataset.
    Taken from the cited prior work [8]. The paper does not re-demonstrate this capability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling." pith.science (2026). https://pith.science/paper/4JHYGR5B

@misc{pith2026250621874,
  author       = {Pith},
  title        = {Pith review of: On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4JHYGR5B}},
  note         = {Machine review of arXiv:2506.21874}
}
read the original abstract

Today's text-to-image generative models are trained on millions of images sourced from the Internet, each paired with a detailed caption produced by Vision-Language Models (VLMs). This part of the training pipeline is critical for supplying the models with large volumes of high-quality image-caption pairs during training. However, recent work suggests that VLMs are vulnerable to stealthy adversarial attacks, where adversarial perturbations are added to images to mislead the VLMs into producing incorrect captions. In this paper, we explore the feasibility of adversarial mislabeling attacks on VLMs as a mechanism to poisoning training pipelines for text-to-image models. Our experiments demonstrate that VLMs are highly vulnerable to adversarial perturbations, allowing attackers to produce benign-looking images that are consistently miscaptioned by the VLM models. This has the effect of injecting strong "dirty-label" poison samples into the training pipeline for text-to-image models, successfully altering their behavior with a small number of poisoned samples. We find that while potential defenses can be effective, they can be targeted and circumvented by adaptive attackers. This suggests a cat-and-mouse game that is likely to reduce the quality of training data and increase the cost of text-to-image model development. Finally, we demonstrate the real-world effectiveness of these attacks, achieving high attack success (over 73%) even in black-box scenarios against commercial VLMs (Google Vertex AI and Microsoft Azure).

Figures

Figures reproduced from arXiv: 2506.21874 by the authors.

Figure 1
Figure 1. An adversarial mislabeling poison (AMP) attack on a diffusion model. An attacker wants to poison a downstream [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Different poison attacks on a training pipeline with [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Based on two (clean) target and reference images, AMP produces an adversarial image (marked by a red box), which [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Adversarial images are success￾fully generated after 𝜖 = 4 255 , reaching 91% MSR at 𝜖 = 16 255 . 0 0.25 0.5 0.75 1.0 2/255 4/255 8/255 16/255 32/255 AAR BAR Normalized Image/text Similarity Perturbation Budget [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 7
Figure 7. Figure 7: Adversarially mislabeled image-caption pairs have [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Examples of images generated by SDXL models after being trained on dirty-label images and adversarially mislabeled [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: The caption quality of benign images decreases sub [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Sample AMP images that are mislabeled by Microsoft and Google online VLMs. They also successfully poison [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Adversarial images are similar in quality (CLIP [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem

    cs.CR 2026-06 unverdicted novelty 7.0 of 10

    PoisonLoRA demonstrates ~100% attack success rates for stealthy LoRA poisoning via concept hijacking and task injection on real platforms, with robustness to base model transfer and multiple remixes.

Reference graph

Works this paper leans on

99 extracted references · 67 canonical work pages · cited by 1 Pith paper

  1. [1]

    Hojjat Aghakhani, Dongyu Meng, Yu-Xiang Wang, Christopher Kruegel, and Giovanni Vigna. 2021. Bullseye polytope: a scalable clean-label poisoning attack with improved transferability. In Proc. of EuroS&P. IEEE

    2021

  2. [2]

    Eugene Bagdasaryan, Rishi Jha, Vitaly Shmatikov, and Tingwei Zhang. 2024. Adversarial illusions in multi-modal embeddings. In Proc. of USENIX Security

    2024

  3. [3]

    Bagheera. 2024. SimpleTuner. https://github.com/bghira/SimpleTuner

    2024

  4. [4]

    Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. 2024. Image hijacks: adversarial images can control generative models at runtime. In Proc. of PMLR

    2024

  5. [5]

    James Betker et al . 2023. Improving image generation with better captions. https://cdn.openai.com/papers/dall-e-3.pdf

    2023

  6. [6]

    Black Forest Labs. 2024. FLUX.1-dev Model Card. https://huggingface.co/ black-forest-labs/FLUX.1-dev

    2024

  7. [7]

    Nicholas Carlini et al. 2024. Are aligned neural networks adversarially aligned?. In Proc. of NeurIPS

    2024

  8. [8]

    Nicholas Carlini et al. 2024. Poisoning web-scale training datasets is practical. In Proc. of IEEE S&P

    2024

Show all 99 references
  1. [9]

    Nicholas Carlini and David Wagner. 2017. Adversarial examples are not easily detected: bypassing ten detection methods. In Proc. of ACM Workshop

    2017

  2. [10]

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Con- ceptual 12M: pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proc. of CVPR

    2021

  3. [11]

    Huanran Chen et al. 2024. Rethinking model ensemble in transfer-based adver- sarial attacks. In Proc. of ICLR

    2024

  4. [12]

    Junsong Chen et al. 2024. PixArt-𝛼: fast training of diffusion transformer for photorealistic text-to-image synthesis. In Proc. of ICLR

    2024

  5. [13]

    Junsong Chen et al. 2024. PixArt- Σ: weak-to-strong training of diffusion trans- former for 4k text-to-image generation. In Proc. of ECCV

    2024

  6. [14]

    Long Chen et al. 2024. Driving with LLMs: fusing object-level vector modality for explainable autonomous driving. In Proc. of IEEE ICRA

    2024

  7. [15]

    Steven Chen, Nicholas Carlini, and David Wagner. 2020. Stateful detection of black-box adversarial attacks. In Proc. of CCS

    2020

  8. [16]

    Weixin Chen, Dawn Song, and Bo Li. 2023. TrojDiff: trojan attacks on diffusion models with diverse targets. In Proc. of CVPR

    2023

  9. [17]

    Xiaoyi Chen et al . 2021. BadNL: backdoor attacks against NLP models with semantic-preserving improvements. In Proc. of ACSAC

    2021

  10. [18]

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)

    2017 arXiv

  11. [19]

    Zhe Chen et al. 2024. InternVL: scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proc. of CVPR

    2024

  12. [20]

    Mehdi Cherti et al. 2023. Reproducible scaling laws for contrastive language- image learning. In Proc. of CVPR

    2023

  13. [21]

    Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho. 2023. How to backdoor diffusion models?. In Proc. of CVPR

    2023

  14. [22]

    Civitai. 2022. What the heck is Civitai? https://civitai.com/content/ guides/what-is-civitai . 13

    2022

  15. [23]

    Google Deepmind. 2024. Imagen 3. https://deepmind.google/ technologies/imagen-3/

    2024

  16. [24]

    Wenxin Ding, Cathy Y Li, Shawn Shan, Ben Y Zhao, and Haitao Zheng. 2024. Understanding implosion in text-to-image generative models. In Proc. of CCS

    2024

  17. [25]

    Yinpeng Dong et al . 2023. How robust is Google’s Bard to adversarial image attacks?. In Proc. of NeurIPS Workshop

    2023

  18. [26]

    Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M. Roy. 2016. A study of the effect of JPG compression on adversarial images. arXiv preprint arXiv:1608.00853 (2016)

    2016 arXiv

  19. [27]

    Patrick Esser et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Proc. of ICML

    2024

  20. [28]

    Kuofeng Gao et al. 2024. Inducing high energy-latency of large vision-language models with verbose images. In Proc. of ICLR

    2024

  21. [29]

    Kuofeng Gao, Yang Bai, Jiawang Bai, Yong Yang, and Shu-Tao Xia. 2024. Adver- sarial robustness for visual grounding of multimodal large language models. In Proc. of ICLR Workshop

    2024

  22. [30]

    Peng Gao et al. 2025. Lumina-T2X: scalable flow-based large diffusion transformer for flexible resolution generation. In Proc. of ICLR

    2025

  23. [31]

    Google. 2024. Vertex AI platform. https://cloud.google.com/vertex-ai? hl=en

    2024

  24. [32]

    Abigail Graese, Andras Rozsa, and Terrance E Boult. 2016. Assessing threat of adversarial examples on deep neural networks. In Proc. of ICMLA

    2016

  25. [33]

    Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2019. BadNets: evaluating backdooring attacks on deep neural networks. IEEE Access (2019)

    2019

  26. [34]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Proc. of NeurIPS

    2020

  27. [35]

    Yihao Huang et al. 2024. Personalization as a shortcut for few-shot backdoor attack against text-to-image diffusion models. In Proc. of AAAI

    2024

  28. [36]

    jhc13. 2024. TagGUI. https://github.com/jhc13/taggui

    2024

  29. [37]

    John. 2023. We need CogVLM support - extremely good image and text analysis, feels like a multi generational step forward. https://github.com/ggerganov/ llama.cpp/issues/4387

    2023

  30. [38]

    Huiying Li, Shawn Shan, Emily Wenger, Jiayun Zhang, Haitao Zheng, and Ben Y Zhao. 2022. Blacklight: scalable defense for neural networks against query-based black-box attacks. In Proc. of USENIX Security

    2022

  31. [39]

    Zhimin Li et al. 2024. Hunyuan-DiT: a powerful multi-resolution diffusion trans- former with fine-grained chinese understanding. arXiv preprint arXiv:2405.08748 (2024)

    2024 arXiv

  32. [40]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruc- tion tuning. In Proc. of NeurIPS

    2024

  33. [41]

    Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. 2017. Delving into transferable adversarial examples and black-box attacks. Proc. of ICLR

    2017

  34. [42]

    LyPreto. 2023. Shinning the spotlight on CogVLM. https://www.reddit.com/ r/LocalLLaMA/comments/18evtgp/shinning_the_spotlight_on_cogvlm/

    2023

  35. [43]

    Microsoft. 2024. Image captions. https://learn.microsoft.com/en- us/azure/ai-services/computer-vision/concept-describe-images- 40?tabs=image

    2024

  36. [44]

    Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco, Sewoong Oh, and Ludwig Schmidt. 2024. Improving multimodal datasets with image captioning. In Proc. of NeurIPS

    2024

  37. [45]

    Tuan Anh Nguyen and Anh Tran. 2020. Input-aware dynamic backdoor attack

    2020

  38. [46]

    Ming Nie et al . 2025. Reason2Drive: towards interpretable and chain-based reasoning for autonomous driving. In Proc. of ECCV

    2025

  39. [47]

    Weili Nie et al. 2022. Diffusion models for adversarial purification. In Proc. of ICML

    2022

  40. [48]

    NovelAI. 2022. NovelAI changelog. https://novelai.net/updates

    2022

  41. [49]

    Zhuoshi Pan et al. 2023. From trojan horses to castle walls: unveiling bilateral backdoor effects in diffusion models. In Proc. NeurIPS Workshop

    2023

  42. [50]

    Dustin Podell et al . 2024. SDXL: improving latent diffusion models for high- resolution image synthesis. In Proc. of ICLR

    2024

  43. [51]

    PseudoTerminal X. 2024. Photo Concept Bucket. https://huggingface.co/ datasets/bghira/photo-concept-bucket

    2024

  44. [52]

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, and Prateek Mittal

  45. [53]

    Alec Radford et al . 2021. Learning transferable visual models from natural language supervision. In Proc. of ICML

    2021

  46. [54]

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen

  47. [55]

    Christoph Reich, Biplob Debnath, Deep Patel, and Srimat Chakradhar. 2024. Differentiable JPEG: the devil is in the details. In Proc. of W ACV

    2024

  48. [56]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proc. of CVPR

    2022

  49. [57]

    Chitwan Saharia et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. In Proc. of NeurIPS

    2022

  50. [58]

    Scenario.gg. 2022. AI-generated game assets. https://www.scenario.gg/

    2022

  51. [59]

    Christoph Schuhmann. 2022. LAION-Aesthetics. https://laion.ai/blog/ laion-aesthetics/

    2022

  52. [60]

    Christoph Schuhmann et al. 2022. LAION-5B: an open large-scale dataset for training next generation image-text models. In Proc. of NeurIPS

    2022

  53. [61]

    Eyal Segalis, Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan

  54. [62]

    Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. 2023. Glaze: protecting artists from style mimicry by text-to-image models. In Proc. of USENIX Security

    2023

  55. [63]

    Shawn Shan, Wenxin Ding, Josephine Passananti, Stanley Wu, Haitao Zheng, and Ben Y. Zhao. 2024. Nightshade: prompt-specific poisoning attacks on text-to- image generative models. In Proc. of IEEE S&P

    2024

  56. [64]

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2024. Jailbreak in pieces: compositional adversarial attacks on multi-modal language models. In Proc. of ICLR

    2024

  57. [65]

    Richard Shin and Dawn Song. 2017. JPEG-resistant adversarial im- ages. https://machine-learning-and-security.github.io/papers/ mlsec17_paper_54.pdf

    2017

  58. [66]

    Ilia Shumailov et al . 2024. AI models collapse when trained on recursively generated data. Nature (2024)

    2024

  59. [67]

    Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising diffusion implicit models. In Proc. of ICLR

    2021

  60. [68]

    Yang Song et al . 2021. Score-based generative modeling through stochastic differential equations. In Proc. of ICLR

    2021

  61. [69]

    Stability AI. 2022. Stable Diffusion 2.0 release. https://stability.ai/blog/ stable-diffusion-v2-release

    2022

  62. [70]

    Stability AI. 2024. Stable Diffusion v1-5 Model Card. https://huggingface. co/stable-diffusion-v1-5/stable-diffusion-v1-5

    2024

  63. [71]

    Stability AI. 2024. Stable Diffusion v2-1 Model Card. https://huggingface. co/stabilityai/stable-diffusion-2-1

    2024

  64. [72]

    Kolors Team. 2024. Kolors: effective training of diffusion model for photorealistic text-to-image synthesis. arXiv preprint (2024). https://github.com/Kwai- Kolors/Kolors/blob/master/imgs/Kolors_paper.pdf

    2024

  65. [73]

    Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. 2020. On adaptive attacks to adversarial example defenses. In Proc. of NeurIPS

    2020

  66. [74]

    Tony Ho Tran. 2022. Image apps like Lensa AI are sweeping the internet, and stealing from artists. https://www.thedailybeast.com/how-lensa-ai-and- image-generators-steal-from-artists

    2022

  67. [75]

    Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. 2023. Exploring clip for assessing the look and feel of images. In Proc. of AAAI

    2023

  68. [76]

    Weihan Wang et al. 2024. CogVLM: visual expert for pretrained language models. In Proc. of NeurIPS

    2024

  69. [77]

    Yizhen Wang and Kamalika Chaudhuri. 2018. Data poisoning attacks against online learning. arXiv preprint arXiv:1808.08994 (2018)

    2018 arXiv

  70. [78]

    Emily Wenger, Josephine Passananti, Arjun Nitin Bhagoji, Yuanshun Yao, Haitao Zheng, and Ben Y Zhao. 2021. Backdoor attacks against deep learning systems in the physical world. In Proc. of CVPR

    2021

  71. [79]

    Chen Henry Wu et al. 2025. Dissecting adversarial robustness of multimodal LM agents. In Proc. of ICLR

    2025

  72. [80]

    Chang Xiao, Peilin Zhong, and Changxi Zheng. 2020. Enhancing adversarial defense by k-Winners-Take-All. InProc. of ICLR

    2020

  73. [81]

    Enze Xie et al. 2025. SANA: efficient high-resolution image synthesis with linear diffusion transformers. In Proc. of ICLR

    2025

  74. [82]

    Chejian Xu et al. 2024. AdvWeb: controllable black-box attacks on vlm-powered web agents. arXiv preprint arXiv:2410.17401 (2024)

    2024 arXiv

  75. [83]

    Yuancheng Xu et al. 2025. Shadowcast: stealthy data poisoning attacks against vision-language models. In Proc. of NeurIPS

    2025

  76. [84]

    Le Xue et al. 2024. xGen-MM (BLIP-3): a family of open large multimodal models. arXiv preprint arXiv:2408.08872 (2024)

    2024

  77. [85]

    Mingfu Xue, Can He, Jian Wang, and Weiqiang Liu. 2020. One-to-N & N-to-One: two advanced backdoor attacks against deep learning models. IEEE TDSC (2020)

    2020

  78. [86]

    Chengyuan Yao et al. 2021. Automated discovery of adaptive attacks on adver- sarial defenses. In Proc. of NeurIPS

    2021

  79. [87]

    Ziyi Yin et al . 2024. VLATTACK: multimodal adversarial attacks on vision- language tasks via pre-trained models. In Proc. of NeurIPS

    2024

  80. [88]

    Weinberger

    Tao Yu, Shengyuan Hu, Chuan Guo, Wei-Lun Chao, and Kilian Q. Weinberger

  81. [89]

    Shengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. 2023. Text-to-image diffusion models can be easily backdoored through multimodal data poisoning. In Proc. of International Conference on Multimedia

    2023

  82. [90]

    Tianyuan Zhang et al. 2024. Visual adversarial attack on vision-language models for autonomous driving. arXiv preprint arXiv:2411.18275 (2024)

    2024 arXiv

  83. [91]

    Xuezhou Zhang, Xiaojin Zhu, and Laurent Lessard. 2020. Online data poisoning attacks. In Proc. of L4DC. PMLR

    2020

  84. [92]

    Yunqing Zhao et al. 2024. On evaluating adversarial robustness of large vision- language models. In Proc. of NeurIPS. 14

    2024

  85. [93]

    Wanqi Zhou, Shuanghao Bai, Qibin Zhao, and Badong Chen. 2024. Revisiting the adversarial robustness of vision language models: a multimodal perspective. arXiv preprint arXiv:2404.19287 (2024)

    2024 arXiv

  86. [94]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2024. MiniGPT-4: enhancing vision-language understanding with advanced large lan- guage models. In Proc. of ICLR

    2024

  87. [95]

    TE” stands for “text en- coder

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: a benchmarking platform for text generation models. In Proc. of SIGIR. A Appendix A.1 Diffusion Models and Training Parameters In Table 12, we outline the key differences between ...

    2018

  88. [2019]

    A new defense against adversarial images: turning a weakness into a strength. In Proc. of NeurIPS

  89. [2022]

    arXiv preprint arXiv:2204.06125 (2022)

    Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022)

    2022 arXiv

  90. [2023]

    arXiv preprint arXiv:2310.16656 (2023)

    A picture is worth a thousand words: principled recaptioning improves image generation. arXiv preprint arXiv:2310.16656 (2023)

    2023 arXiv

  91. [2024]

    AAAI Press

    Visual adversarial examples jailbreak large language models. AAAI Press

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.