{"id":"b2dbfcd8-44ad-4993-a489-85bd441ac326","arxiv_id":"2506.21874","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An adversarial mislabeling attack on VLM captioners can inject dirty-label poison samples into text-to-image training data and corrupt a model's output for specific prompts with a small number of images.","lead":"This paper shows that small pixel changes added to images can trick the AI systems that write captions for training data, and that those wrongly captioned images can then poison text-to-image models so they draw the wrong thing for a prompt. The work suggests a realistic way for attackers to corrupt image-generation models at the data-preparation stage, and finds that current defenses are only partially effective.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PSR is defined as \"not the target concept,\" not \"matches the reference concept\"; the headline 94–99% success may overstate prompt-specific substitution, since any non-target output counts as a success.","rationale":"The reader's concern about perfect mislabeling is valid, but I identify a more directly load-bearing measurement issue: PSR only checks that generated images are not the target concept, whereas the attack objective defined in Section 4.1 is to make the model output images associated with the reference concept. The clean-model baseline PSR of about 0.09 shows the CLIP classifier is meaningful for detecting target images, but it never quantifies whether poisoned-model outputs are actually the intended reference images. Because generated images are classified into exactly one of the top-100 concepts, \"not C\" is a much easier bar than \"is the reference concept,\" so the headline 94–99% success rates may substantially overstate the specificity of the attack. The perfect-mislabeling issue is partially mitigated by the attacker's ability to test and select successful AMP images before deployment, which the paper itself exploits in Section 8. I therefore would not change the overall CONDITIONAL verdict, but the conditions should be expanded to require reference-match rates and unconditional injection PSRs. This makes my recommendation UNCHANGED relative to the reader's verdict, with partial agreement: both concerns reduce the quantitative strength of the central claim, but through different mechanisms.","tokens_in":22769,"tokens_out":7979,"duration_ms":89640,"concrete_test":"Re-run the poisoning evaluations for Tables 4, 5, and 6, and compute a reference-match rate (RMR): the fraction of generated images whose CLIP classification over the top-100 concepts is the reference concept (or whose CLIP similarity to the reference caption exceeds that of all other top-100 concepts). Compare RMR to the reported PSR and include a per-concept confusion matrix for the 25 target/reference pairs. If RMR is close to PSR (within roughly 0.1), the 94–99% headline is validated; if RMR is substantially lower, the attack is better described as prompt corruption than as targeted substitution toward the reference concept.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.2 defines PSR(C) as 1 minus the fraction of generated images classified as C by a zero-shot CLIP model over the top-100 concept set. This metric rewards any non-C output, including unrelated or degenerate images; it does not require the output to correspond to the attacker's reference concept (e.g., statue for the prompt \"cat\"). The abstract's stronger claim that prompts for \"cow\" produce sunflowers is therefore not actually measured by the headline 94–99% numbers in Tables 4–5. A model that simply produced arbitrary non-C images would also score a high PSR, so the central \"specific prompt poisoning\" claim is only as strong as the unquantified examples in Figure 8. The perfect-mislabeling assumption flagged by the reader is real but secondary: an attacker with white-box or API access can verify and select successful AMP images before posting, and Section 8 does exactly this for commercial VLMs. Reporting unconditional injection rates would refine the attack-cost model without changing the mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Adversarial Mislabeling Poison (AMP) attacks, in which an attacker adds imperceptible perturbations to benign images so that Vision-Language Model (VLM) captioners produce attacker-chosen captions, thereby creating mislabeled image-caption pairs that act as dirty-label poison samples for text-to-image diffusion models. The authors evaluate mislabeling success on three open-source VLMs (LLaVA, BLIP-3, CogVLM), poison effectiveness on four diffusion models (SD1.5, SD2.1, SDXL, FLUX) in both fine-tuning and from-scratch settings, several countermeasures and adaptive variants, and a black-box transfer attack against commercial VLMs (Microsoft Azure and Google Vertex AI). The headline results include 70-93% mislabel success rates, 94-99% poison success rates (PSR) under a perfect-mislabeling assumption, effectiveness at low doses (25 images per concept gives PSR 0.58), and >73% downstream poison success in black-box scenarios. The paper ships open-source white-box attack implementations and includes a responsible-disclosure note.","tokens_in":22885,"tokens_out":7670,"duration_ms":73072,"significance":"If the qualitative conclusions hold, this paper identifies a new and plausible poisoning vector that exploits the VLM captioning stage of modern text-to-image training pipelines, with the potential to affect multiple downstream models through a single captioner. The experimental scope is broad: multiple VLMs, multiple diffusion architectures, training from scratch and fine-tuning, a range of defenses, and tests against commercial APIs. The low-dose result and the black-box transferability results are particularly valuable for grounding the threat model. The paper also provides open-source code for its white-box attacks, which supports reproducibility. However, the headline poison success metric is defined as 'not the target concept' rather than 'matches the attacker's reference concept', so the paper's central claim of prompt-specific substitution is not quantitatively established by the reported PSR values. This issue, together with the reliance on perfectly mislabeled poison samples in the main experiments, means the quantitative strength of the central claims is currently overstated.","major_comments":[{"comment":"PSR(C) is defined as 1 minus the fraction of generated images classified as the target concept C by a zero-shot CLIP model over the top-100 concept set. This metric treats any output outside C, including unrelated or degenerate images, as a success. The 94-99% PSR values in Tables 4-5 therefore demonstrate suppression of the target concept but do not measure whether the model outputs the attacker's reference concept (e.g., sunflowers for 'cow'). The abstract's claim that prompts for 'cow' produce images of sunflowers is supported only by the qualitative examples in Figure 8 and the unquantified sentence in Section 5.4 that 'generated images often directly match the destination concept'. Since prompt-specific substitution is the central contribution, the authors should report a reference-match rate (the fraction of generated images classified as the reference concept) or an equivalent metric, either from the existing generated images or from new experiments, before the headline success rates can support the stated claim.","section":"Section 5.2, PSR definition"},{"comment":"The end-to-end poisoning experiments train on 125 poison samples per concept pair that are all guaranteed to be successfully mislabeled, even though Table 2 reports per-image mislabel success rates (MSR) of only 0.70-0.93. The paper acknowledges this assumption in the text, but the headline PSR values in Tables 4-5 therefore represent the outcome of successful mislabeling, not of a raw injection of 125 images. Using Table 6 as a guide, an attacker who injects 125 images with an 80% MSR would obtain roughly 100 effective poison samples and a PSR of about 0.89 rather than 0.95. The authors should report PSR as a function of injected images (not just successfully mislabeled samples), or factor the measured MSR into the reported attack success, so that the real-world cost of the attack is accurately represented.","section":"Section 5.4"},{"comment":"The fine-tuning dataset consists of 12,500 image-caption pairs, of which 125 x 25 = 3,125 pairs (25%) are poison samples. The abstract and Section 4.1 frame the attack as requiring only a 'small number' of poison samples per concept, but the paper never reports the per-concept frequency of benign samples in the fine-tuning or from-scratch training subsets. Without this, the reader cannot judge whether 25 or 125 images per concept is small relative to the concept's total training volume. The low-dose experiment in Table 6 partially mitigates this concern, but the authors should report the average (or distribution) of benign images per concept and the resulting poison-to-benign ratio per target concept to substantiate the 'small number' claim and to help readers assess attack feasibility in realistic web-scale datasets.","section":"Section 5.1 and Section 4.1"}],"minor_comments":[{"comment":"Table 5 reports no error bars or number of independent runs for SD2.1, SDXL, and FLUX; the claimed ordering (FLUX 0.99 vs SD2.1 0.95) may reflect a single fine-tuning run and should be labeled as such.","section":"Table 5"},{"comment":"The claim that adversarial images remain 'effectively mislabeled' across different captioning prompts is supported only by AAR/BAR values; the mislabel success rate (MSR) reported for the default prompt is not given for the alternative prompts, so the reader cannot assess whether MSR changes with the prompt.","section":"Section 5.3, Table 3"},{"comment":"The adaptive attack's CLIP-similarity weight alpha in Eq. (2) is not specified in the text or the appendix; please report the value used and, if possible, a brief sensitivity analysis, since the behavior of the adaptive attack may depend on this hyperparameter.","section":"Section 7.1, Eq. (2)"},{"comment":"The black-box experiments in Section 8.2 use the same 'top 4 most potent target-reference concept pairs' selected in Section 6; results on these pairs may be optimistic relative to average concept pairs, and this selection should be acknowledged when interpreting the black-box MSR numbers.","section":"Section 6 and Section 8.2"},{"comment":"The conclusion states 'achieving on average 95% poison success rate with only 125 samples'; the reported per-setting values in Tables 4-5 average around 0.93 rather than 0.95, and the relevant qualifier (PCB dataset, CogVLM, SD2.1) should be stated.","section":"Conclusion"},{"comment":"The paper honestly discloses in Section 5.4 that every poison sample is successfully mislabeled in the main experiments, but the abstract's headline '94%-99% success rate' and the conclusion's '95% poison success rate' do not carry this qualifier; adding the qualifier would improve accuracy without weakening the qualitative contribution.","section":"Section 5.4 and Abstract"}],"recommendation":"major_revision","confidential_remarks":"The main blocker is the PSR metric: the current definition does not measure the claimed prompt-specific substitution, so the headline numbers are not evidence for the abstract's stronger statements. This is fixable by re-analyzing the existing generated images (e.g., reporting reference-match rates) or by running a small additional evaluation. The perfect-mislabeling assumption is disclosed and can be addressed by reporting injection-based curves. If these two points are handled, the paper's contribution is likely to be a solid empirical attack study suitable for this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a real new attack surface and the paper earns its place, but the headline numbers overstate prompt specificity. The combination is new—targeted adversarial perturbations against VLM captioners to manufacture dirty-label poison samples for downstream text-to-image models. The components are known (VLM adversarial attacks, Nightshade-style prompt poisoning), but the contribution is the coupling, and the paper builds the case carefully.\n\nThe documentation survey showing all disclosed modern text-to-image models use VLM captioners is useful. The experimental sweep is extensive: three VLMs, four diffusion models, from-scratch and fine-tuning regimes, plus black-box transfer to Google and Microsoft APIs. The black-box success (over 42% MSR on commercial VLMs, with >73% PSR after selecting successful images) is practically interesting. The adaptive attacks against JPEG and alignment filtering are well done, and the DiffPure cost analysis is honest.\n\nSoft spots, in proportion. The main one is the PSR metric. PSR(C) counts any generated image classified as not C as success, so the 94–99% numbers do not measure the abstract's claim that a \"cow\" prompt produces sunflowers. The examples in Figure 8 do show that effect, but they are selected. To make the central claim stick, report the fraction of outputs matching the reference concept. That said, the mechanism does not collapse; even under a stricter metric, the qualitative shift in behavior is real.\n\nSecond, the end-to-end fine-tuning experiments assume all injected images are successfully mislabeled (125 per concept pair), while per-image MSR is 70–90%. The low-dose results (25 images give PSR 0.58) show non-trivial impact, and the black-box section explicitly filters for successful images, so this is an attack-cost gap rather than a mechanism flaw. Reporting unconditional rates would sharpen the real-world risk.\n\nMinor: the adaptive attack's alpha in Eq. (2) is not given; several tables lack error bars; the MSR threshold justification in Appendix A.2 reads a bit post hoc, though the empirical support is reasonable.\n\nWho this is for: security folks working on data poisoning, model training pipelines, and adversarial robustness of VLMs. It deserves a serious referee. I'd send it out and ask for conditional success metrics, variance estimates, and the alpha value. The paper is more than salvageable—it's a solid feasibility study standing on a real attack surface.","headline":"A real new attack surface and a thorough feasibility study, but the headline PSR metric overstates prompt-specific substitution; still deserves a serious referee.","tokens_in":23524,"tokens_out":2421,"would_cite":true,"duration_ms":24903,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adversarially mislabeled image–caption pairs, produced by perturbing benign images so that VLM captioners output attacker-chosen captions, act as highly potent dirty-label poison samples that redirect text-to-image…","keywords":["Text-to-image diffusion models","Data poisoning attacks","Adversarial perturbations","Vision language models","Dirty-label poisoning","Adversarial mislabeling","VLM captioning","Prompt-specific poisoning"],"falsifier":"Fine-tune SD2.1 on a dataset where injected perturbed images pass through the actual VLM captioner and only the measured ~70–90% fraction receive the attacker's target caption (the rest receive their true or unrelated captions), then measure poison success for the target prompts; if PSR falls to near the clean-model baseline (≈0.09), the practical potency claim is unsupported. A cheaper check is to rerun the 25-image low-dose condition while randomly discarding 10–30% of the poison samples to mimic realistic mislabel rates.","tokens_in":22481,"feed_emoji":"🎨","tokens_out":13666,"duration_ms":125582,"temperature":0.7,"pith_summary":"This paper asks whether the widespread practice of using vision-language models (VLMs) to caption scraped training images creates a new, low-cost poisoning vector for text-to-image models. The authors propose that attackers can take benign images, add imperceptible perturbations, and make the VLM generate captions that describe an attacker-chosen target concept instead of the image's true content. These adversarially mislabeled pairs function as dirty-label poison samples: a diffusion model trained on them learns to output the reference concept's imagery whenever it is prompted with the poisoned target concept, so a prompt for 'cow' yields sunflowers. The authors measure 94%–99% poison success rates across Stable Diffusion 2.1, SDXL, and FLUX with 125 samples per concept, and show the attack also transfers to black-box commercial captioners. If correct, this is a practical way to manipulate the behavior of future image generators through the data preparation stage alone.","feed_headline":"Perturbed images trick AI captioners into poisoning training data","feed_subtitle":"Only a few dozen doctored images can make a text-to-image model swap one concept for another.","key_machinery":"The central mechanism is the AMP perturbation optimization, $\\min_{\\delta} \\mathrm{Dist}(\\phi(x_r + \\delta), \\phi(x_t))$ subject to $|\\delta| < \\epsilon$, where $\\phi$ is the VLM's image feature extractor, $x_r$ a reference-concept image, and $x_t$ a target-concept image. Minimizing this feature-space distance makes the captioner describe the perturbed reference image as if it depicted the target, so the VLM's own text output becomes the wrong label. The paper argues this 'captioner writes the label' step is what makes AMP stealthy: the image still looks benign, the caption reads naturally, and the misalignment hides from quality filters that check caption length or aesthetic scores. The authors attribute AMP's higher potency relative to dirty-label pairs to the larger training loss that perturbed images induce.","core_discovery":"The core discovery is that adversarial mislabeling attacks on VLMs can be weaponized as a data-preparation attack on text-to-image diffusion models. The attack, which the authors call AMP (Adversarial Mislabeling Poison), starts with a benign reference image (e.g., a statue) and perturbs it within a small $\\ell_\\infty$ budget (default $16/255$) so that its representation in the VLM's image-feature space moves close to a target image (e.g., a cat). The captioner then outputs a caption describing the target, and the perturbed image plus that caption becomes a poison training pair — a dirty-label sample produced by the captioning system itself. In fine-tuning experiments on SD2.1, SDXL, and FLUX, 125 such samples per concept raise the poison success rate (the probability that a prompt for the target concept produces an image a CLIP classifier does not label as the target) to 0.88–0.99, consistently matching or exceeding manually misaligned dirty-label pairs; a from-scratch SD1.5 training shows the same effect. An ensemble-ViT variant of the attack transfers to black-box commercial VLMs (Azure and Vertex AI) with per-image mislabel success above 42%, and the resulting pairs still poison downstream models.","pith_inferences":["The attack's dependence on a shared ViT visual backbone suggests that captioners built on radically different vision encoders may be substantially more resistant; comparing AMP success across CNN-based captioners would be a direct test of this inference.","Because AMP potency tracks the training loss of the perturbed images, an attacker could tune the perturbation budget to sit just below a trainer's loss-based filter threshold, preserving poison effect while escaping detection — a trade-off the paper does not explore.","If the same adversarial mislabeling trick is applied to other VLM-mediated labeling tasks, such as video captioning or automatic alt-text generation, the poisoning mechanism could generalize beyond still-image text-to-image models.","The paper's finding that captioning prompts barely affect mislabel robustness hints that the vulnerability lives in the visual feature space rather than the language decoding stage; retraining or replacing the vision encoder may be a more promising defense than changing prompt design."],"forward_implications":["Any text-to-image training pipeline that uses a public VLM for captioning can be poisoned without the attacker touching the diffusion model or its training code.","Poison effectiveness scales with dose but remains nontrivial at low doses: 25 mislabeled samples per concept already push poison success to roughly 0.6.","The same poisoned caption set transfers across model architectures (SD2.1, SDXL, and FLUX), so a single batch of perturbed images can affect multiple downstream models.","Basic defenses — JPEG compression, blur, noise, and image-caption alignment filtering — can be bypassed by an adaptive attacker that incorporates them into the perturbation optimization, leaving only costly purification (e.g., DiffPure) as an effective mitigation.","Adversarially mislabeled pairs are more potent than ordinary dirty-label pairs, suggesting that the perturbation itself, not just the mismatched caption, contributes to the poison effect."],"supporting_citations":[{"why":"Supplies the image-to-image adversarial perturbation algorithm (feature-space distance minimization) that AMP uses to generate mislabeled images.","marker":"[92]"},{"why":"The prior prompt-specific poisoning attack on text-to-image models that AMP builds on and compares against as the dirty-label baseline.","marker":"[63]"},{"why":"Demonstrates targeted adversarial attacks against a VLM, establishing that VLMs can be steered to emit attacker-chosen captions.","marker":"[25]"},{"why":"Shows stealthy data poisoning against VLMs, supporting the claim that VLM captioners are a vulnerable link in training pipelines.","marker":"[83]"},{"why":"The first technical report documenting that VLM-generated captions improve text-to-image training, grounding the survey of captioning practices.","marker":"[5]"},{"why":"Supplies the ensemble of ViT image encoders used in the black-box transferability optimization.","marker":"[20]"},{"why":"The large-scale web-scraped dataset that motivates the threat model of adversaries injecting images into scraped training data.","marker":"[60]"},{"why":"An example of a modern text-to-image model that uses a public VLM for captioning, supporting the survey finding.","marker":"[12]"}],"fun_headline_variants":["Adversarial mislabeling poisons text-to-image models with few samples","Tricking captioners to poison image generators with few samples","125 mislabeled captions can retarget a text-to-image model","Misleading captions via adversarial images can poison AI image models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline poison-success numbers assume every injected image is successfully mislabeled by the VLM, whereas the measured per-image mislabel success rate is only 70–90%; in a real attack the effective poison dose would be smaller, and the reported 94–99% figures are conditional on perfect mislabeling.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial mislabeling poisons text-to-image models with few samples","Tricking captioners to poison image generators with few samples","125 mislabeled captions can retarget a text-to-image model","Misleading captions via adversarial images can poison AI image models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001026,"raw_usage":{"total_tokens":4389,"prompt_tokens":1071,"completion_tokens":3318,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":687,"completion_tokens_details":{"reasoning_tokens":3243}},"tokens_in":687,"tokens_out":3318,"duration_ms":23096,"temperature":1.0,"reasoning_tokens":3243,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:17:04.735212+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune SD2.1 on a dataset where injected perturbed images pass through the actual VLM captioner and only the measured ~70–90% fraction receive the attacker's target caption (the rest receive their true or unrelated captions), then measure poison success for the target prompts; if PSR falls to near the clean-model baseline (≈0.09), the practical potency claim is unsupported. A cheaper check is to rerun the 25-image low-dose condition while randomly discarding 10–30% of the poison samples to mimic realistic mislabel rates.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the image-to-image adversarial perturbation algorithm (feature-space distance minimization) that AMP uses to generate mislabeled images."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The prior prompt-specific poisoning attack on text-to-image models that AMP builds on and compares against as the dirty-label baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates targeted adversarial attacks against a VLM, establishing that VLMs can be steered to emit attacker-chosen captions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows stealthy data poisoning against VLMs, supporting the claim that VLM captioners are a vulnerable link in training pipelines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The large-scale web-scraped dataset that motivates the threat model of adversaries injecting images into scraped training data."}],"review_version":1}