{"id":"9598e6c8-9ccb-4c2f-a787-5624c2700603","arxiv_id":"2412.05538","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Typography inserted into input images can manipulate CLIP-guided image generation models to produce harmful or biased content, and existing text-focused defenses do not catch it.","lead":"Researchers show that inserting text into input images, such as the word 'naked' or 'bloody', can steer AI image generators toward producing harmful or biased content, even when the text prompt is benign. The finding exposes a security gap in image editing tools and comes with a benchmark dataset for testing this vulnerability.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack's semantic impact is quantified only by CLIP Score, computed with the same CLIP encoder the generation models use; without human validation, score gains could reflect residual typo text in outputs rather than genuine concept depiction.","rationale":"The paper's core contribution is the claim that typography in the vision modality causes semantic steering of image generation. The quantitative backbone of that claim is the CLIP Score delta between generated images and typo words. Because the attacked models and the evaluation metric share the same CLIP vision encoder, and because the typo is a literal text token inserted into the input image, the metric can be inflated by the model copying the word into the output rather than by semantic adoption of the concept. This is not a question of internal consistency: the numbers are computed as stated, and the visual examples are suggestive. The risk is external validity. Human annotation is the natural missing control, and it is absent. The defense results are less load-bearing because it is unsurprising that text-prompt guards do not inspect image inputs; the 'ignore text' prompt test is a useful attempt but still does not repair the metric issue. The overall verdict should remain conditional: the phenomenon is plausible and the dataset is a useful resource, but the central quantitative evidence needs independent validation before the strong claim of semantic and harmful steering is accepted.","tokens_in":24151,"tokens_out":4056,"duration_ms":42438,"concrete_test":"Run a human annotation study on a stratified sample (e.g., 50 images per model per typo for visible and invisible MT, roughly 600 images) from the released dataset. For each generated image, ask three annotators: (1) Does the image contain the literal typo text, visible or legible? (2) Does it clearly depict the typo concept, e.g., a naked person, blood, a hat, or the demographic attribute? (3) Is the content harmful, biased, or neutral? Compute the correlation between CLIP Score delta and annotator judgments. If most high-delta images are labeled 'text visible, concept not depicted,' or if annotators do not confirm the concept, the semantic-steering claim is unsupported; if annotators do confirm it, the concern is resolved. Supplement with OCR to quantify residual text persistence in the generated outputs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that inserting typography into the input image steers generation toward harmful, biased, or neutral content—rests on the CLIP Score deltas in Tables 2 and 3. The metric is computed between the generated image and the typo word using CLIP, which is also the encoder used by UnCLIP and IP-Adapter to condition generation. This creates two confounds. First, if the output image retains the typo as rendered text (e.g., the word 'naked' remains visible), CLIP similarity to the text 'naked' rises by construction even though no harmful concept is depicted. Second, CLIP Score measures embedding closeness, not human-visible semantics; a gain of 0.6–5 units on a low absolute baseline (about 16–24) has no established perceptual anchor. The paper's visual examples are anecdotal, and no human evaluation, OCR check, or independent detector is provided. The 'invisible' typography results in Table 3, where some deltas are 0.2–1.0, are especially vulnerable to this concern. The FID results in Appendix C partially address image-level deviation, but FID measures distribution distance from clean inputs, not whether the deviation is semantic or harmful. Without content-based validation, the quantitative evidence does not distinguish 'the model copied the word' from 'the model depicted the concept.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Vision Modal Threats in Image Generation Models (VMT-IGMs) dataset and demonstrates typographic attacks in the vision modality for CLIP-guided image generation models in image-to-image tasks. It evaluates four models (UnCLIP, SD1.5, SDXL, FLUX) using CLIP Score and FID across factor-modification and malicious-threat settings, tests five text-prompt-based defenses plus a post-generation safety checker, and proposes the dataset as a benchmark for future defenses.","tokens_in":24441,"tokens_out":4613,"duration_ms":43230,"significance":"If validated, the paper identifies a real and previously under-explored attack surface in mainstream image editing pipelines and provides a reusable benchmark. The qualitative examples in Figures 1-2 and Appendix A provide direct evidence that inserting typography into input images can shift generated content, and the release of code and dataset is a practical contribution. However, the primary quantitative metric (CLIP Score) is self-referential, no human validation or error bars are provided, the defense evaluation covers only text-prompt guards, and the FID methodology is underspecified. These issues currently limit the strength of the benchmark claims.","major_comments":[{"comment":"The CLIP Score is computed between the generated image and the inserted typo word using the same CLIP encoder that conditions the attacked models (UnCLIP and IP-Adapter). This makes the metric self-referential: a higher score can arise simply because the generated image retains the rendered typo text, or because the embedding shift is perceptually meaningless. The paper does not provide human judgments, OCR checks, or an independent concept detector, so the deltas in Tables 2 and 3 do not by themselves establish that the model depicts the intended harmful or biased content.","section":"§4.1 (Metrics)"},{"comment":"All reported CLIP Scores are single values without standard deviations, confidence intervals, or significance tests. This is particularly problematic for the invisible typography results, where average deltas are small (0.2-1.0) and some individual entries are negative (e.g., Table 3, FLUX \"naked\" at -0.44, or SD1.5 \"Muslim\" at +0.21). Without per-image statistics or significance testing, the claim of a persistent effect from invisible typography is not supported.","section":"Tables 2-3 (§4.2-§4.4)"},{"comment":"The FID methodology is underspecified. The tables appear to measure FID between generated outputs and their corresponding original clean input images, but the \"clean\" column already yields large FID values (42-58 for the first prompt), indicating that the style-transfer prompt itself causes substantial distribution shift. The incremental FID increase attributed to typography is not shown to correspond to semantic harm (as opposed to general image distortion), and details on sample size, resolution, and the Inception feature layer are missing.","section":"Appendix C (FID)"},{"comment":"The defense evaluation covers only text-prompt guards (Text Blacklist, Detoxify, CLIP-Score-based filtering, LLM-based detection, Latent Guard) and a post-generation NSFW safety checker. It does not test any defense designed for the vision modality, such as image-based typography detection or input image sanitization. Consequently, the conclusion that \"current mainstream guarding methods ... are ineffective\" is broader than the evidence supports.","section":"§4.5 (Defenses)"}],"minor_comments":[{"comment":"The \"invisible\" typography is rendered in RGB (15,15,15) within black borders (0,0,0). Please clarify whether common preprocessing steps such as resizing, cropping, or border removal could eliminate these pixels, and how the models receive the border text in practice.","section":"§3.1"},{"comment":"Reference [47] for CLIP Score appears to be the same paper as reference [46]; the citations should be merged or clearly distinguished.","section":"§4.1"},{"comment":"The notation is inconsistent: ft is used for both image latents and CLIP features, and the reverse transition is written as pθ(ft−1|xt) while the update formula uses ft. Please harmonize the notation throughout the algorithm and the surrounding text.","section":"Algorithm 1"},{"comment":"The safety checker blocks only 16.6% of \"naked\" images on average, so describing it as \"effective\" for harmful content is misleading; the text should be phrased in terms of relative detection rates rather than effectiveness.","section":"Table 4"},{"comment":"Some FID values are anomalously large and inconsistent with the pattern of smaller effects for invisible typography, e.g., Table 8 shows FLUX invisible \"naked\" with an FID increase of 93.25 while visible \"naked\" increases by only 0.04. Please investigate or explain these outliers.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The paper tackles a timely and important problem, and the qualitative evidence is suggestive. The main concern is that the quantitative backbone (CLIP Score) is not independent of the attack mechanism, and the lack of error bars, human evaluation, and FID details undermines the benchmark claims. A revised version with human validation, variance reporting, and a clearer FID description would make the contribution solid. The narrow defense evaluation is also worth revising."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful, well-scoped attack paper. Typographic text in an input image steers CLIP-guided image-to-image diffusion models toward unintended content, and the released dataset is a genuine contribution. The existence claim is backed by visual examples and FID, so I read it as mostly true. But the quantitative backbone is a CLIP Score computed with the same CLIP encoder that conditions generation, so the factor-by-factor and invisible-typography numbers need human validation before they carry the weight the paper puts on them.\n\nWhat's new: prior work showed typographic attacks on CLIP classifiers and LVLMs; this transfers the attack to UnCLIP and IP-Adapter (SD1.5, SDXL, FLUX), adds a systematic study of size/quantity/opacity/position, an invisible variant hidden in black borders, a defense evaluation, and a 37k-image benchmark. The visual examples are persuasive: 'naked' and 'bloody' typography produces NSFW outputs, 'African' shifts demographics, and the 'ignore text' prompt prefix does nothing. The SD3 negative control is a nice touch—VAE-based models are far less affected, which strengthens the CLIP-conditioning story. Citations to prior typographic work are appropriate, and the self-citation to the group's ECCV paper is legitimate since it's the direct predecessor.\n\nSoft spots, in order of importance. First, CLIP Score is self-referential: the attacked models use CLIP, so a score increase partly reflects the model's embedding moving toward the text, not necessarily a human-visible concept. The invisible results in Table 3, with gains of 0.2–1.0, are weak evidence without an OCR check or human rating. Second, no error bars or significance tests anywhere, so we can't separate the small effects from noise. Third, the defense study only tests text-prompt guards (blacklist, Detoxify, CLIP, LLM, Latent Guard) and one NSFW safety checker; it doesn't test vision-input defenses like image filters or adversarial perturbation, so 'current defenses are ineffective' is true only for the tested class. Fourth, FID methodology is lightly described; the appendix numbers have some bizarre FLUX invisible FID jumps (e.g., 68→161) that suggest the metric is unstable at that scale.\n\nThe central argument holds up qualitatively. This deserves a serious referee: the benchmark is useful, the attack is real, and the fixable weaknesses are about evaluation rigor, not a broken core. I'd ask the authors to add human-rated or OCR-validated semantic checks, report variances, and broaden the defense comparison. A conference reviewer should engage; a desk reject would be wrong.","headline":"Solid transfer of typographic attacks to diffusion-based image generation with a useful benchmark, but the quantitative core leans on a self-referential CLIP Score and needs human validation.","tokens_in":24942,"tokens_out":2734,"would_cite":true,"duration_ms":25547,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding printed words to an input image can steer AI image generation toward harmful or biased content, and current text-only defenses miss it.","keywords":["typographic attack","vision modality threat","image-to-image generation","CLIP-guided diffusion models","prompt defense","image generation security","VMT-IGMs dataset"],"falsifier":"Have human annotators label whether images generated from typographic inputs actually contain the target concept, and compare those labels with the CLIP Score deltas; if the deltas rise while the labels show no concept appearing, the claim that typography steers generated content is not supported.","tokens_in":24003,"feed_emoji":"⚠️","tokens_out":7784,"duration_ms":52258,"temperature":0.7,"pith_summary":"This paper tries to establish that image-generation models are vulnerable to typographic attacks through the image input, not just the text prompt. In image-to-image tasks, overlaying a word such as “naked” or “African” on the input image shifts the generated output toward that concept across UnCLIP, SD1.5, SDXL, and FLUX. The paper further argues that existing safety mechanisms—prompt filters, blacklists, latent-space guards, post-generation NSFW checkers, and even an “ignore text” prompt prefix—do not prevent this vision-modality threat. The authors introduce the VMT-IGMs dataset as a baseline for measuring such vulnerabilities. If correct, the finding means real-world image editing pipelines have an uncompensated attack surface.","feed_headline":"Hidden words in a photo can hijack AI image editing","feed_subtitle":"Text-only safety filters miss the attack, so edited images can turn harmful or biased without any toxic prompt.","key_machinery":"The load-bearing mechanism is the typographic attack: overlaying printed text on an input image so that the CLIP vision encoder reads the text as part of the image semantics and encodes it into the conditioning feature $f = \\mathrm{CLIP}(x, p)$ that guides the diffusion reverse process. Because the text and image are fused before generation, the inserted word acts as an additional semantic instruction that the model follows even though it never appears in the user prompt. The quantitative instrument is CLIP Score, computed between the generated image and the inserted typo word; larger positive deltas over clean-input baselines are read as stronger attack success. The evaluation scaffold is the VMT-IGMs dataset, built by overlaying controlled typography on 1000 CelebA-HQ faces, with factor-modification and malicious-threat subtypes.","core_discovery":"The central discovery is that inserting typographic text directly into the input image of an image-to-image task redirects the semantics of the generated image even when the text prompt is entirely benign. Using CLIP Score between the generated image and the inserted word as the measure, the paper reports that visible typography raises alignment by roughly four to seven points on average across models for nouns, with harmful words (“naked”, “bloody”), bias words (“African”, “Asian”), and neutral words (“hat”, “Muslim”) all producing consistent increases. Invisible typography, rendered in near-black on the black border of the input, retains a smaller but still positive effect. The paper also reports that five pre-generation guard methods and the post-generation safety checker fail to flag the attack because they inspect only the language prompt or NSFW content, and that prepending “ignore text” to the prompt does not reduce the typography's influence. The authors conclude that the vision modality is an exposed and currently undefended attack surface for CLIP-guided diffusion models.","pith_inferences":["Because the attack travels through the CLIP image encoder, any downstream model that conditions on CLIP image embeddings, including newer multimodal generators, is a plausible target; this can be tested by repeating the typographic-overlay procedure on those models.","The reported CLIP Score gains may overstate human-perceived semantic change, since the attacked model and the metric share the same encoder; a human-rating study of the generated images would separate embedding alignment from content actually visible to viewers.","A defense direction implied by the mechanism is to sanitize or certify the image input itself—for instance, detecting and removing embedded text before generation—rather than filtering the text prompt.","The same mechanism could be triggered by text already present in ordinary photographs, such as signs, watermarks, or captions, making the vulnerability relevant beyond deliberately crafted attack images."],"forward_implications":["User-supplied images in editing, style-transfer, and conditional-generation applications become a viable vector for injecting unintended, harmful, or biased content into the generated output.","Current text-prompt guards—blacklists, toxicity classifiers, CLIP-score screening, LLM-based checks, latent-space guards, and the post-generation safety checker—cannot catch the attack because the offensive material lives in the image, not the prompt.","Prompt-level mitigations such as instructing the model to “ignore text” do not remove the typography's influence in CLIP-guided diffusion models.","Near-invisible typography still biases the output, so an attacker can alter generated content without visibly changing the source photograph.","VMT-IGMs provides a shared benchmark with both factor-modification and malicious-threat subtypes for future vision-modality defenses."],"supporting_citations":[{"why":"Supplies the CLIP vision and text encoders whose image-text fusion the typographic attack exploits.","marker":"[46]"},{"why":"Provides the CLIP Score metric used to quantify semantic alignment between generated images and inserted typos.","marker":"[47]"},{"why":"The UnCLIP/DALL-E 2 model, one of the CLIP-guided diffusion models tested and shown vulnerable.","marker":"[48]"},{"why":"Provides the IP-Adapter architecture through which SD1.5, SDXL, and FLUX are evaluated under typographic inputs.","marker":"[60]"},{"why":"The DDPM diffusion backbone whose reverse process carries the conditioned semantics into the generated output.","marker":"[20]"},{"why":"Establishes typographic vulnerability in large vision-language models, the precedent this paper extends to image generation.","marker":"[8]"},{"why":"Source of the defense-prefix idea for typographic attacks on CLIP, which the ablation tests as “ignore text.”","marker":"[3]"},{"why":"Provides the post-generation safety checker evaluated and found ineffective against vision-modality typography.","marker":"[49]"},{"why":"A latent-space prompt guard evaluated as a pre-generation defense and found unable to catch the attack.","marker":"[32]"},{"why":"A text toxicity classifier used as a pre-generation prompt defense in the evaluation.","marker":"[17]"}],"fun_headline_variants":["Image text hijacks AI editing despite safe prompts","Hidden photo text overrides AI safety filters","Typographic attack: words in images manipulate AI edits","Invisible text in photos redirects AI image editing","Safe prompt, unsafe edit: image text attack on AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative evidence rests on assuming that a higher CLIP Score between the generated image and the inserted word means the word actually changed the image's meaning, but the attacked models and the metric share the same CLIP encoder and the paper does not validate this with human judgments.","fun_headline_variants_meta":{"raw":{"variants":["Image text hijacks AI editing despite safe prompts","Hidden photo text overrides AI safety filters","Typographic attack: words in images manipulate AI edits","Invisible text in photos redirects AI image editing","Safe prompt, unsafe edit: image text attack on AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000341,"raw_usage":{"total_tokens":1874,"prompt_tokens":935,"completion_tokens":939,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":865}},"tokens_in":551,"tokens_out":939,"duration_ms":8643,"temperature":1.0,"reasoning_tokens":865,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:36:38.323850+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human annotators label whether images generated from typographic inputs actually contain the target concept, and compare those labels with the CLIP Score deltas; if the deltas rise while the labels show no concept appearing, the claim that typography steers generated content is not supported.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP vision and text encoders whose image-text fusion the typographic attack exploits."},{"cited_title":"Un- veiling typographic deceptions: Insights of the typographic vulnerability in large vision-language model.European Con- ference on Computer Vision (ECCV), 2024","cited_arxiv_id":null,"evidence_quote":"Establishes typographic vulnerability in large vision-language models, the precedent this paper extends to image generation."},{"cited_title":"Defense-prefix for pre- venting typographic attacks on clip","cited_arxiv_id":null,"evidence_quote":"Source of the defense-prefix idea for typographic attacks on CLIP, which the ablation tests as “ignore text.”"},{"cited_title":"Latent guard: a safety frame- work for text-to-image generation","cited_arxiv_id":null,"evidence_quote":"A latent-space prompt guard evaluated as a pre-generation defense and found unable to catch the attack."},{"cited_title":"Detoxify","cited_arxiv_id":null,"evidence_quote":"A text toxicity classifier used as a pre-generation prompt defense in the evaluation."}],"review_version":1}