Pith.

Cryptography and Security · arXiv:2610.11112 v1 · submitted 8 Oct 2026 · 117 references

False Claims, Credible Images: A Red-Teaming Benchmark for Commercial Image Generators

Zeyu Ye, Yanchun Li, Sibei He, Meng Xie, Hangtao Zhang, Xianlong Wang, Li Zeng, Jiahao Chen

show 3 more authors

Yichen Wang, Junhui Wang, Ziqi Zhou

The paper says

“Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as news reports and textbook pages.”

more of the abstract

Yet, the same capability introduces a new risk: these models can just as easily fabricate visual misinformation. Even commercial models (e.g., GPT-Image-2) readily produce it. Curiously, we find that these models can recognize a claim as false when asked, yet still render that very claim as credible visual evidence. This discrepancy points to a blind spot in current alignment: safeguards judge what an image shows, not what it asserts; however, existing red-teaming benchmarks target conventional harmful content, such as violent or explicit imagery, and say little about where the alignment boundaries lie for visual misinformation, especially in commercial models. To fill this gap, we introduce EpiReal-Bench, the first systematic benchmark for evaluating visual misinformation risks in commercial image generators, comprising 10k false-claim prompts and 10k corresponding generated images that span 10 real-world claim categories and 10 credible visual formats. We further introduce EpiReal-Attack, a skill-guided black-box optimization framework that uses Pareto-based selection and multimodal feedback to identify commands that bypass alignment safeguards while preserving visual realism, textual legibility, and semantic fidelity. Experiments on four commercial models reveal that more than 70% of false-claim prompts elicit images that faithfully depict the corresponding misinformation, and EpiReal-Attack pushes this rate to 95%. Most worryingly, these models are only a click away, and their outputs are cheap to spread yet hard to disbelieve, leaving this dimension of alignment largely unguarded.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 11 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.

Solid benchmark for a real safety gap, but the headline ASR numbers rest on a single VLM judge without human calibration for Misinformation Realization — that's the one load-bearing weakness in an otherwise well-done paper. This line is wrong

2 major objections · 8 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 117 references say.

The paper says, at §4.1 (MR evaluation) and Appendix A.2

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The paper claims that 'more than 70% of false-claim prompts elicit images that faithfully depict the corresponding misinformation' (abstract) and 'the average ASR across the evaluated models exceeds 70%' (§4.2, Table 3). However, the overall average ASR from Table 3 is (77.01+55.82+89.43+73.20)/4 = 73.87%, which is indeed above 70% but only barely.

more of objection 1

Given that the MR threshold for success is the strictest (score=5), small fluctuations in MR scoring could change whether the average stays above 70%. This reinforces the need for VLM–human calibration: a systematic bias of 0.2 points in MR could move the average below 70%.

The paper says, at §4.5, Fig. 6 (VC human evaluation)

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The human evaluation for VC covers only 1,000 images from a single model (GPT-Image-2). While this provides some evidence that generated images look credible, it does not validate the ASR metric, which couples MR (not VC) with generation success. The authors should clarify why human annotation for MR was not conducted, given that MR is arguably more important to the paper's claims than VC.

8 minor objections

The paper says, at Abstract and §1

Pith says: minor objection 1 of 8

The paper states it is 'the first systematic benchmark for evaluating visual misinformation risks in commercial image generators.' While the benchmark is large and taxonomically broad, related work such as SynCred-Bench (Yang et al., 2026) also evaluates generation-time misinformation with multiple categories and visual forms. The authors should soften this claim or more explicitly differentiate their scope.

The paper says, at §3.4 (Claim Verification and Deduplication)

Pith says: minor objection 2 of 8

The deduplication process uses CLIP text embeddings followed by manual inspection. It would be helpful to report the number of claims removed at each stage and the inter-annotator agreement for the manual review.

show 6 more

The paper says, at Table 3

Pith says: minor objection 3 of 8

The abbreviation 'DP' (documentary photo) and other codes used in the table header should be defined in the caption or in §3.3, where the full names are listed. Currently the reader must cross-reference two pages.

The paper says, at §4.4 (Defenses, Table 4)

Pith says: minor objection 4 of 8

The VGP interception rate for Nano Banana 2 is 93.00%, but the text (§5) describes VGP as 'simple yet effective.' The mechanism of VGP is only sketched; a more precise description of the prompt template and the criteria for interception (beyond MR<5) would aid reproducibility. Appendix C.4 provides the template, but the full evaluation setup for VGP should be referenced earlier.

The paper says, at Fig. 6 and Appendix C.3

Pith says: minor objection 5 of 8

The VC scores from human evaluators are consistently lower than those from most VLMs (e.g., for GPT-Image-2, human: 4.12→4.29; GPT-5.4: 3.74→4.31). This suggests that VLMs may have a different calibration for visual credibility. The authors should briefly discuss whether this gap might indicate that VLMs are not perfect proxies for human credibility judgments even for VC.

The paper says, at §5 (Limitations)

Pith says: minor objection 6 of 8

The limitation paragraph mentions predefined categories and formats, and the lack of downstream-belief measurement, but does not mention the unvalidated MR metric. This should be explicitly acknowledged given its centrality to the results.

The paper says, at Appendix A.2 (MR prompt)

Pith says: minor objection 7 of 8

The MR evaluation prompt states 'The claim is already labeled as false. Your task is only to assess how credible and persuasive the generated image makes the false claim appear.' This explicit labeling could anchor the VLM to seek evidence for the claim rather than critically evaluating. The authors should discuss this potential bias and justify why it is appropriate for their definition of MR.

The paper says, at General

Pith says: minor objection 8 of 8

Several figures (e.g., Fig. 4, Fig. 6) have small font sizes and may be difficult to read in print. The resolution should be increased, and the radar/spider plots in Fig. 6 would benefit from a more distinct legend.

What would settle it

Randomly sample 300 prompt–image pairs from GPT-Image-2 direct-prompting outputs across all claim categories and visual formats. Have three independent human annotators (blinded to generation method and to the GPT-5.4 score) rate each image using the same 1–5 MR rubric (Appendix A.2). Compute inter-annotator agreement (Fleiss' kappa) and GPT-5.4–human agreement (quadratic weighted Cohen's kappa). If kappa ≥ 0.6 for the binary success criterion (MR=5 vs. <5), the concern is partially mitigated. If kappa < 0.4, the ASR figures are likely inflated and should be reported with a caveat or recalculated after human relabeling.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 11 Oct 2026

  1. Reader holds up confidence moderate
  2. Skeptic agrees 25 s · 25,751 tokens read, 2,439 written
  3. Editor wrote the verdict above 13 s · 26,599 tokens read, 1,421 written
  4. Referee major revision: 2 major objections, 8 minor 23 s · 26,054 tokens read, 2,760 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper does two things well. First, it builds EpiReal-Bench, a 10k-prompt benchmark spanning 10 claim categories and 10 visual formats with human-verified false claims. That is a genuine contribution: prior work covered at most two domains and two formats. Second, it documents the verification-generation gap cleanly — models that can label a claim false will still render it as credible visual evidence. The result holds across four commercial models and multiple VLM judges for visual credibility. The observed 70%+ direct ASR and the increase to 95% under attack are consistent and worrying.

The main soft spot is that the misin formation realzation (MR) score — the core of ASR — comes entirely from GPT-5.4 with no human ground-truth comparison for that specific scoring dimension. The paper provides human evaluation for visual credibility (VC) on 1,000 images from one model, but VC and MR measure different things. Without human calibration for MR, we cannot rule out that GPT-5.4 overestimates how faithfully these images support the false claim. The stress-test note is fair on this point. That said, the paper's own limitation section acknowledges the focus on MR and VC rather than downstream effects; it just does not flag the single-judge dependency explicitly. The weakness is proportionate: the results are unlikely to collapse even if MR scores are inflated, because the trend across models and the VC human study support the same conclusion.

This paper is for anyone working on generative AI safety, content moderation, or misinform ation. The benchmark will be useful even if you disagree with the ASR interpretation. I would bring it to a reading group and cite it in related work. It deserves a serious peer review: the methodology is transparent, the claims are bounded, and the finding is important. In review I would ask the authors to either run a human MR study on a representative subset or report a conservative ASR ceiling.

The strongest objection

The central quantitative claim—that >70% direct and >95% adaptive-attack prompts generate images faithfully depicting misinformation—depends entirely on the Misinformation Realization (MR) score assigned by GPT-5.4 (Section 4.1, Appendix A.2). The MR prompt explicitly tells the judge the claim is false and asks to assess how 'convincingly' the image supports it. This framing could inflate scores by directing attention to surface plausibility rather than actual claim–image alignment. Critically, the paper provides human evaluation for Visual Credibility (VC) (Section 4.5, 1,000 images, one model) but not for MR. VC and MR are fundamentally different: VC judges realism, while MR requires verifying that a specific false proposition is accurately depicted. Without human calibration for MR, we cannot rule out systematic overestimation. The reported ASR numbers—the paper's headline result—therefore rest on an unvalidated single-VLM measurement. The paper's own limitation statement (Section 5) does not mention this dependency, stating only that it 'primarily evaluates misinformation realization and visual credibility rather than downstream effects.' The lack of ground-truth validation for the core metric is the most load-bearing weakness.

The reader correctly identifies GPT-5.4's role as sole MR judge as the weakest assumption. I agree that this is the most load-bearing concern because every quantitative headline—the 70%+ direct ASR, the 95%+ attack ASR, the verification-generation gap quantification—depends on MR being measured correctly. The paper provides strong independent support for other claims: the benchmark construction is thorough with human verification of claims, the VC human evaluation shows VLM–human alignment for a related but distinct dimension, and the mitigation VGP is a concrete proposal. However, none of this substitutes for human validation of MR. The concrete test I propose is minimal, feasible, and directly addresses whether the central metric is trustworthy. If the test finds low agreement, the central numbers weaken; if it finds high agreement, the main results are robust. I keep the verdict unchanged because the paper is well-executed overall and acknowledges broader scope limitations, but the confidence remains moderate as the reader stated. No ad hominem, no theatrical language; the critique is on the measurement of the key metric.

The referee's report

If the reported ASR numbers are reliable, this work exposes a consequential blind spot in current alignment strategies for image-generation models. The benchmark is carefully constructed (human-verified claims, multi-stage quality selection, risk annotations) and the attack framework is principled (skill-guided evolution, Pareto selection). The finding that models exhibit a verification-generation gap is novel and actionable. The dataset and code are released, supporting reproducibility and follow-up work. However, the significance hinges on the validity of the core metric—Misinformation Realization (MR)—which determines ASR and drives the headline claims.

Pith's reading of the paper (a machine's summary, not the authors')

This paper identifies a 'verification-generation gap' in commercial image-generation models: models can correctly label a claim as misinformation when asked, yet still render that same false claim as a convincingly realistic image resembling a newspaper, broadcast, or official document. To measure this risk, the authors introduce EPIREAL-BENCH, a benchmark of 10,000 false-claim prompts spanning 10 real-world categories and 10 credible visual formats, with corresponding curated images. They also develop EPIREAL-ATTACK, a black-box prompt optimization framework that uses Pareto-based selection to push the attack success rate above 95% across four leading commercial models. The paper further shows that existing input- and output-level defenses intercept such images at low rates, while a simple verification-before-generation prompt (VGP) raises interception substantially on most models. The central claim is that visual misinformation is a systematic and previously overlooked alignment failure, because current safeguards judge what an image shows, not what factual claim the image asserts.

What carries the argument. The paper is built on two main tools. EPIREAL-BENCH is a curated benchmark of 1,000 verified false claims, each instantiated across 10 credible visual formats (e.g., newspaper page, broadcast news, social platform post) to yield 10,000 prompt-image pairs with risk rationales. EPIREAL-ATTACK is a skill-guided black-box optimization that applies four prompt transformation operators (claim reframing, contextual enrichment, artifact specification, presentation refinement) and uses Pareto-based selection with multimodal feedback to iteratively refine prompts that bypass safeguards while preserving visual realism and semantic fidelity. The core metric is Attack Success Rate (ASR), defined as the proportion of prompts that produce an image achieving a perfect Misinformation Realization score of 5 (the strictest criterion).

Core claim. The paper discovers a verification-generation gap: commercial image-generation models (GPT-Image-2, Nano Banana 2, Grok Imagen 2.0, Seedream 5.0 Flash) can recognize false claims as misinformation under explicit factual verification, yet still generate visually credible images that embody the same false claims. Under direct prompting, more than 70% of false-claim prompts produce images that faithfully depict the misinformation; an adaptive attack (EPIREAL-ATTACK) raises that rate to over 95%. This reveals that current alignment mechanisms do not prevent the generation of visually benign but factually false evidence, constituting a blind spot in model safety.

The premise it rests on. The central quantitative claim (ASR >70% direct, >95% under attack) depends on the reliability of GPT-5.4 as the sole judge for Misinformation Realization (MR) scores; if GPT-5.4 systematically overestimates how faithfully images convey false claims, the reported attack success rates could be inflated, and the human evaluation provided covers only visual credibility, not MR.

If the paper is right (Pith's inferences, not the authors')

If these results hold, commercial image generators pose a significant channel for spreading visual misinformation at low cost, as false claims can be packaged as authoritative-looking evidence without adversarial expertise.

Current safety defenses—keyword filters, NSFW classifiers, image checkers—are largely ineffective against this class of misinformation, suggesting that alignment efforts must shift from content-based filtering to claim-level verification.

The verification-generation gap implies that factual awareness learned during training does not automatically transfer to generation-time behavior, pointing to a need for training objectives that align recognition and generation.

The paper's VGP mitigation, which prepends a factual verification instruction, shows that explicit reasoning prompts can partially close the gap without retraining, offering a low-cost interim defense.

The variation in ASR across claim categories (economy most vulnerable, education least) indicates that safeguard behavior is domain-dependent, which could inform targeted hardening.

The reliance on a single VLM (GPT-5.4) as the sole judge of Misinformation Realization (the strict criterion for ASR) means the headline 70–95% numbers hinge on how well that evaluator corresponds to human judgment. A human-study replication of the MR scoring—especially for borderline cases—would be a natural stress test.

The paper's taxonomy of 10 credible visual formats is limited to document-like artifacts. Future misinformation could adopt emerging formats (e.g., AI-generated video, deepfake voice clips, interactive web pages), which may present different alignment challenges.

The VGP mitigation, while effective on GPT-Image-2 and Nano Banana 2, performs poorly on Grok Imagen 2.0 (35.7% interception), suggesting that some models lack the internal factual knowledge or compliance to follow verification instructions. This implies that VGP's efficacy depends on model-specific capabilities, not just the prompt design.

If EPIREAL-ATTACK generalizes to other models and domains, the paper provides a methodology for red-teaming that could be adopted by safety evaluators. The Pareto-based multi-objective optimization is a key contribution that may be applicable beyond image generation to other generative modalities.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as news reports and textbook pages. Yet, the same capability introduces a new risk: these models can just as easily fabricate visual misinformation. Even commercial models (e.g., GPT-Image-2) readily produce it. Curiously, we find that these models can recognize a claim as false when asked, yet still render that very claim as credible visual evidence. This discrepancy points to a blind spot in current alignment: safeguards judge what an image shows, not what it asserts; however, existing red-teaming benchmarks target conventional harmful content, such as violent or explicit imagery, and say little about where the alignment boundaries lie for visual misinformation, especially in commercial models. To fill this gap, we introduce EpiReal-Bench, the first systematic benchmark for evaluating visual misinformation risks in commercial image generators, comprising 10k false-claim prompts and 10k corresponding generated images that span 10 real-world claim categories and 10 credible visual formats. We further introduce EpiReal-Attack, a skill-guided black-box optimization framework that uses Pareto-based selection and multimodal feedback to identify commands that bypass alignment safeguards while preserving visual realism, textual legibility, and semantic fidelity. Experiments on four commercial models reveal that more than 70% of false-claim prompts elicit images that faithfully depict the corresponding misinformation, and EpiReal-Attack pushes this rate to 95%. Most worryingly, these models are only a click away, and their outputs are cheap to spread yet hard to disbelieve, leaving this dimension of alignment largely unguarded.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-09

The authors' main claim, as the classifier picked it out 0.77

“Most worryingly, these models are only a click away, and their outputs are cheap to spread yet hard to disbelieve, leaving this dimension of alignment largely unguarded.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.94 · states limitation 0.21 · human participants 0.07 · controlled comparison 0.21 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.68 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as news reports and textbook pages. background 1.00 0.06 0.03 asserted 0.94
Yet, the same capability introduces a new risk: these models can just as easily fabricate visual misinformation. background 0.98 0.08 0.09 asserted 0.69
Even commercial models (e.g., GPT-Image-2) readily produce it. result 0.51 0.41 0.05 asserted 1.00
Curiously, we find that these models can recognize a claim as false when asked, yet still render that very claim as credible visual evidence. result 0.95 0.91 0.24 asserted 1.00
This discrepancy points to a blind spot in current alignment: safeguards judge what an image shows, not what it asserts; however, existing red-teaming benchmarks target conventional harmful content, such as violent or explicit imagery, and say little about where the alignment boundaries lie for visual misinformation, especially in commercial models. gap 0.98 0.26 0.18 asserted 0.89
To fill this gap, we introduce EpiReal-Bench, the first systematic benchmark for evaluating visual misinformation risks in commercial image generators, comprising 10k false-claim prompts and 10k corresponding generated images that span 10 real-world claim categories and 10 credible visual formats. contribution 0.97 0.96 0.09 asserted 1.00
We further introduce EpiReal-Attack, a skill-guided black-box optimization framework that uses Pareto-based selection and multimodal feedback to identify commands that bypass alignment safeguards while preserving visual realism, textual legibility, and semantic fidelity. method 0.85 0.93 0.05 asserted 0.99
Experiments on four commercial models reveal that more than 70% of false-claim prompts elicit images that faithfully depict the corresponding misinformation, and EpiReal-Attack pushes this rate to 95%. result 1.00 0.96 0.38 asserted 1.00
Most worryingly, these models are only a click away, and their outputs are cheap to spread yet hard to disbelieve, leaving this dimension of alignment largely unguarded. interpretation 1.00 0.26 0.51 asserted 1.00

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-09: 1 request, 7,159 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (19)
Figure 1 of the paper
Figure 1. Overview of EPIREAL-BENCH. ABSTRACT Image-generation models can now produce text-rich, natural-looking visual arti￾facts that are hard to distinguish from real-world evidence, such as news reports and textbook pages. Yet, the same capability introduces a new risk: these mod￾els can just as easily fabricate visual misinformation. Even commercial models (e.g., GPT-Image-2) readily produce it. Curiously, we find that these models can recognize a claim as false when asked, yet still render that very claim as credible visual evidence. This discrepancy points to a blind spot in current…
Figure 2 of the paper
Figure 2. Misalignment between misinfor￾mation recognition and safeguards. Our journey begins with a surprising observation. As shown in
Figure 3 of the paper
Figure 3. Comparison between traditional safety benchmarks and
Figure 4 of the paper
Figure 4. FCR comparison with ASR of four commercial image-generation models. We begin with a fundamental alignment question: “Can commercial image-generation models explicitly recog￾nize the false claims? ” To investigate this question, we measure the False Claim Recognition Rate (FCR), de￾fined as the proportion of claims in EPIREAL-BENCH that a model explicitly identifies as misinformation un￾der verification (detailed in Appendix A), and compare it with the ASR defined in Sec. 4.1. As an additional sanity check, we evaluate the same models on benign factual prompts. All evaluated models…
Figure 5 of the paper
Figure 5. Failed generations and successful generations after applying E
Figure 6 of the paper
Figure 6. VC scores of GPT-Image-2 generations across 10 visually credible formats, evaluated by
Figure 7 of the paper
Figure 7. VC scores of Nano Banana 2 generations across 10 visually credible formats, evaluated by
Figure 8 of the paper
Figure 8. VC scores of Grok Imagen 2.0 generations across 10 visually credible formats, evaluated
Figure 9 of the paper
Figure 9. VC scores of Seedream 5.0 Flash generations across 10 visually credible formats, evaluated
Figure 10 of the paper
Figure 10. A government false claim presented as a newspaper page
Figure 11 of the paper
Figure 11. A military false claim presented as a social platform post
Figure 12 of the paper
Figure 12. An election false claim presented as a broadcast news screen
Figure 13 of the paper
Figure 13. A disaster false claim presented as a desktop platform screenshot
Figure 14 of the paper
Figure 14. A crime false claim presented as a documentary-style photograph
Figure 15 of the paper
Figure 15. A health false claim presented as an official red-headed administrative document
Figure 16 of the paper
Figure 16. A science false claim presented as a smartphone mobile alert
Figure 17 of the paper
Figure 17. An economy false claim presented as a public bulletin notice
Figure 18 of the paper
Figure 18. An educational false claim presented as an educational textbook page
Figure 19 of the paper
Figure 19. An environment false claim presented as an instant messaging chat record

Figures are the authors', reproduced from the paper.

References (117)
  1. [1] 2026 , howpublished = on Pith
  2. [2] Fortin, Alisa and Raisinghani, Naina , title =. 2025 , howpublished = on Pith
  3. [3] Zhao, Bing and Wu, Chenfei and Li, Deqing and Meng, Hao and Li, Jiahao and Zhang, Jie and Zhou, Jingren and Lin, Junyang and Gao, Kaiyuan and Cao, Kuan and others , journal = on Pith
  4. [4] 2026 , month = jul, howpublished = on Pith
  5. [5] Beyond Generation, It Understands Design | Introducing Seedream 5.0 Pro , year = on Pith
  6. [6] Imagine Image 2.0 , year = on Pith
  7. [7] HunyuanImage 3.0 Technical Report , author=. arXiv preprint arXiv:2509.23951 , year= on PithPith's reviewarXiv
  8. [8] TextDiffuser: Diffusion Models as Text Painters , author=. Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS'23) , volume= on Pith
  9. [9] TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering , author=. Proceedings of the 18th European Conference on Computer Vision (ECCV'24) , pages= on Pith
  10. [11] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models , author=. Proceedings of the 36th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'23) , pages= on Pith
  11. [12] Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models , author=. Proceedings of the 30th ACM SIGSAC Conference on Computer and Communications Security (CCS'23) , pages= on Pith
  12. [13] T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation , author=. Proceedings of the 38th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'25) , pages= on Pith
  13. [14] T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model , author=. Proceedings of the 40th AAAI Conference on Artificial Intelligence (AAAI'26) , pages= on Pith
  14. [15] ART: Automatic Red-Teaming for Text-to-Image Models to Protect Benign Users , author=. Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS'24) , volume= on Pith
  15. [16] Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation , author=. Proceedings of the 7th ACM Conference on Fairness, Accountability, and Transparency (FAccT'24) , pages= on Pith
  16. [17] Multimodal Pragmatic Jailbreak on Text-to-image Models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL'25) , pages= on Pith
  17. [18] Nonprobative Photographs (or Words) Inflate Truthiness , author=. Psychonomic Bulletin & Review , volume=. 2012 , doi= on Pith
  18. [19] Pictures and Repeated Exposure Increase Perceived Accuracy of News Headlines , author=. Applied Cognitive Psychology , volume=. 2020 , doi= on Pith
  19. [20] Misinformed by Images: How Images Influence Perceptions of Truth and What Can Be Done About It , author=. Current Opinion in Psychology , volume=. 2024 , doi= on Pith
  20. [21] The Visual Counter Turing Test (VCT ^2 ): A Benchmark for Evaluating AI-Generated Image Detection and the Visual AI Index (V\_AI) , author=. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter... on Pith
show 80 more
  1. [23] TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering , author=. Proceedings of the 19th IEEE/CVF International Conference on Computer Vision (ICCV'23) , pages= on Pith
  2. [24] VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL'24) , pages= on Pith
  3. [25] MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? , author=. Proceedings of the 39th Conference on Neural Information Processing Systems (NeurIPS'25) , volume= on Pith
  4. [26] GPT-4o System Card , author=. 2024 , url= on Pith
  5. [27] Learning Transferable Visual Models From Natural Language Supervision , author=. Proceedings of the 38th International Conference on Machine Learning (ICML'21) , pages= on Pith
  6. [28] DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , author=. arXiv preprint arXiv:2606.19348 , year= on PitharXiv
  7. [29] Evaluation of Generative Artificial Intelligence Safeguards Against the Creation of Images and Videos Harmful to Public Health , author=. Public Health Reports , volume=. 2026 , doi= on Pith
  8. [30] Choi, Wonwoo and Seo, Minjae and Song, Minkyoo and Heo, Hwanjo and Shin, Seungwon and You, Myoungsung , journal= on Pith
  9. [31] M3A: A Multimodal Misinformation Dataset for Media Authenticity Analysis , author=. Computer Vision and Image Understanding , volume=. 2024 , publisher= on Pith
  10. [32] MiRAGeNews: Multimodal Realistic AI-Generated News Detection , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages= on Pith
  11. [33] Decoding Synthetic News: An Interpretable Multimodal Framework for the Classification of News Articles in a Novel News Corpus , author=. Artificial Intelligence Review , volume= on Pith
  12. [34] From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation , author=. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages= on Pith
  13. [36] Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models , author=. Proceedings of the 14th International Conference on Learning Representations (ICLR'26) , year= on Pith
  14. [37] TextRich: A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2 , author=. arXiv preprint arXiv:2606.19259 , year= on PithPith's reviewarXiv
  15. [38] AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents , author=. arXiv preprint arXiv:2602.20569 , year= on PithPith's reviewarXiv
  16. [39] When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents , author=. arXiv preprint arXiv:2604.25213 , year= on PithPith's reviewarXiv
  17. [40] DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis , author=. arXiv preprint arXiv:2603.01433 , year= on PithPith's reviewarXiv
  18. [41] The CIFAR Synthetic Evidence Corpus for Detecting AI-Generated Evidence , author=. arXiv preprint arXiv:2606.07916 , year= on PithPith's reviewarXiv
  19. [42] MMA-Diffusion: MultiModal Attack on Diffusion Models , author=. Proceedings of the 37th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'24) , pages= on Pith
  20. [43] JailbreakDiffBench: A Comprehensive Benchmark for Jailbreaking Diffusion Models , author=. Proceedings of the 20th IEEE/CVF International Conference on Computer Vision (ICCV'25) , pages= on Pith
  21. [44] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning , author=. Proceedings of the 14th International Conference on Learning Representations (ICLR'26) , year= on Pith
  22. [45] Illuminating Search Spaces by Mapping Elites , author=. arXiv preprint arXiv:1504.04909 , year= on PithPith's reviewarXiv
  23. [46] Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? , author=. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT'22) , pages=. 2022 , publisher= on Pith
  24. [47] 2024 , howpublished = on Pith
  25. [48] 2020 , howpublished = on Pith
  26. [49] Advclip: Downstream-agnostic adversarial examples in multimodal contrastive learning , author=. Proceedings of the 31st ACM International Conference on Multimedia , pages= on Pith
  27. [50] PSFD: Proactive Spatial-Frequency Defense against Malicious Exemplar-Guided Image Editing , author=. 2025 IEEE International Conference on Multimedia and Expo (ICME) , pages=. 2025 , organization= on Pith
  28. [52] Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics , author=. 35th USENIX Security Symposium (USENIX Security 26) , pages= on Pith
  29. [53] Fine-grained poisoning framework against federated learning , author=. IEEE Transactions on Dependable and Secure Computing , year= on Pith
  30. [54] Segtrans: Transferable adversarial examples for segmentation models , author=. IEEE Transactions on Multimedia , year= on Pith
  31. [55] Darkhash: A data-free backdoor attack against deep hashing , author=. IEEE Transactions on Information Forensics and Security , year= on Pith
  32. [56] Advedm: Fine-grained adversarial attack against vlm-based embodied agents , author=. Advances in Neural Information Processing Systems , volume= on Pith
  33. [57] Breaking barriers in physical-world adversarial examples: Improving robustness and transferability via robust feature , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume= on Pith
  34. [58] Unlearnable 3d point clouds: Class-wise transformation is all you need , author=. Advances in Neural Information Processing Systems , volume= on Pith
  35. [59] Pb-uap: Hybride universal adversarial attack for image segmentation , author=. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2025 , organization= on Pith
  36. [60] Reverse backdoor distillation: Towards online backdoor attack detection for deep neural network models , author=. IEEE Transactions on Dependable and Secure Computing , volume=. 2024 , publisher= on Pith
  37. [61] Test-time backdoor detection for object detection models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages= on Pith
  38. [62] Denial-of-Service or Fine-Grained Control: Towards Flexible Model Poisoning Attacks on Federated Learning , author=. Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence , pages= on Pith
  39. [63] Detector Collapse: Backdooring Object Detection to Catastrophic Overload or Blindness in the Physical World , author=. Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages= on Pith
  40. [64] Trojanrobot: Physical-world backdoor attacks against vlm-based robotic manipulation , author=. arXiv preprint arXiv:2411.11683 , year= on PitharXiv
  41. [65] BadRobot: Jailbreaking Embodied LLM Agents in the Physical World , author=. Proceedings of the Thirteenth International Conference on Learning Representations , year= on Pith
  42. [67] GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models , author=. Proceedings of the 34th ACM International Conference on Multimedia , year= on Pith
  43. [68] PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis , author=. Proceedings of the 34th ACM International Conference on Multimedia , year= on Pith
  44. [70] ByteDance Seed Team . Seedream 5.0 Flash . https://docs.volcengine.com/docs/ark/seedream-5-0-pro, 2026. ByteDance Seedream API documentation. Accessed: 2026-09-16 on Pith
  45. [71] Zhaorun Chen, Zichen Wen, Yichao Du, Yiyang Zhou, Chenhang Cui, Siwei Han, Jen Weng, Chaoqi Wang, Zhengwei Tong, Leria Huang, Canyu Chen, Haoqin Tu, Qinghao Ye, Zhihong Zhu, Yuqing Zhang, Jiawei Zhou, Zhuokai Zhao, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Mj-bench: Is yo... on Pith
  46. [72] Wonwoo Choi, Minjae Seo, Minkyoo Song, Hwanjo Heo, Seungwon Shin, and Myoungsung You. PC ^2 : Politically controversial content generation via jailbreaking attacks on gpt-based text-to-image models. arXiv preprint arXiv:2601.05150, 2026 on PithPith's reviewarXiv
  47. [73] Bianca Chu, Natansh D. Modi, Bradley D. Menz, Erik Cornelisse, Stephen Bacchi, Norma Bulamu, Shahid Ullah, Ross A. McKinnon, Kacper Gradon, Andrew Rowland, Michael J. Sorich, and Ashley M. Hopkins. Evaluation of generative artificial intelligence safeguards against the creatio... on Pithdoi:10.1177/00333549261418596
  48. [74] DeepSeek-AI . Deepseek-v4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348, 2026. doi:10.48550/arXiv.2606.19348 on Pithdoi:10.48550/arxiv.2606.19348
  49. [75] Alisa Fortin and Naina Raisinghani. Build with nano banana 2, our gemini 3 pro image model. https://blog.google/innovation-and-ai/technology/ai/nano-banana-2/, 2025. Google Blog. Accessed: 2026-09-01 on Pith
  50. [76] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the 19th IEEE/CVF International Conference on Computer Vision (... on Pith
  51. [77] Runsheng Huang, Liam Dugan, Yue Yang, and Chris Callison-Burch. Miragenews: Multimodal realistic ai-generated news detection. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp.\ 16436--16448, 2024 on Pith
  52. [78] Hugging Face . Stable Diffusion Safety Checker . https://huggingface.co/CompVis/stable-diffusion-safety-checker, 2024. Hugging Face model repository. Accessed: 2026-09-01 on Pith
  53. [79] Jaided AI . EasyOCR : Ready-to-use optical character recognition. https://github.com/JaidedAI/EasyOCR, 2020. Software repository on Pith
  54. [80] Xiaolong Jin, Zixuan Weng, Hanxi Guo, Chenlong Yin, Siyuan Cheng, Guangyu Shen, and Xiangyu Zhang. Jailbreakdiffbench: A comprehensive benchmark for jailbreaking diffusion models. In Proceedings of the 20th IEEE/CVF International Conference on Computer Vision (ICCV'25), pp.\ 1... on Pith
  55. [81] Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL'24), pp.\ 12268--12290, 2024 on Pith
  56. [82] LAION-AI . CLIP-based-NSFW-Detector . https://github.com/LAION-AI/CLIP-based-NSFW-Detector, 2024. GitHub repository. Accessed: 2026-09-01 on Pith
  57. [83] Guanlin Li, Kangjie Chen, Shudong Zhang, Jie Zhang, and Tianwei Zhang. Art: Automatic red-teaming for text-to-image models to protect benign users. In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS'24), volume 37, pp.\ 91184--91219, 2024 on Pith
  58. [84] Haiyang Li, Yaxiong Wang, Lianwei Wu, Lechao Cheng, and Zhun Zhong. Towards unified multimodal misinformation detection in social media: A benchmark dataset and baseline. arXiv preprint arXiv:2509.25991, 2025 a on PithPith's reviewarXiv
  59. [85] Lijun Li, Zhelun Shi, Xuhao Hu, Bowen Dong, Yiran Qin, Xihui Liu, Lu Sheng, and Jing Shao. T2isafety: Benchmark for assessing fairness, toxicity, and privacy in image generation. In Proceedings of the 38th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'25... on Pith
  60. [86] Minghui Li, Hangtao Zhang, Yanjun Zhang, Li Zeng, Chao Chen, Qiyun Shao, Wei Wan, Shengshan Hu, and Leo Yu Zhang. Fine-grained poisoning framework against federated learning. IEEE Transactions on Dependable and Secure Computing, 2025 c on Pith
  61. [87] Tong Liu, Zhixin Lai, Jiawen Wang, Gengyuan Zhang, Shuo Chen, Philip Torr, Vera Demberg, Volker Tresp, and Jindong Gu. Multimodal pragmatic jailbreak on text-to-image models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL'25), p... on Pith
  62. [88] Eryn J. Newman and Norbert Schwarz. Misinformed by images: How images influence perceptions of truth and what can be done about it. Current Opinion in Psychology, 56: 0 101778, 2024. doi:10.1016/j.copsyc.2023.101778 on Pithdoi:10.1016/j.copsyc.2023.101778
  63. [89] Eryn J. Newman, Maryanne Garry, Daniel M. Bernstein, Justin Kantner, and D. Stephen Lindsay. Nonprobative photographs (or words) inflate truthiness. Psychonomic Bulletin & Review, 19 0 (5): 0 969--974, 2012. doi:10.3758/s13423-012-0292-0 on Pithdoi:10.3758/s13423-012-0292-0
  64. [90] OpenAI . GPT-5.4 Model . https://developers.openai.com/api/docs/models/gpt-5.4, 2026 a . OpenAI API model documentation. Accessed: 2026-09-01 on Pith
  65. [91] OpenAI . GPT Image 2 Model . https://developers.openai.com/api/docs/models/gpt-image-2, 2026 b . OpenAI API model documentation. Accessed: 2026-09-01 on Pith
  66. [92] Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models. In Proceedings of the 30th ACM SIGSAC Conference on Computer and Communications Security (CCS'... on Pith
  67. [93] Jessica Quaye, Alicia Parrish, Oana Inel, Charvi Rastogi, Hannah Rose Kirk, Minsuk Kahng, Erin Van Liemt, Max Bartolo, Jess Tsang, Justin White, Nathan Clement, Rafael Mosquera, Juan Ciro, Vijay Janapa Reddi, and Lora Aroyo. Adversarial nibbler: An open red-teaming method for ... on Pith
  68. [94] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th... on Pith
  69. [95] Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and in turn reflecting on inappropriate content? In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT'22), pp.... on Pith
  70. [96] Patrick Schramowski, Manuel Brack, Bj \"o rn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the 36th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'23), pp.\ 22522--22531, 2023 on Pith
  71. [97] Thomas J. Smelter and Dustin P. Calvillo. Pictures and repeated exposure increase perceived accuracy of news headlines. Applied Cognitive Psychology, 34 0 (5): 0 1061--1071, 2020. doi:10.1002/acp.3684 on Pithdoi:10.1002/acp.3684
  72. [98] Yufei Song, Ziqi Zhou, Minghui Li, Xianlong Wang, Hangtao Zhang, Menghao Deng, Wei Wan, Shengshan Hu, and Leo Yu Zhang. Pb-uap: Hybride universal adversarial attack for image segmentation. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Proces... on Pith
  73. [99] Yufei Song, Ziqi Zhou, Qi Lu, Hangtao Zhang, Yifan Hu, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. Segtrans: Transferable adversarial examples for segmentation models. IEEE Transactions on Multimedia, 2026 on Pith
  74. [100] Junhui Wang, Hangtao Zhang, Zhirun Zheng, Li Zeng, Jiejun Xiao, Xi Luo, Lihua Yin, and Saiqin Long. Pvdetector: Detecting prompt injection attacks on purpose-specific llm agents through policy-violation concept analysis. In Proceedings of the 34th ACM International Conference ... on Pith
  75. [101] Xianlong Wang, Minghui Li, Wei Liu, Hangtao Zhang, Shengshan Hu, Yechao Zhang, Ziqi Zhou, and Hai Jin. Unlearnable 3d point clouds: Class-wise transformation is all you need. Advances in Neural Information Processing Systems, 37: 0 99404--99432, 2024 on Pith
  76. [102] Xianlong Wang, Wenbo Pan, Shijia Zhou, Ke Li, Yuqi Wang, Zeyu Ye, Hangtao Zhang, Leo Yu Zhang, and Xiaohua Jia. Image-to-video diffusion: From foundations to open frontiers. arXiv preprint arXiv:2605.17248, 2026 b on PithPith's reviewarXiv
  77. [103] Xianlong Wang, Hangtao Zhang, Wenbo Pan, Ziqi Zhou, Changsong Jiang, Li Zeng, and Xiaohua Jia. Dual-branch robust unlearnable examples. arXiv preprint arXiv:2605.01718, 2026 c on PithPith's reviewarXiv
  78. [104] Yichen Wang, Yuxuan Chou, Ziqi Zhou, Hangtao Zhang, Wei Wan, Shengshan Hu, and Minghui Li. Breaking barriers in physical-world adversarial examples: Improving robustness and transferability via robust feature. In Proceedings of the AAAI Conference on Artificial Intelligence, v... on Pith
  79. [105] Yichen Wang, Hangtao Zhang, Hewen Pan, Ziqi Zhou, Xianlong Wang, Peijin Guo, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. Advedm: Fine-grained adversarial attack against vlm-based embodied agents. Advances in Neural Information Processing Systems, 38: 0 136551--136575, 2026 d on Pith
  80. [106] Jiaying Wu, Fanxiao Li, Zihang Fu, Min-Yen Kan, and Bryan Hooi. Seeing through deception: Uncovering misleading creator intent in multimodal news with vision-language models. In Proceedings of the 14th International Conference on Learning Representations (ICLR'26), 2026 on Pith

The first 100 of 117 references are listed here. All of them are in this paper's citation record.

Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment