Pith. sign in

REVIEW 3 major objections 5 minor 34 references

A multilingual obfuscation attack can make GPT-based text-to-image systems generate politically controversial images of real public figures, bypassing safety filters up to 86% of the time.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:44 UTC pith:44EGEPM5

load-bearing objection The attack idea is genuinely new and the paper deserves a referee, but its headline 86% ASR counts any non-refused image as a success, so the actual politically-content generation rate is likely lower. the 3 major comments →

arxiv 2601.05150 v3 pith:44EGEPM5 submitted 2026-01-08 cs.CR

PC²: Politically Controversial Content Generation via Jailbreaking Attacks on GPT-based Text-to-Image Models

Wonwoo Choi , Minjae Seo , Minkyoo Song , Hwanjo Heo , Seungwon Shin , Myoungsung You This is my paper
classification cs.CR
keywords text-to-image jailbreakpolitically sensitive contentmultilingual promptssafety filtersidentity-preserving descriptive mappinggeopolitical obfuscationGPT image generationadversarial prompting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that commercial text-to-image systems that refuse explicitly political prompts can be induced to generate politically sensitive images of real public figures by rewriting the prompt in a clever multilingual form. The authors propose a two-step transformation: replace names and symbols with neutral but identifying descriptions, then translate each piece into a geopolitically distant language. They argue that this prevents safety filters from seeing the toxic relationship between the person and the action, while still giving the image generator enough cues to depict the intended figure. If correct, the work exposes a practical vulnerability in current AI image services and demonstrates that political disinformation imagery can be produced at scale.

Core claim

The central claim is that the safety filters of GPT-based text-to-image models evaluate political sensitivity through linguistic context, so splitting a politically sensitive prompt into fragments expressed in different, geopolitically distant languages makes the filter unable to relate those fragments into a toxic scenario. The authors demonstrate this with PC^2, which first maps each sensitive keyword (a public figure or a symbolic object) to a neutral but identity-preserving description, then selects a language for each description based on a composite geopolitical sensitivity score that balances semantic preservation with reduced political association. On a benchmark of 240 prompts invol

What carries the argument

The load-bearing mechanism is the combination of Identity-Preserving Descriptive Mapping (IPDM) and Geopolitically Distal Translation. IPDM turns exact names (e.g., Donald Trump) into 1-2 sentence quiz-style descriptions that carry enough visual and historical cues for the generator to depict the person, while avoiding keyword blocklists. The descriptions are then translated into a pool of 72 languages, and a set of four embedding-based metrics (keyword common-knowledge, country common-knowledge, bias, and politics) are combined into a single score that ranks languages by how well they fragment political semantics while retaining enough meaning for generation. The final prompt uses an indexe

Load-bearing premise

The central assumption is that 'attack success' — defined as the system returning an image instead of a refusal — actually means the image depicts the intended public figure in the intended politically controversial scenario; the paper does not verify identity or action preservation per generated image.

What would settle it

Manually inspect or automatically classify a random sample of the images produced by PC^2 prompts across the three GPT interfaces, measuring how many clearly show the target public figure in the target controversial action (e.g., via human raters or a vision-language model that checks both identity and action). If the fraction of such 'true positive' images is far below 86% (because many outputs are irrelevant, anonymized, or altered), then the paper's central security claim is not established.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Commercial text-to-image systems that rely on prompt-side filtering can be induced to generate images of real current officeholders in fabricated, controversial situations, despite all original prompts being blocked.
  • The attack transfers across GPT-based interfaces (GPT-4o, GPT-5, GPT-5.1) with attack success rates of 86%, 68%, and 76%, respectively, and stays effective across object-based and phrase-based prompt types.
  • A system-prompt-level safety reminder fully blocks the attack but at the cost of wrongly rejecting 22–36 of 36 benign political prompts, revealing a trade-off between safety coverage and usability.
  • The attack's success does not hinge on low-resource languages alone; even high-fidelity translations (e.g., Danish, Hungarian) appear frequently in successful prompts, so defenses must address relational reasoning across languages rather than just language coverage.
  • A multi-layered filtering defense (relevant-language alignment plus system prompting) reduces ASR to about 10%, but language alignment alone is insufficient to fully mitigate the attack.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • I infer that the same multilingual fragmentation technique could be adapted to other safety categories that are relational, such as incitement to violence against specific groups or defamation, where the harm depends on combining two entities.
  • If ASR is measured as non-refusal only, the true rate of 'politically sensitive image actually generated' may be lower; the paper's own percentile analysis shows that low semantic similarity leads to irrelevant images, so the 86% figure is an upper bound on meaningful attack success.
  • I would expect the vulnerability to be broader than GPT models: any T2I system that uses an LLM-based pre-filter and supports multilingual input could be susceptible, though the paper only evaluates GPT interfaces.
  • The metric-guided language selection could be repurposed as a diagnostic tool for safety filters, identifying which language pairs and contextual juxtapositions are most likely to blind a given filter.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PC^2, the first black-box framework for jailbreaking political safety filters in text-to-image (T2I) models. It uses Identity-Preserving Descriptive Mapping (IPDM) to replace sensitive political entities with neutral, identity-preserving descriptions, and Geopolitically Distal Translation to express these descriptions in a mixture of low-resource, geopolitically distant languages, with a metric-guided selection of languages. The authors construct a benchmark of 240 politically sensitive prompts involving 36 public figures. Evaluating on GPT-4o, GPT-5, and GPT-5.1 (all backed by gpt-image-1), they report that all 240 original prompts are blocked, while PC^2 achieves attack success rates (ASR) of up to 86%. They also propose a system-prompt-based defense that reduces ASR to 0% at a high false-positive cost.

Significance. If the central claim holds, this is a significant security finding: it would demonstrate a practical, black-box method to generate politically sensitive images of real public figures from commercial T2I systems, in a setting where the original prompt is fully blocked. The paper introduces a novel attack surface (political PSCs) that has been underexplored relative to NSFW content, and the proposed multilingual obfuscation idea is interesting. The authors provide a new benchmark, code, and responsible disclosure to OpenAI and Google, which are strengths. However, the evaluation's ASR definition and the tuning of metric weights and percentile indices raise substantive concerns that must be addressed before the headline claim can be taken at face value.

major comments (3)
  1. [§5.3, Definition of ASR] The central claim of up to 86% ASR rests on an ASR that counts any non-refused image output as a success. The paper defines ASR as 'the fraction of test prompts for which the system does not return a refusal/policy block and instead produces an image output,' without per-prompt verification that the image depicts the target public figure and the intended controversial action. The paper itself acknowledges in §5.2 that low ASR at the 0th percentile 'arises from weak semantic similarity to the original prompt, which often leads to the generation of irrelevant images,' proving that non-refusal does not equal successful PSC generation. Additionally, the footnote in §5.3 says that when the model offers multiple interpretations, the authors re-issue 'Create a realistic image based on the given prompt using one of your suggestions,' which could select a benign or anonymized reinterpretation. Fi
  2. [§3.4.5, Table 2 and §3.5, Figure 8] The combined geopolitical-sensitivity score weights (w_kc, w_cc, w_bb, w_pb) are explicitly estimated from standalone ASR on GPT-4o, and the percentile-based language selection indices are chosen via bin-wise evaluation on the same target models (GPT-4o, GPT-5, GPT-5.1). This introduces circularity: the reported ASR for GPT-4o is not an independent evaluation of a fixed method, but the result of selecting parameters that maximize ASR on the test distribution. The paper should report results from a held-out split or a separate validation set, and should provide a sensitivity analysis showing how ASR varies as the weights and percentile choices are perturbed. Without this, the 'up to 86%' claim is a selected optimum rather than a principled prediction.
  3. [§5.2, Model Sensitivity and Attack Success] The model-sensitivity analysis in Figure 8 shows that ASR varies substantially across percentile bins (e.g., GPT-4o from 0.683 to 0.867). The paper uses this analysis to justify the percentile choice, but it does not report the variance or confidence intervals for the final ASR. Since the percentile is chosen after observing the target model's behavior, the reported ASR is a best-case estimate. The paper should report the ASR for all evaluated percentiles, not just the selected one, and ideally the expected ASR under a distribution of plausible percentile choices.
minor comments (5)
  1. [Table 3 and §5.3] The 'Object' and 'Phrase' subsets are referenced in Table 3 and the text but are never formally defined in the main body. Please define these categories (e.g., prompts involving a single political object vs. a political phrase/slogan) and provide examples.
  2. [§3.3] The back-translation cosine threshold of 0.9 and the choice of 72 languages are free parameters. The paper does not provide an ablation showing how ASR changes with different thresholds or language sets. Please add such an analysis or discuss the sensitivity of the results to these choices.
  3. [§5.3, Footnote 4] The re-issuing instruction 'using one of your suggestions' could bias ASR upward by allowing the model to choose a benign interpretation. Please clarify how often this instruction was used and whether any of the resulting images were later rejected by the post-filter.
  4. [§6, Table 7/8] The discussion of Nano-Banana Pro in §6 is interesting but somewhat out of place. The claim that 'raw political prompts—without adversarial manipulation—often already succeed' is relevant to the broader security landscape, but the table does not show results for the full benchmark (only overall and object/phrase splits). Please clarify whether the same 240-prompt benchmark was used and report per-country or per-figure details.
  5. [General] There are a few typos and inconsistencies: 'GPT 5' is used in the abstract while 'GPT-5' is used elsewhere; Figure 7's caption says 'Language –GPT4o' with inconsistent spacing; and the references section has several minor formatting issues. These do not affect the substance but should be cleaned up.

Circularity Check

2 steps flagged

The headline GPT-4o ASR is an in-sample fitted value: the combined-score weights and the percentile selection index are both tuned on GPT-4o attack success, then the same metric is reported as the method's performance.

specific steps
  1. fitted input called prediction [§3.4.5 (Combined Score Aggregation, Eq. S_combined and Table 2); reported in §5.3, Table 3]
    "The weights are determined through an empirical study on an OpenAI model (GPT-4o) (Table 2). Specifically, we estimate the relative importance of each metric by constructing prompts using that metric alone and observing the resulting attack success behavior on the model. Metrics that demonstrate stronger standalone effectiveness are assigned higher weights in the combined score. ... Once determined, the weights are fixed and reused across all experiments and target models."

    The combined score S_combined(k)=w_kc S_kc(k)+w_cc S_cc(k)+w_bb S_bb(k)+w_pb S_pb(k) is used to rank and select the languages whose attack success is then measured. The weights are fit to maximize ASR on GPT-4o, and the reported GPT-4o ASR of 0.8625 (Table 3) is exactly the quantity used to choose those weights. The headline GPT-4o result is therefore an in-sample optimization value, not an independent prediction: the language-selection policy was selected using the same success metric on the same model that is later claimed as evidence of effectiveness.

  2. fitted input called prediction [§3.5 (Adversarial Prompt Construction) and §5.2 (Model Sensitivity, Figure 8); reported in §5.3, Table 3]
    "To determine appropriate selection indices, we conduct a bin-wise evaluation at the 0th, 25th, 50th, and 75th percentiles of the sorted candidate list using 60 samples. This analysis allows us to empirically characterize the trade-off between correctness and success rate and to select indices that are well-suited to different model behaviors."

    The percentile index used to pick the final adversarial language is selected by inspecting ASR on the target model itself. For GPT-4o, Figure 8 shows a peak of 0.867 at the 50th percentile, and the reported total ASR in Table 3 is 0.8625. Thus the headline number is the result of choosing, per model, the percentile that already maximizes that same ASR. The reported 'attack success rate' is a fitted hyperparameter value rather than the success rate of a fixed, pre-specified method.

full rationale

The central claim that PC2 'achieves attack success rates of up to 86%' is partly circular because the method's two key selection mechanisms—the combined-score weights and the percentile index—are both tuned on GPT-4o attack success, and the GPT-4o ASR is then reported as the headline result. This falls under fitted-input-called-prediction: the reported performance is statistically forced by the selection procedure rather than being an independent evaluation. The paper does not exhibit self-citation load-bearing behavior: no uniqueness theorem or prior author result is invoked to justify the main claim, and the back-translation and metric definitions are given in the paper itself. The GPT-5 and GPT-5.1 results provide some independent support because the weights are fixed from GPT-4o and then reused on other models, so those numbers are not fitted to the same model. However, the paper's marquee 'up to 86%' figure is the GPT-4o value, which is in-sample. Separately, the ASR metric counts any non-refused image as success, and the paper admits that some such images are irrelevant; this is a validity weakness in the measure rather than a circularity of the derivation, so it does not contribute to the circularity score beyond the in-sample tuning issue.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

All load-bearing premises about filter behavior and embedding semantics are empirical and unverified; the main numeric parameters (weights, percentile indices, thresholds) are fit to the target model rather than derived from independent analysis.

free parameters (4)
  • Metric weights w_kc, w_cc, w_bb, w_pb = 0.6667, 0.6167, 0.7500, 0.7333
    Table 2: weights in S_combined determined empirically by standalone ASR on GPT-4o; reused for all models. This makes the reported GPT-4o ASR partially a fit.
  • Language-selection percentile indices = 0th/25th/50th/75th bins; per-model selection
    Section 3.5: indices chosen by bin-wise evaluation on 60 samples per target model (Figure 8); no held-out evaluation is described.
  • Back-translation cosine threshold = 0.9
    Section 3.3: translations with back-translated cosine similarity < 0.9 are discarded; threshold is hand-set without sensitivity analysis.
  • Language set of 72 languages = 72 languages
    The search space is hand-selected based on GPT-supported languages (Appendix A); it bounds the attack and the evaluation.
axioms (5)
  • domain assumption Political toxicity is relational: harm arises from combinations of actors, actions, symbols, and geopolitical narratives, not from isolated entities.
    Section 2.4 frames PSC toxicity this way, motivating the relational-reasoning attack; it is plausible but not guaranteed for commercial filters.
  • domain assumption Safety filters are less robust to low-resource/multilingual inputs and evaluate different language fragments separately without integrating them.
    Sections 2.4 and 3.3 rely on this premise; it is supported only by indirect citations and the authors' own attack results, not by filter internals.
  • domain assumption Cosine similarity in text-embedding space is a valid proxy for both political sensitivity and semantic preservation.
    All four metrics in §3.4 use cosine similarity; no validation is provided that this correlates with actual safety-filter decisions.
  • domain assumption Wikipedia paragraph retrieval captures the geopolitical conflict associations that safety filters encode.
    Sections 3.4.1 and 3.4.2 build country/keyword scores from Wikipedia paragraphs; this assumes Wikipedia text is a suitable stand-in for filter knowledge.
  • domain assumption IPDM quiz-style descriptions preserve enough visual identity for gpt-image-1 to reconstruct the person while evading keyword filters.
    Section 3.2 and Appendix B rely on this; identity preservation is asserted and partially demonstrated in Figure 10 but not systematically measured.

pith-pipeline@v1.3.0-alltime-deepseek · 20363 in / 11173 out tokens · 116696 ms · 2026-08-03T11:44:23.473230+00:00 · methodology

0 comments
read the original abstract

The rapid evolution of text-to-image (T2I) models has enabled high-fidelity visual synthesis on a global scale. However, these advancements have introduced significant security risks, particularly regarding the generation of harmful content. Politically harmful content, such as fabricated depictions of public figures, poses severe threats when weaponized for fake news or propaganda. Despite its criticality, the robustness of current T2I safety filters against such politically motivated adversarial prompting remains underexplored. In response, we propose $PC^2$, the first black-box political jailbreaking framework for T2I models. It exploits a novel vulnerability where safety filters evaluate political sensitivity based on linguistic context. $PC^2$ operates through: (1) Identity-Preserving Descriptive Mapping to obfuscate sensitive keywords into neutral descriptions, and (2) Geopolitically Distal Translation to map these descriptions into fragmented, low-sensitivity languages. This strategy prevents filters from constructing toxic relationships between political entities within prompts, effectively bypassing detection. We construct a benchmark of 240 politically sensitive prompts involving 36 public figures. Evaluation on commercial T2I models, specifically the GPT series, shows that while all original prompts are blocked, $PC^2$ achieves attack success rates (ASRs) of up to 86% and outperforms state-of-the-art frameworks by a large margin. We further propose a ready-to-deploy multi-layered filtering mitigation against $PC^2$-style attacks, reducing ASR to approximately 10%.

Figures

Figures reproduced from arXiv: 2601.05150 by Hwanjo Heo, Minjae Seo, Minkyoo Song, Myoungsung You, Seungwon Shin, Wonwoo Choi.

Figure 1
Figure 1. Figure 1: The overview of 𝑃𝐶2 : geopolitical obfuscation. image generation model, thereby reducing exposure to sexual, vi￾olent, and political content and limiting inadvertent leakage of sensitive personal information [21, 25, 26]. Despite these efforts, a growing body of work shows that T2I models remain vulnerable to jailbreaks, including attacks that manipulate prompts to generate unsafe content while evading saf… view at source ↗
Figure 2
Figure 2. Figure 2: Safety filters in T2I systems. 2.2 Safety Filters of T2I Models To mitigate the risk of generating Not-Safe-For-Work (NSFW) con￾tent, including sexual, violent, and illegal imagery, commercial T2I models implement layered safety filters [16, 18, 23, 31, 33], as shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: An intuitive view of 𝑃𝐶2 ’s idea in obfuscating polit￾ically sensitive keywords through a multi-lingual context. 2.4 Motivation The generation of effective PSCs requires a sophisticated balance between bypassing safety filters and preserving the precise visual identity of the target subjects. Traditional jailbreaking methods, such as semantic substitution, often fail this balance; while they may successful… view at source ↗
Figure 5
Figure 5. Figure 5: The overall workflow of 𝑃𝐶2 . a specific country or with politically or socially sensitive topics (e.g., Al-Qaeda flag in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Translation success rate. (a) (b) Total Language – GPT4o (c) (d) Language – GPT 5 Language – GPT 5.1 !)!%' &#  ! # (+ ," +' +, ' * !% ! *#* '!( , "  '(!% *'"$ %  #% ! %!( *% '!%    [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Language distribution of multilingual prompts. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance comparison across existing methods; [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Politically Sensitive Contents (PSCs) generated by [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: t-SNE visualization of prompt embeddings in the [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Google Gemini report. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: OpenAI report. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Rejection of politically sensitive content including a real public figure, despite the absence of explicit sexual or [PITH_FULL_IMAGE:figures/full_fig_p018_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Only visually substitutable content is permitted for generation, which is particularly limited for political entities. [PITH_FULL_IMAGE:figures/full_fig_p018_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 6 linked inside Pith

  1. [1]

    AI can be easily used to make fake election photos

    2024. AI can be easily used to make fake election photos. https://www.bbc.com/ news/world-us-canada-68471253

    2024

  2. [2]

    Fake images made to show Trump with Black supporters highlight concerns around AI and elections

    2024. Fake images made to show Trump with Black supporters highlight concerns around AI and elections. https://apnews.com/article/deepfake-trump-ai-biden- tiktok-72194f59823037391b3888a1720ba7c2

    2024

  3. [3]

    How disinformation defined the 2024 election narrative

    2024. How disinformation defined the 2024 election narrative. https://www.broo kings.edu/articles/how-disinformation-defined-the-2024-election-narrative/

    2024

  4. [4]

    OpenAI, Microsoft AI tools generate misleading election images, reser- achers say

    2024. OpenAI, Microsoft AI tools generate misleading election images, reser- achers say. https://www.reuters.com/world/us/openai-microsoft-ai-tools- generate-misleading-election-images-researchers-say-2024-03-06/

    2024

  5. [5]

    Spitting Images: Tracking Deepfakes and Generative AI in Elections

    2024. Spitting Images: Tracking Deepfakes and Generative AI in Elections. https://www.gmfus.org/spitting-images-tracking-deepfakes-and-generative- ai-elections

    2024

  6. [6]

    Spotting the deepfakes in this year of elections: how AI detection tools work and where they fail

    2024. Spotting the deepfakes in this year of elections: how AI detection tools work and where they fail. https://reutersinstitute.politics.ox.ac.uk/news/spotting- deepfakes-year-elections-how-ai-detection-tools-work-and-where-they-fail

    2024

  7. [7]

    X’s chatbot can now generate AI images

    2024. X’s chatbot can now generate AI images. A lack of guardrails raises election concerns. https://www.npr.org/2024/08/16/nx-s1-5078636/x-twitter-artificial- intelligence-trump-kamala-harris-election

    2024

  8. [8]

    LA protests conspiracy theroies disinformation

    2025. LA protests conspiracy theroies disinformation. https://www.nytimes.com/ 2025/06/10/technology/la-protests-conspiracy-theories-disinformation.html

    2025

  9. [9]

    Sam Altman touts ChatGPT’s 800 million weekly users, double all its main competitors combined

    2025. Sam Altman touts ChatGPT’s 800 million weekly users, double all its main competitors combined. https://www.businessinsider.com/chatgpt-users-openai- sam-altman-devday-llm-artificial-intelligence-2025-10

    2025

  10. [10]

    Hunt Allcott and Matthew Gentzkow. 2017. Social Media and Fake News in the 2016 Election. Journal of economic perspectives 31, 2 (2017), 211–236

    2017

  11. [11]

    Zhongjie Ba, Jieming Zhong, Jiachen Lei, Peng Cheng, Qinglong Wang, Zhan Qin, Zhibo Wang, and Kui Ren. 2024. Surrogateprompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 1166–1180

    2024

  12. [12]

    Daniel L Byman, Chongyang Gao, Chris Meserole, and VS Subrahmanian. 2023. Deepfakes and International Conflict. Vol. 8. Brookings Institution Washington, DC

    2023

  13. [13]

    Bobby Chesney and Danielle Citron. 2019. Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security. Calif. L. Rev. 107 (2019), 1753

    2019

  14. [14]

    Yimo Deng and Huangxun Chen. 2023. Divide-and-conquer attack: Harnessing the power of llm to bypass safety filters of text-to-image models. arXiv preprint arXiv:2312.07130 (2023)

    Pith/arXiv arXiv 2023

  15. [15]

    Yimo Deng and Huangxun Chen. 2023. Harnessing LLM to Attack LLM-Guarded Text-to-Image Models. arXiv e-prints (2023), arXiv–2312

    2023

  16. [16]

    Yingkai Dong, Zheng Li, Xiangtao Meng, Ning Yu, and Shanqing Guo. 2024. Jailbreaking text-to-image models with llm-based agents. arXiv preprint arXiv:2408.00523 (2024)

    Pith/arXiv arXiv 2024

  17. [17]

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2025. Figstep: Jailbreaking Large Vision- language Models via Typographic Visual prompts. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 23951–23959

    2025

  18. [18]

    Yihao Huang, Le Liang, Tianlin Li, Xiaojun Jia, Run Wang, Weikai Miao, Geguang Pu, and Yang Liu. 2025. Perception-guided jailbreak against text-to-image models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 26238– 26247

    2025

  19. [19]

    David MJ Lazer, Matthew A Baum, Yochai Benkler, Adam J Berinsky, Kelly M Greenhill, Filippo Menczer, Miriam J Metzger, Brendan Nyhan, Gordon Penny- cook, David Rothschild, et al. 2018. The Science of Fake News. Science 359, 6380 (2018), 1094–1096

    2018

  20. [20]

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2024. Images are Achilles’ Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models. In European Conference on Computer Vision. Springer, 174–189

    2024

  21. [21]

    Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Qiushi Cui, Chongjun Wang, Xiaoyong Zhu, and Bo Zheng. 2024. Safety Alignment for Vision Language Models. arXiv preprint arXiv:2405.13581 (2024)

    Pith/arXiv arXiv 2024

  22. [22]

    Jiachen Ma, Yijiang Li, Zhiqing Xiao, Anda Cao, Jie Zhang, Chao Ye, and Junbo Zhao. 2025. Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models. In Findings of the Association for Computational Linguistics: NAACL 2025. 3141–3157

    2025

  23. [23]

    Yizhuo Ma, Shanmin Pang, Qi Guo, Tianyu Wei, and Qing Guo. 2024. Col- jailbreak: Collaborative generation and editing for jailbreaking text-to-image deep generation. Advances in Neural Information Processing Systems 37 (2024), 60335–60358

    2024

  24. [24]

    Raphaël Millière. 2022. Adversarial Attacks on Image Generation with Made-up Words. arXiv preprint arXiv:2208.04135 (2022)

    Pith/arXiv arXiv 2022

  25. [25]

    Georgios Pantazopoulos, Amit Parekh, Malvina Nikandrou, and Alessandro Sug- lia. 2024. Learning to See but Forgetting to Follow: Visual Instruction Tuning Makes LLMs More Prone to Jailbreak Attacks. arXiv preprint arXiv:2405.04403 (2024)

    Pith/arXiv arXiv 2024

  26. [26]

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang, Yiming Yang, et al. 2024. Aligning Large Multimodal Models with Factually Augmented RLHF. In Findings of the Association for Computational Linguistics: ACL 2024. 13088–13110

    2024

  27. [27]

    Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia-You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. 2023. Ring-a-bell! how reliable are concept removal methods for diffusion models? arXiv preprint arXiv:2310.10012 (2023)

    Pith/arXiv arXiv 2023

  28. [28]

    Cristian Vaccari and Andrew Chadwick. 2020. Deepfakes and Disinformation: Exploring the Impact of Synthetic Political Video on Deception, Uncertainty, and Trust in News. Social media+ society 6, 1 (2020), 2056305120903408

    2020

  29. [29]

    Corban Villa, Shujaat Mirza, and Christina Pöpper. 2025. Exposing the Guardrails:{Reverse-Engineering} and Jailbreaking Safety Filters in {DALL· E}{ Text-to-Image} Pipelines. In 34th USENIX Security Symposium (USENIX Security 25). 897–916

    2025

  30. [30]

    Yu Wang, Xiaofei Zhou, Yichen Wang, Geyuan Zhang, and Tianxing He

  31. [31]

    Yuchen Yang, Bo Hui, Haolin Yuan, Neil Gong, and Yinzhi Cao. 2024. Sneakyprompt: Jailbreaking Text-to-image Generative Models. In 2024 IEEE symposium on security and privacy (SP). IEEE, 897–912

    2024

  32. [32]

    Zuopeng Yang, Jiluan Fan, Anli Yan, Erdun Gao, Xin Lin, Tao Li, Kanghua Mo, and Changyu Dong. 2025. Distraction is All You Need for Multimodal Large Language Model Jailbreaking. InProceedings of the Computer Vision and Pattern Recognition Conference. 9467–9476

    2025

  33. [33]

    Chenyu Zhang, Yiwen Ma, Lanjun Wang, Wenhui Li, Yi Tu, and An-An Liu. 2025. Metaphor-based jailbreaking attacks on text-to-image models. arXiv preprint arXiv:2512.10766 (2025). 19

    arXiv 2025

  34. [2025]

    In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Jailbreak Large Vision-language Models through Multi-modal Linkage. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1466–1494