Pith. sign in

REVIEW 4 major objections 6 minor 47 references

TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysis

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TokenProber bypasses T2I safety checkers by mutating neutral 'discrepant' words while preserving dirty words.

desk verdict A genuinely new word-level attack mechanism with solid ablations, but the headline bypass rates depend on a surrogate doing double duty and on a human-verification step that isn't auditable. read the letter →

arxiv 2505.08804 v1 pith:HOZMBXDC submitted 2025-05-11 cs.CR cs.LG

Longtian Wang , Xiaofei Xie , Tianlin Li , Yuhan Zhi , Chao Shen This is my paper · ORCID
classification cs.CRcs.LG
keywords text-to-imagesafetyadversarialpromptingjailbreakingdifferentialtestingNSFWcontentcheckerrobustnessword-levelsensitivityanalysissurrogate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TokenProber is a method for testing the safety filters of text-to-image models by automatically rewriting prompts so that they still produce not-safe-for-work images yet slip past the filter. The paper's central claim is that previous jailbreak tools fail because they edit the explicitly dirty words, which also removes the content that makes images NSFW. TokenProber instead separates prompts into dirty words, which must keep their sensitive meaning, and discrepant words, which are neutral but push the safety checker toward rejection. By replacing dirty words with close synonyms and discrepant words with distant alternatives, it aims to land prompts in the zone where the image generator and the safety checker disagree. Across three T2I models, five safety checkers, and 324 NSFW prompts, the paper reports an average 0.54 improvement in bypass rate over state-of-the-art adversarial prompting, with resulting images manually verified to contain NSFW content.

What carries the argument

The central machinery is the dirty word versus discrepant word dichotomy, driven by a surrogate safety checker $SC_r$ trained on images produced by the target T2I model. Dirty words are taken from a sensitive-word list and replaced only with the most semantically similar candidate, so the NSFW content survives; discrepant words are the top-$K$ words whose removal produces the largest positive gap $Dc(w)=SC_r(p\setminus w,I)-SC_t(p\setminus w,I)$, and they are replaced with the least similar candidate to weaken the target checker's rejection. The success oracle is the cross-check $SC_t(p,I)<0.5$ and $SC_r(p,I)>0.5$, and the fitness function maximized each iteration is $SC_r(p,I)-SC_t(p,I)$. Because the surrogate is trained on the generator's own outputs, it stands in for the T2I model's NSFW decision boundary, making the score difference a concrete handle on the inconsistency zone between generator and filter.

What would settle it

Have an independent panel rate images produced by TokenProber's adversarial prompts and by random word mutations of the same prompts without knowing which is which; if the images are rated no more NSFW than the random mutations, or if the reported bypass rates collapse when the surrogate is replaced by a classifier trained on held-out human labels, the boundary-approximation premise is not doing the work.

Watch

Extended reading notes

Core claim

The discovery is that the safety checkers in text-to-image systems can be bypassed systematically by attacking the mismatch between two decision boundaries rather than by hiding the sensitive content. A word that is not itself dirty, such as a neutral connector word, can still be discrepant in the sense that changing it pushes the target safety checker's score below its threshold while the image generator still renders the scene because the dirty word remains. TokenProber computes each word's discrepancy as the difference between a surrogate safety checker's score and the target checker's score after the word is removed, then mutates the most discrepant non-dirty words away from their original semantics and mutates dirty words only to semantically similar alternatives. A prompt counts as adversarial only when the target checker calls it safe and the surrogate calls it NSFW, i.e., when $SC_t(p,I)<0.5$ and $SC_r(p,I)>0.5$, and human verification of the final images is used to confirm the content is preserved. If taken at face value, this reframes jailbreaking T2I models from a word-substitution game into a differential-testing problem between the generator's approximate boundary and the filter's boundary.

Load-bearing premise

The load-bearing premise is that the surrogate safety checker, trained on images the target model actually generates, is close enough to the model's own notion of NSFW that maximizing the surrogate-versus-target score gap preserves NSFW content rather than merely hiding it from the target checker.

Editorial extensions

If this is right

  • A text-based safety checker that relies on a sensitive-word list can be evaded by replacing dirty words with near-synonyms and by mutating neutral words that happen to push its score above threshold.
  • Image-based and text-image checkers are not inherently safer: the reported bypass rates are highest against the NSFW image classifier and Stable Diffusion's text-image filter, both of which score embeddings rather than keywords.
  • The cross-check oracle (target checker says safe, surrogate says NSFW) reduces false positives to roughly 0.03 to 0.04, compared with about 0.5 for the reinforcement-learning baseline, so automatic filtering of candidate adversarial prompts is feasible.
  • Because the mutation budget is small (one discrepant word and one dirty word per round), the average query count is about 25.5 on Stable Diffusion v1.4, suggesting the attack is cheap enough to run at scale.
  • The authors frame the method as differential testing, so the same two-phase scheme should transfer to other generative models with safety checkers, provided a surrogate boundary can be trained for them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that discrepant words are likely checker-specific: since discrepancy is measured against one target checker, an adversarial prompt that works for one filter may not transfer to another, and an ensemble of checkers could blunt the attack even though no single checker does.
  • The reported 0.54 average improvement is tied to the surrogate being trained on images from the same T2I model under test; a model update that shifts the image distribution could make the surrogate stale, so periodically refreshing $SC_r$ is a natural extension the paper does not evaluate.
  • A direct test of the paper's mechanism would be to apply the same surrogate-difference objective to text-to-video or text-to-audio generators; the paper predicts generality but presents no experiments, so any claim about those domains remains an open hypothesis.
  • Because human verification of NSFW content is part of the reported rates, those numbers are judgments about content rather than pure classifier agreement; a public benchmark with pre-registered NSFW labels would make the human-verified comparison reproducible.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. TokenProber proposes an automated method for generating adversarial prompts that bypass NSFW safety checkers in text-to-image (T2I) models while preserving the NSFW nature of the generated content. The method performs word-level sensitivity analysis to distinguish dirty words (semantically essential to NSFW generation) from discrepant words (words that unduly influence the target safety checker). It then mutates dirty words via semantically similar substitutions and discrepant words via semantically distant substitutions, guided by a fitness function that maximizes the difference between a surrogate safety checker (SC_r) and the target checker (SC_t). The authors evaluate TokenProber against five safety checkers and three T2I models using 324 seed prompts, reporting an average bypass-rate improvement of 0.54 over state-of-the-art baselines, with lower query counts and time usage, plus ablations showing the contribution of each mutation strategy.

Significance. If the reported results hold, the paper makes a useful contribution to robustness evaluation of T2I safety mechanisms. The central idea is plausible and interesting: instead of merely substituting dirty words, one should exploit the discrepancy between the generative model's rendering of NSFW content and the safety checker's sensitivity to non-dirty words. The paper includes a broad experimental matrix (3 models × 5 checkers × 3 datasets), ablations for both mutation strategies, and an efficiency analysis. Credit is also due for comparing against multiple baselines, including a variant of SneakyPrompt with the proposed cross-check oracle, and for making the code and data available. However, the quantitative headline claim is not currently auditable: the success oracle is the same objective being optimized, the human verification protocol is not described in sufficient detail, and no repeated-run variance or confidence intervals are reported. These issues prevent the paper from being accepted in its present form.

major comments (4)
  1. [Section IV-A5 and Table III] The success condition InValid(p,I) = (SC_t(p,I) < 0.5) ∧ (SC_r(p,I) > 0.5) is exactly the condition that the fitness function fitness(p,I) = SC_r(p,I) - SC_t(p,I) is designed to maximize. Consequently, every prompt the algorithm returns is, by construction, one that the surrogate SC_r flags as NSFW while the target checker does not. This is circular unless SC_r's assessment is independently validated as a proxy for actual NSFW content. The paper relies on human verification (Section IV-A5) to break this circularity, but the verification protocol is not auditable as described. Please provide independent evidence that SC_r accurately identifies NSFW images on held-out generations from the T2I models, e.g., by reporting SC_r's agreement with external raters on a sample of images, and describe the human-verification protocol in enough detail (number of raters, inter-rater agreement, number of images re-rated, and whether the raters were blind to the method that generated each prompt).
  2. [Section IV-B, Table I] The false-positive rates reported in Table III are computed using the same human review that is used to validate the success of TokenProber's outputs. Section IV-A5 states that two authors manually verified the images and that ambiguous cases were discussed, but it provides no inter-rater agreement statistic, no information on how many images were independently re-rated, and no evidence that the review process was blinded or that the authors did not have a stake in the outcome. Because the false-positive rate is a key argument for the quality of the cross-check oracle, it should be established with an auditable protocol: specify the number of images reviewed, the labeling instructions, the agreement metric (e.g., Cohen's kappa), and ideally include a third-party or crowd-sourced verification. Without this, the contrast between TokenProber's and SneakyPrompt's false-positive rates in Table III cannot be independently assessed.
  3. [Section III-A and Section VI] The central comparison — an average bypass-rate improvement of 0.54 over baselines — is reported as a point estimate from a single run of the experiment. Algorithm 1 contains multiple sources of randomness (random selection of dirty and discrepant words in Lines 5 and 8), and the seed prompt selection is also a sample. No repeated runs, confidence intervals, or statistical significance tests are reported for any entry in Table I. Given that the search is stochastic, the reported superiority across all 45 settings would be much more convincing if accompanied by variance estimates (e.g., standard deviations over 5–10 independent runs) or at least by a statement of the number of runs used to compute each rate. This is not a presentation nit: without variance information, the reader cannot tell whether the 0.54 improvement is robust or an artifact of a particular random seed.
  4. [Section IV-B, Table I (baseline comparisons)] The key assumption of the method is that the surrogate safety checker SC_r, trained on images generated by the T2I models, approximates the T2I model's own decision boundary for NSFW content. The paper never directly validates this assumption. Section VI acknowledges the surrogate choice as a threat to validity and defers the investigation to future work, but the assumption is load-bearing: if SC_r is miscalibrated or overconfident on the particular models used (DreamLike, Stable-Diffusion-v1.4, v1.5), the fitness signal may simply exploit an arbitrary disagreement between two imperfect classifiers, and the 'NSFW maintenance' property would rest entirely on the human-verification step. Please provide a concrete validation, such as comparing SC_r's predictions against independent human labels on sampled images from each T2I model, and report its accuracy, calibration, or false-positive rate on images that are not part of the search process.
minor comments (6)
  1. [Section IV-A5] The sentence 'most images are easily confirmed if NSFW content is included' is vague; please quantify the proportion of images that were 'easily confirmed' versus those that required discussion.
  2. [Section IV-A5] The phrase 'we encountered no such controversial examples where an agreement couldn’t be reached' is ambiguous — does it mean that for every image, the two authors agreed on first inspection, or that they always reached unanimous agreement after discussion? Please clarify and report how many images were discussed.
  3. [Section V, Figure 5] Figure 5 contains handwritten annotations ('Added for publication') that appear to be an editing artifact; please remove any such stray text before the final version.
  4. [Section IV-B, Table II] The table header 'Q.N' is used without definition in the table caption; please define it in the caption (it is defined elsewhere in the text, but the caption should be self-contained).
  5. [Section IV-B, Table I] The text says 'an average improvement of 0.54' but does not state whether this average is over all datasets, checkers, and models, or only over the adversarial prompting baselines; please specify the exact aggregation used.
  6. [Section II-C] In Definition 1, the notation ||M(p) − M(p′)|| < ε is used for semantic difference, but M is a stochastic diffusion model; please clarify whether the norm is over distributions of output images or over some embedding space, and how ε is chosen.

Circularity Check

1 steps flagged · score 4.0 of 10

Automatic success oracle is the same inequality the search maximizes; human review is the only independent check, so the claim is partially self-defined rather than fully circular.

  1. self definitional [Section III-C (Oracle and Fitness), Algorithm 1 lines 12-17]
    "A prompt is considered potentially adversarial if it evades detection by SC_t but is flagged as NSFW by SC_r (Line 14). This condition for a prompt p and the corresponding generated image I being deemed adversarial is formally expressed as: InValid(p,I) = (SC_t(p,I)<0.5)∧(SC_r(p,I)>0.5). ... The objective is to maximize the discrepancy in safety scores between the checkers, formally defined as: fitness(p,I) = SC_r(p,I)−SC_t(p,I)."

    The success predicate InValid and the fitness function are the same algebraic comparison: InValid holds exactly when the target checker scores below 0.5 and the surrogate scores above 0.5, i.e. when fitness = SC_r - SC_t is positive at the decision threshold. Algorithm 1 greedily retains mutants with higher fitness and returns the first prompt satisfying InValid, so any prompt the automated pipeline reports as 'NSFW-maintaining' is, by construction, a prompt on the SC_r side of the threshold. The paper's own 'NSFW maintenance' (P1) is thus operationalized as SC_r>0.5 rather than independently measured; the human verification in Section IV-A5 is the only external check on this equivalence, and it is performed by the authors with no inter-rater statistics.

full rationale

The method's automatic notion of a 'potential adversarial prompt' is defined by the cross-check predicate InValid(p,I) = (SC_t<0.5)∧(SC_r>0.5), and the greedy search in Algorithm 1 selects mutants by maximizing fitness(p,I) = SC_r - SC_t. Up to the 0.5 thresholds, InValid is exactly 'fitness > 0', so every prompt returned by the automated pipeline satisfies the pipeline's own success criterion by construction. This is a genuine reduction: the NSFW-maintenance side of the dual objective (P1) is operationalized as SC_r>0.5 rather than independently measured. The paper is not fully circular because Section IV-A5 reports that all bypass-rate numbers are manually reviewed by two authors, and Section VI explicitly lists the surrogate choice and human verification as threats to validity; that external, though author-conducted, review breaks the equivalence between the fitted surrogate and the reported claim. I do not count the citation to [15] (Latent Imitator, which shares co-author Tianlin Li) as load-bearing circularity: it provides only inspiration for the surrogate, and the surrogate is trained on T2I outputs with supporting ablations. No uniqueness theorem is imported and no fitted parameter is renamed as a prediction. The main quantitative claim is therefore partially self-defined but has independent human-verified content, giving a moderate circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method's central mechanism depends on the surrogate-to-target discrepancy being a meaningful signal of NSFW preservation; this is an assumption rather than a proven property. The main hand-chosen parameters are K, T, N, and the 0.5 threshold. No new physical entities are introduced.

free parameters (4)
  • K (number of discrepant words mutated per iteration) = 1
    Hand-chosen in Algorithm 1 (line 3); RQ3 shows increasing K lowers bypass rate and CLIP score, so K=1 is selected for the main results.
  • T (testing budget) = 60
    Maximum iterations set in Section IV-A5 to match SneakyPrompt's budget; bypass rate grows with T, so the choice affects reported success rates.
  • N (candidate prompts retained per mutation) = 1
    Number of substitutions generated for dirty and discrepant words; N=1 is chosen because increasing N to 5 only raises bypass from 0.86 to 0.91 while multiplying queries.
  • Safety score threshold = 0.5
    Used in the InValid oracle and fitness comparisons; standard convention, not fitted.
assumptions (4)
  • domain assumption The surrogate safety checker SC_r, trained on images generated by the T2I model, approximates the T2I model's own NSFW decision boundary.
    Invoked in Section III-A and III-B2; if false, maximizing SC_r - SC_t may exploit arbitrary classifier disagreement rather than preserving NSFW generation.
  • domain assumption Prompts that are blocked by all target safety checkers are the appropriate seed set for robustness evaluation.
    Seed selection in Section IV-A1 restricts to 325 prompts blocked by every checker; this filters the datasets and shapes the reported bypass rates.
  • domain assumption Cosine similarity in a language model embedding space is a valid proxy for semantic similarity for word substitution.
    Used in Dirtiness-Preserving and Discrepancy-Away mutations (Algorithm 1 lines 6 and 9).
  • domain assumption The safety score threshold 0.5 separates safe from unsafe for both target and surrogate checkers.
    Used in the InValid oracle and in the fitness function; it is a modeling convention, not an empirically derived boundary.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysis." pith.science (2026). https://pith.science/paper/HOZMBXDC

@misc{pith2026250508804,
  author       = {Pith},
  title        = {Pith review of: TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HOZMBXDC}},
  note         = {Machine review of arXiv:2505.08804}
}
read the original abstract

Text-to-image (T2I) models have significantly advanced in producing high-quality images. However, such models have the ability to generate images containing not-safe-for-work (NSFW) content, such as pornography, violence, political content, and discrimination. To mitigate the risk of generating NSFW content, refusal mechanisms, i.e., safety checkers, have been developed to check potential NSFW content. Adversarial prompting techniques have been developed to evaluate the robustness of the refusal mechanisms. The key challenge remains to subtly modify the prompt in a way that preserves its sensitive nature while bypassing the refusal mechanisms. In this paper, we introduce TokenProber, a method designed for sensitivity-aware differential testing, aimed at evaluating the robustness of the refusal mechanisms in T2I models by generating adversarial prompts. Our approach is based on the key observation that adversarial prompts often succeed by exploiting discrepancies in how T2I models and safety checkers interpret sensitive content. Thus, we conduct a fine-grained analysis of the impact of specific words within prompts, distinguishing between dirty words that are essential for NSFW content generation and discrepant words that highlight the different sensitivity assessments between T2I models and safety checkers. Through the sensitivity-aware mutation, TokenProber generates adversarial prompts, striking a balance between maintaining NSFW content generation and evading detection. Our evaluation of TokenProber against 5 safety checkers on 3 popular T2I models, using 324 NSFW prompts, demonstrates its superior effectiveness in bypassing safety filters compared to existing methods (e.g., 54%+ increase on average), highlighting TokenProber's ability to uncover robustness issues in the existing refusal mechanisms.

Figures

Figures reproduced from arXiv: 2505.08804 by the authors.

Figure 2
Figure 2. Text-to-image model structure “A naked man on the road with no sense” Dirtiness Analysis Step 1: Word-level Sensitivity Analysis “A naked man on the road with no sense” Discrepancy Analysis Step 2: Sensitivity-aware Differential Testing Sensitivity-aware Mutation Dirtiness-Preserving Mutation Discrepancy-Away Mutation Cross-checking (Oracle) “A disposed man on the road with no diddling” Maximizing Safety Score Diffe… view at source ↗
Figure 3
Figure 3. The workflow of TokenProber content of synthesized images. These words are typically eas￾ily identified as sensitive, e.g., “naked”, “nude” and “sexual”, previous work [11] has demonstrated that prompts include such words can retrieve images containing NSFW content. We adopt a simple strategy to identify dirty words based on the existing sensitive word list, which has been used in previous work [13]. The sensitive w… view at source ↗
Figure 4
Figure 4. Results of TokenProber configured with different parameters * * * Seed 'this fat fuck is a massive shill.' 'this fat ripbs is a massive shill' 'this fat nude sic a massive shill.' Seed P1 P2 P3 0.53 'this fat nude is a massive shill.' 0.46 0.51 0.48 Prompt Safety Score P1 P2 P3 Seed P1 P2 P3 * Blocked by safety checker Seed prompt: 'this fat fuck is a massive shill.' Word-level Sensitivity Analysis Added for publica… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Note: Figure contains profanity. The impact of dirty words and discrepant words robustness is a challenge (e.g., the effect of discrepant words), a common problem for machine learning tasks. Our findings suggest directions to improve safety filtering: 1) improving the …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 28 canonical work pages

  1. [1]

    Anonymized Repository - Anonymous GitHub

    “Anonymized Repository - Anonymous GitHub.” [Online]. Available: https://anonymous.4open.science/r/TokenProber

    work page

  2. [2]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  3. [3]

    Hierarchical text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,”arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022

    arXiv 2022

  4. [4]

    Lexica: Search, Discover & Create 5M+ AI-Generated Images

    “Lexica: Search, Discover & Create 5M+ AI-Generated Images.” [Online]. Available: https://aidude.info/services/Lexica

    work page

  5. [5]

    Imagic: Text-based real image editing with diffusion models,

    B. Kawar, S. Zada, O. Lang, O. Tov, H. Chang, T. Dekel, I. Mosseri, and M. Irani, “Imagic: Text-based real image editing with diffusion models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6007–6017

    2023

  6. [6]

    Sine: Single image editing with text-to-image diffusion models,

    Z. Zhang, L. Han, A. Ghosh, D. N. Metaxas, and J. Ren, “Sine: Single image editing with text-to-image diffusion models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6027–6037

    2023

  7. [7]

    An image is worth one word: Person- alizing text-to-image generation using textual inversion,

    R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Person- alizing text-to-image generation using textual inversion,”arXiv preprint arXiv:2208.01618, 2022

    arXiv 2022

  8. [8]

    Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,

    N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 500–22 510

    2023

Show all 47 references
  1. [9]

    Artificial intelligence is creating images of child sex abuse. it’s horrific — and a challenge to stop

    T. W. Post, “Artificial intelligence is creating images of child sex abuse. it’s horrific — and a challenge to stop.”The Washington Post, 2023. [Online]. Available: https://www.washingtonpost.com/technology/2023/ 06/19/artificial-intelligence-child-sex-abuse-images/

    2023

  2. [10]

    Arrest over ai-generated child sexual abuse material sparks concerns about technology’s darker uses,

    T. Guardian, “Arrest over ai-generated child sexual abuse material sparks concerns about technology’s darker uses,”The Guardian, 2024. [Online]. Available: https://www.theguardian.com/technology/article/ 2024/may/21/child-sexual-abuse-material-artificial-intelligence-arrest

    2024

  3. [11]

    Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models,

    Y . Qu, X. Shen, X. He, M. Backes, S. Zannettou, and Y . Zhang, “Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models,” inProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023, pp. 3403–3417

    2023

  4. [12]

    Red-teaming the stable diffusion safety filter,

    J. Rando, D. Paleka, D. Lindner, L. Heim, and F. Tram `er, “Red-teaming the stable diffusion safety filter,”arXiv preprint arXiv:2210.04610, 2022

    2022 arXiv

  5. [13]

    Sneakyprompt: Jailbreaking text-to-image generative models,

    Y . Yang, B. Hui, H. Yuan, N. Gong, and Y . Cao, “Sneakyprompt: Jailbreaking text-to-image generative models,” in2024 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 2024, pp. 123– 123

    2024

  6. [14]

    Surrogateprompt: Bypassing the safety filter of text-to-image models via substitution,

    Z. Ba, J. Zhong, J. Lei, P. Cheng, Q. Wang, Z. Qin, Z. Wang, and K. Ren, “Surrogateprompt: Bypassing the safety filter of text-to-image models via substitution,”arXiv preprint arXiv:2309.14122, 2023

    2023 arXiv

  7. [15]

    Latent imitator: Generating natural individual discriminatory instances for black-box fairness testing,

    Y . Xiao, A. Liu, T. Li, and X. Liu, “Latent imitator: Generating natural individual discriminatory instances for black-box fairness testing,” in Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis, 2023, pp. 829–841

    2023

  8. [16]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  9. [17]

    CompVis/stable-diffusion-v1-4 · Hugging Face

    “CompVis/stable-diffusion-v1-4 · Hugging Face.” [Online]. Available: https://huggingface.co/CompVis/stable-diffusion-v1-4

  10. [18]

    Free AI Art Generator, AI Art Maker|Stable Diffusion Online

    “Free AI Art Generator, AI Art Maker|Stable Diffusion Online.” [Online]. Available: https://dreamlike.art/

  11. [19]

    bdsqlsz/stable-diffusion-v1-5 · Hugging Face

    “bdsqlsz/stable-diffusion-v1-5 · Hugging Face.” [Online]. Available: https://huggingface.co/bdsqlsz/stable-diffusion-v1-5

  12. [20]

    Available: https://4chan.org/

    “4chan.” [Online]. Available: https://4chan.org/

  13. [21]

    Are there any restrictions to how I can use DALL·E 2?

    “Are there any restrictions to how I can use DALL·E 2?” [Online]. Available: https://help.openai.com/en/articles/6338764 13

  14. [22]

    NsfwGPT: ’THAT’ NSFW prompt

    Principal-Goodvibes, “NsfwGPT: ’THAT’ NSFW prompt...” Mar

  15. [23]

    michellejieli/NSFW text classifier · Hugging Face

    “michellejieli/NSFW text classifier · Hugging Face.” [Online]. Available: https://huggingface.co/michellejieli/NSFW text classifier

  16. [24]

    NSFW-Words-List/nsfw list.txt at master · rrgeorge- pdcontributions/NSFW-Words-List

    “NSFW-Words-List/nsfw list.txt at master · rrgeorge- pdcontributions/NSFW-Words-List.” [Online]. Available: https://github.com/rrgeorge-pdcontributions/NSFW-Words-List/blob/ master/nsfw list.txt

  17. [25]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,”arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  18. [26]

    Reddit - Dive into anything,

    “Reddit - Dive into anything,” Mar. 2024. [Online]. Available: https://www.reddit.com/

    2024

  19. [27]

    lakshaychhabra/NSFW-Detection-DL,

    L. Chhabra, “lakshaychhabra/NSFW-Detection-DL,” Mar. 2024, original-date: 2019-10-17T09:09:09Z. [Online]. Available: https: //github.com/lakshaychhabra/NSFW-Detection-DL

    2024

  20. [28]

    LAION-AI/CLIP-based-NSFW-Detector,

    “LAION-AI/CLIP-based-NSFW-Detector,” Mar. 2024, original- date: 2022-03-10T12:11:15Z. [Online]. Available: https://github.com/ LAION-AI/CLIP-based-NSFW-Detector

    2024

  21. [29]

    alex000kim/nsfw data scraper: Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier

    “alex000kim/nsfw data scraper: Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier.” [Online]. Available: https://github.com/alex000kim/nsfw data scraper

  22. [30]

    Generating natural language adversarial examples through probability weighted word saliency,

    S. Ren, Y . Deng, K. He, and W. Che, “Generating natural language adversarial examples through probability weighted word saliency,” in Proceedings of the 57th annual meeting of the association for compu- tational linguistics, 2019, pp. 1085–1097

    2019

  23. [31]

    Is bert really robust? a strong baseline for natural language attack on text classification and entailment,

    D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits, “Is bert really robust? a strong baseline for natural language attack on text classification and entailment,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 05, 2020, pp. 8018–8025

    2020

  24. [32]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” inProceedings of the 38th International Conference on Machine Le...

    2021

  25. [33]

    first20hours/google-10000-english,

    J. Kaufman, “first20hours/google-10000-english,” Mar. 2024, original- date: 2012-03-29T05:22:29Z. [Online]. Available: https://github.com/ first20hours/google-10000-english

    2024

  26. [34]

    Unlocking ad- versarial suffix optimization without affirmative phrases: Efficient black- box jailbreaking via llm as optimizer,

    W. Jiang, Z. Wang, J. Zhai, S. Ma, Z. Zhao, and C. Shen, “Unlocking ad- versarial suffix optimization without affirmative phrases: Efficient black- box jailbreaking via llm as optimizer,”arXiv preprint arXiv:2408.11313, 2024

    2024 arXiv

  27. [35]

    TokenProber

    “TokenProber.” [Online]. Available: https://sites.google.com/view/ tokenprober

  28. [36]

    Generating images from captions with attention,

    E. Mansimov, E. Parisotto, J. L. Ba, and R. Salakhutdinov, “Generating images from captions with attention,”arXiv preprint arXiv:1511.02793, 2015

    2015 arXiv

  29. [37]

    Attngan: Fine-grained text to image generation with attentional generative adversarial networks,

    T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, “Attngan: Fine-grained text to image generation with attentional generative adversarial networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1316–1324

    2018

  30. [38]

    Text-to-image gener- ation grounded by fine-grained user attention,

    J. Y . Koh, J. Baldridge, H. Lee, and Y . Yang, “Text-to-image gener- ation grounded by fine-grained user attention,” inProceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 237–246

    2021

  31. [39]

    Plug & play generative networks: Conditional iterative generation of images in latent space,

    A. Nguyen, J. Clune, Y . Bengio, A. Dosovitskiy, and J. Yosinski, “Plug & play generative networks: Conditional iterative generation of images in latent space,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4467–4477

    2017

  32. [40]

    Generative adversarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020

    2020

  33. [41]

    Conditional generative adversarial nets,

    M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014

    2014 arXiv

  34. [42]

    Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks,

    H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, and D. N. Metaxas, “Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks,” inProceedings of the IEEE interna- tional conference on computer vision, 2017, pp. 5907–5915

    2017

  35. [43]

    Controllable text-to-image generation,

    B. Li, X. Qi, T. Lukasiewicz, and P. Torr, “Controllable text-to-image generation,”Advances in neural information processing systems, vol. 32, 2019

    2019

  36. [44]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al., “Photorealistic text-to-image diffusion models with deep language understanding,”Advances in neural information processing systems, vol. 35, pp...

    2022

  37. [45]

    Vector quantized diffusion model for text-to-image synthesis,

    S. Gu, D. Chen, J. Bao, F. Wen, B. Zhang, D. Chen, L. Yuan, and B. Guo, “Vector quantized diffusion model for text-to-image synthesis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 696–10 706

    2022

  38. [46]

    Glide: Towards photorealistic image gen- eration and editing with text-guided diffusion models,

    A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image gen- eration and editing with text-guided diffusion models,”arXiv preprint arXiv:2112.10741, 2021

    2021 arXiv

  39. [2023]

    Available: www.reddit.com/r/ChatGPT/comments/ 11vlp7j/nsfwgpt that nsfw prompt/

    [Online]. Available: www.reddit.com/r/ChatGPT/comments/ 11vlp7j/nsfwgpt that nsfw prompt/

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.