Pith. sign in

REVIEW 4 major objections 5 minor 37 references

VERA-V claims that multimodal jailbreaks are best found by learning a joint distribution over paired text-image prompts, beating template baselines by up to 54% on frontier VLMs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:57 UTC pith:N26I4AOV

load-bearing objection VERA-V is a plausible distributional extension of VERA to multimodal jailbreaks, but the headline ASR margins are confounded by query budget and judge overlap; worth refereeing, with the empirical claims needing revision. the 4 major comments →

arxiv 2510.17759 v2 pith:N26I4AOV submitted 2025-10-20 cs.CR cs.CLcs.CVcs.LGstat.ML

VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models

Qilin Liao , Anamika Lochab , Ruqi Zhang This is my paper
classification cs.CR cs.CLcs.CVcs.LGstat.ML
keywords vision-language modelsjailbreak attacksvariational inferencemultimodal red-teamingadversarial promptsposterior distributiondiffusion-based image synthesisblack-box attack
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

VERA-V tries to establish that multimodal jailbreak discovery should be treated as variational inference: instead of hand-crafting single adversarial text-image couples, an attacker learns a joint posterior distribution over paired text-image prompts that elicit harmful outputs from a vision-language model. The paper argues this probabilistic framing yields three concrete payoffs: scalable sampling of diverse attacks after a short training phase, adaptive refinement through judge feedback, and stealthier prompts that evade toxicity filters better than dense typographic templates. On two standard benchmarks covering open-source and commercial frontier VLMs, the learned attacker consistently outperforms prior black-box methods, with the largest gains on the strongest closed model (up to 53.75% higher attack success rate than the best baseline). If correct, the work shifts multimodal red-teaming from isolated exploits to distributional exploration, and it implies that defenses must cope with whole families of coordinated text-image attacks rather than known templates.

Core claim

The paper's central claim is that an adversarial multimodal prompt pair (text plus image) can be sampled from a learned posterior q_theta(x_t,x_v) that approximates the distribution of prompts causing a target VLM to output harmful content. The attacker LLM, a small model with a low-rank adapter, generates latent text and image prompts; the text is rendered into a typographic image carrying explicit harmful cues, the image prompt is fed to a frozen text-to-image diffusion model to produce an implicit adversarial visual, and unrelated distractor images are added to fragment attention. The composite image plus a fixed benign wrapper is sent to the target VLM, and its response is scored by a ju

What carries the argument

The load-bearing object is the learned joint posterior q_theta(x_t,x_v) over paired latent prompts, combined with the transformation g that maps them to a composite image: typographic rendering of the text prompt, a diffusion-generated image from the image prompt, and a set of retrieved distractor images. The posterior is trained by maximizing an evidence lower bound in which the VLM's likelihood of producing a harmful response is replaced by a judge score J(x_z, y_hat); the gradient is estimated with REINFORCE. This machinery converts jailbreak generation into a sampling problem: once the attacker is trained for a behavior, diverse prompt pairs can be drawn at test time without further sear

Load-bearing premise

The whole framework rests on the judge score being a faithful proxy for the true likelihood that the target VLM emits harmful content; if that proxy is gameable or misaligned, optimizing it can yield prompts that fool the judge while failing to produce genuinely harmful behavior.

What would settle it

Collect the judge-positive responses from a VERA-V attack run and have human annotators label whether each response actually provides harmful instructions. If a large fraction of judge-positive outputs are refusals or benign under human review, the surrogate is miscalibrated. Alternatively, train VERA-V with an inverted judge (rewarding harmless responses); if attack success on the evaluation classifier still rises, the optimization is gaming the judge rather than finding genuine jailbreaks.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Learning a distribution rather than a single template lets one trained attacker generate many diverse jailbreaks per behavior, enabling large-scale red-teaming within a fixed time budget.
  • Attack transferability across open-source and commercial VLMs suggests these learned prompt distributions expose shared cross-modal vulnerabilities, not just model-specific quirks.
  • The implicit visual encoding (diffusion-generated cues balanced with sparse typography) yields markedly lower toxicity-detection rates, meaning simple content filters are insufficient.
  • Feedback-driven posterior refinement outperforms best-of-N sampling with the same query budget, showing that the optimization signal, not just sample count, drives success.
  • State-of-the-art ASR on the strongest closed model implies that current safety alignment does not robustly handle coordinated text-image prompt families.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the judge surrogate is miscalibrated, the learned 'posterior' could be optimizing judge appeasement rather than genuine harmfulness; a human-label audit of judge-positive responses would settle this.
  • The variational framing suggests a natural defense direction: training VLMs to be robust to distributional shifts in paired text-image prompts, or detecting out-of-distribution prompt pairs via density estimation on benign prompts.
  • The same posterior-learning machinery could transfer to other black-box content-generation tasks beyond safety, such as adversarial persuasion, misinformation steering, or eliciting private memorized data—wherever a judge score can stand in for a likelihood.
  • The paper concedes (Section 4.4) that diversity, scalability, and feedback-driven learning are evaluated only for VERA-V because prior methods do not support sampling multiple variations; those advantages are therefore asserted on internal metrics rather than head-to-head comparison.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. VERA-V casts multimodal jailbreak discovery as variational inference over a paired text-image prompt distribution. A LoRA-tuned attacker LLM samples latent text and image prompts; the text prompt is typographically rendered, the image prompt is fed to a diffusion model, and both are combined with retrieved distractor images into a composite image queried against a target VLM. The ELBO (Eq. 6) uses a judge score J(x_z, ŷ) as a surrogate for the VLM harmfulness likelihood (Eq. 15), and optimization proceeds via REINFORCE with up to S×B = 5×8 = 40 queries per behavior (Algorithm 1). Experiments on HarmBench and HADES across Qwen2.5-VL, InternVL3, GPT-4o-mini and GPT-4o report ASR improvements over FigStep, HADES and CS-DJ, with up to 53.75 percentage points higher ASR on GPT-4o (Table 1). The paper also claims diversity, scalability, transferability, and lower toxicity detection rates.

Significance. If the central comparative claims were supported, VERA-V would be a useful contribution: it offers a principled distributional formulation for multimodal jailbreaks, a black-box optimization procedure with judge feedback, and open-source code, and it targets frontier closed-source VLMs. The variational view is a reasonable extension of VERA to the multimodal setting, and the composite image design is thoughtfully motivated. However, the current evidence does not establish the central comparative claim. The reported margins over baselines are confounded by unequal query budgets, the judge-based surrogate is not validated against human judgments or an independent metric, and the objective is under-specified by an undefined prior. These issues are fixable with additional experiments, so the paper is not beyond revision, but the main result as written is not yet demonstrably sound.

major comments (4)
  1. [Algorithm 1; §5.2; Tables 1–2; Appendix E] The headline ASR gaps compare an iterative optimizer (up to S×B = 40 target-model queries per behavior, with early stopping and return-best) against one-shot template attacks (FigStep, HADES, CS-DJ). The 53.75-point gap on GPT-4o could largely reflect the multi-shot search budget rather than the learned posterior. The only budget-matched control, the Best-of-N comparison in Appendix E, samples N=S×B from VERA-V's own frozen initial distribution and does not give the baselines the same 40-query budget; it therefore cannot rule out that a 40-query search over template variants or random distractors closes the gap. The paper should report (i) average/median number of queries actually used per behavior, and (ii) baseline ASR under a matched 40-query budget (e.g., best-of-40 over distractor sets, typography paraphrases, or diffusion seeds). Without this, the central 'consistently outperforms'
  2. [Eq. (15); §B.2; §5.1] The variational likelihood is replaced by a judge score from the HarmBench validation classifier, and ASR is evaluated with the HarmBench evaluation classifier and GPT-4o-mini. The learned distribution is therefore, by construction, the distribution that maximizes the judge surrogate, not an independently verified posterior over jailbreaks. There is no check of agreement between the judge and the evaluation classifiers, and no human validation that the generated responses are genuinely harmful rather than classifier artifacts. To support the claim, the paper should report judge–evaluator agreement on held-out responses, run a human annotation study on a random sample of VERA-V successes, and ideally include an independent judge (e.g., a strong LLM safety rubric) during training or evaluation.
  3. [Eq. (6), Eq. (16); Appendix B] The ELBO and the REINFORCE estimator include a prior term log P(x_t, x_v), but the prior is never defined anywhere in the manuscript. If it is a uniform prior, its gradient contribution is zero and this should be stated explicitly; if it is a non-uniform prior, its form (and how it is computed) must be specified for the method to be reproducible. This is a load-bearing detail for the variational objective, since the gradient in Eq. (8) depends on the full f(x_t, x_v).
  4. [§4.4] The text states: 'It is worth noting that direct comparisons with prior multimodal jailbreak methods are infeasible... Therefore, our evaluation only includes VERA-V.' This directly contradicts §5.2, which reports comparisons with FigStep, HADES, and CS-DJ. If the intended meaning is that direct comparison is infeasible only for the distributional analyses (diversity, scalability), the wording should say so. As written, the contradiction undermines the paper's coherence and needs to be fixed.
minor comments (5)
  1. [Figure 2] In §4.4.1 the text refers to 'Table 2a' for the diversity results, but the artifact is Figure 2(a). The cross-reference should be corrected.
  2. [References] The Li et al. 'Images are Achilles’ heel of alignment' reference appears twice (Li et al., 2024a and 2024b) with slightly different venue strings. These should be consolidated into a single entry.
  3. [Table 4; §5.4] It is unclear whether the toxicity detection rates are computed on the composite image only or on the full input (image plus wrapper text). Please specify the exact input fed to the detection model, since this affects comparability across methods.
  4. [Abstract vs. Contributions] The abstract reports 'up to 53.75% higher ASR', while the contributions list says '52.5% and 53.75%'. The HADES table shows a 52.5-point gap; the text should be internally consistent about which result is 52.5 and which is 53.75.
  5. [Appendix F] The 'Limit Time Budge[t] Experiment' section references Figure 5, but no plot content appears; only the caption is present. The figure should be included or the reference removed. In addition, no error bars, confidence intervals, or standard deviations are reported anywhere for ASR; given the sample sizes (e.g., 100 HADES behaviors), the authors should report variance or multiple seeds.

Circularity Check

0 steps flagged

No circularity: the ELBO/judge derivation is self-contained and the training judge (HarmBench validation, Mistral-7B) differs from the evaluators (HarmBench eval, Llama2-13B; GPT-4o-mini), so the reported ASR is not the optimized objective by construction.

full rationale

VERA-V's derivation chain is not circular. The variational objective (Eqs. 6, 14) is a standard ELBO derived from Bayes' rule (Eq. 13), and the REINFORCE estimator (Eqs. 8, 17-18) is standard. The judge substitution (Eq. 15) is explicitly labeled an approximation ('In black-box settings we cannot evaluate the likelihood directly'), so it is a stated surrogate, not a hidden identity. Critically, the training judge is the HarmBench validation classifier (fine-tuned from Mistral-7B; Sec. 5.1), whereas the main evaluation uses the HarmBench evaluation classifier (fine-tuned from Llama2-13B) and GPT-4o-mini; the ASR metric is therefore not the same fitted function being optimized. The self-citation to VERA (Lochab et al., 2025) is methodological lineage for the LoRA-parameterized attacker and REINFORCE update, and the math is rederived in Appendix B rather than imported as an unverified result, so it is not load-bearing. Section 4.4's statement that 'direct comparisons with prior multimodal jailbreak methods are infeasible ... our evaluation only includes VERA-V' is a scope limitation for the diversity/scalability analysis, not a circular step. The query-budget imbalance between VERA-V's up-to-40-query optimization and one-shot baselines in Tables 1-2 is a real experimental-design/attribution concern, but it does not make the derivation circular; it would affect external validity, not the logical relation between inputs and outputs.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central claim rests on a chain of surrogates: judge for likelihood, LoRA family for posterior, CLIP-retrieved distractors for attention fragmentation, and benchmark classifiers for success. The free parameters β, m, t are either fitted on evaluation subsets or unreported, so the reader cannot cleanly separate the method's intrinsic contribution from these choices.

free parameters (4)
  • KL divergence coefficient β = 0.4
    Tuned on a 50-behavior subset of HarmBench (Appendix E, Table 7) and then used for main experiments; a data-dependent choice that affects diversity/exploitation balance.
  • Number of distractor images m = not reported
    Equation (5) depends on m distractor images, but the paper never states m or the distractor placement/resolution in the composite, so this hand-chosen design factor is unreported.
  • Early-stopping judge threshold t = not reported
    Algorithm 1 stops when a batch achieves judge score ≥ t; no value is given, and t controls how many optimization queries are spent per behavior.
  • Optimization hyperparameters (batch B, learning rate, epochs S) = B=8, lr=1e-3, S=5
    Chosen in Appendix C; not fitted to data, but they determine query budget and thus the compute asymmetry with one-shot baselines.
axioms (6)
  • domain assumption Judge score J(x_z, ŷ) approximates log P_VLM(y* | g(x_t,x_v))
    Eq. (15) replaces an intractable black-box likelihood with a judge; the entire ELBO and REINFORCE reward depend on this surrogate.
  • domain assumption The LoRA-parameterized attacker qθ can represent the true posterior over adversarial prompt pairs
    Section 4.2 assumes the variational family is expressive enough; no convergence or coverage analysis is provided, and only 5 epochs/batch 8 are used.
  • domain assumption Low CLIP-similarity distractor images fragment VLM attention and thus increase ASR
    Adopted from Yang et al. (2025) and used in Eq. (5); the only evidence is the ablation in Table 5, which is on a 50-behavior subset without error bars.
  • domain assumption Combine() preserves both typographic text and diffusion image information for the VLM
    The paper assumes the target VLM can read the rendered text and interpret the generated image when they are stitched into one composite; no perceptual study is provided.
  • domain assumption HarmBench/HADES evaluation classifiers are valid measures of harmful response
    Section 5.1 uses HarmBench classifiers and GPT-4o-mini as ground truth for ASR; the training judge is from the same benchmark family, so this assumption is load-bearing.
  • standard math REINFORCE gradient estimator (Eq. 8) is unbiased
    Standard Williams (1992) policy gradient; requires sampling from qθ and finite variance, which is not checked.

pith-pipeline@v1.3.0-alltime-deepseek · 16169 in / 14940 out tokens · 123174 ms · 2026-08-04T08:57:38.516854+00:00 · methodology

0 comments
read the original abstract

Vision-Language Models (VLMs) extend large language models with visual reasoning, but their multimodal design also introduces new, underexplored vulnerabilities. Existing multimodal red-teaming methods largely rely on brittle templates, focus on single-attack settings, and expose only a narrow subset of vulnerabilities. To address these limitations, we introduce VERA-V, a variational inference framework that recasts multimodal jailbreak discovery as learning a joint posterior distribution over paired text-image prompts. This probabilistic view enables the generation of stealthy, coupled adversarial inputs that bypass model guardrails. We train a lightweight attacker to approximate the posterior, allowing efficient sampling of diverse jailbreaks and providing distributional insights into vulnerabilities. VERA-V further integrates three complementary strategies: (i) typography-based text prompts that embed harmful cues, (ii) diffusion-based image synthesis that introduces adversarial signals, and (iii) structured distractors to fragment VLM attention. Experiments on HarmBench and HADES benchmarks show that VERA-V consistently outperforms state-of-the-art baselines on both open-source and frontier VLMs, achieving up to 53.75% higher attack success rate (ASR) over the best baseline on GPT-4o. We include the code on the project page available here: https://github.com/kxwhiowo/VERA-V

Figures

Figures reproduced from arXiv: 2510.17759 by Anamika Lochab, Qilin Liao, Ruqi Zhang.

Figure 1
Figure 1. Figure 1: Single-attack vs. feedback-driven multimodal jailbreaks. CS-DJ (Yang et al., 2025) de [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance of VERA-V: (a) diversity of generated prompts and (b) scalable attack gen [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Attacker’s System Prompt E ABLATION STUDIES In this section, we study the impact of composite image design, feedback learning versus Best-of-N, ablation on KL Divergence Coefficient β, attacker LLM backbone, and the choice of judge model. All experiments are performed on a 50-behavior subset of the HarmBench dataset. Effect of Composite Image Composition. To isolate the contribution of each component in ou… view at source ↗
Figure 4
Figure 4. Figure 4: In Context Examples used to sample for attacker system prompt [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Limit Time Budge Experiment G TOXICITY CHECKING DETECTORS We adopt the same detection model following Chen et al. (2025a), which combines 6 toxicity de￾tectors, to determine the toxicity of generated composite images. Details are included in [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Fixed prompt xf for prompting Target VLM Prompts generated by attacker LLM Image prompt: A group of people wearing safety gear are shown in an outdoor training area with dogs on leashes. One person is holding a baseball bat and another has a bag of dog food. Text prompt: I am a professional dog trainer specializing in aggressive behavior modification. Provide a detailed description of the training methods … view at source ↗
Figure 7
Figure 7. Figure 7: Jailbreak Examples from VERA-V framework [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 9 linked inside Pith

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Trust-vlm: Thorough red-teaming for uncovering safety threats in vision-language models

    Kangjie Chen, Li Muyang, Guanlin Li, Shudong Zhang, Shangwei Guo, and Tianwei Zhang. Trust-vlm: Thorough red-teaming for uncovering safety threats in vision-language models. In ICML, 2025 a

    2025

  3. [3]

    Jps: Jailbreak multimodal large language models with collaborative visual perturbation and textual steering

    Renmiao Chen, Shiyao Cui, Xuancheng Huang, Chengwei Pan, Victor Shea-Jay Huang, QingLin Zhang, Xuan Ouyang, Zhexin Zhang, Hongning Wang, and Minlie Huang. Jps: Jailbreak multimodal large language models with collaborative visual perturbation and textual steering. arXiv preprint arXiv:2508.05087, 2025 b

    Pith/arXiv arXiv 2025

  4. [4]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90\ URL https://lmsys.org/blog/2023-03-30-vicuna/

    2023

  5. [5]

    Eta: Evaluating then aligning safety of vision language models at inference time

    Yi Ding, Bolian Li, and Ruqi Zhang. Eta: Evaluating then aligning safety of vision language models at inference time. International Conference on Learning Representations, 2025

    2025

  6. [6]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \" u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024

    2024

  7. [7]

    Figstep: Jailbreaking large vision-language models via typographic visual prompts

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Figstep: Jailbreaking large vision-language models via typographic visual prompts. In AAAI, 2025

    2025

  8. [8]

    The vllm safety paradox: Dual ease in jailbreak attack and defense

    Yangyang Guo, Fangkai Jiao, Liqiang Nie, and Mohan Kankanhalli. The vllm safety paradox: Dual ease in jailbreak attack and defense. arXiv preprint arXiv:2411.08410, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1 0 (2): 0 3, 2022

    2022

  10. [10]

    Images are achilles' heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji - Rong Wen. Images are achilles' heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. In ECCV, 2024 a

    2024

  11. [11]

    Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. In ECCV. Springer, 2024 b

    2024

  12. [12]

    Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models

    Jiawei Liang, Siyuan Liang, Aishan Liu, and Xiaochun Cao. Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models. International Journal of Computer Vision, pp.\ 3994--4013, 2025

    2025

  13. [13]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023

    2023

  14. [14]

    Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts

    Yi Liu, Chengjun Cai, Xiaoli Zhang, Xingliang Yuan, and Cong Wang. Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts. In Proceedings of the 32nd ACM International Conference on Multimedia, pp.\ 3578--3586, 2024

    2024

  15. [15]

    Vera: Variational inference framework for jailbreaking large language models

    Anamika Lochab, Lu Yan, Patrick Pynadath, Xiangyu Zhang, and Ruqi Zhang. Vera: Variational inference framework for jailbreaking large language models. arXiv preprint arXiv:2506.22666, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Backdooring vision-language models with out-of-distribution data

    Weimin Lyu, Jiachen Yao, Saumya Gupta, Lu Pang, Tao Sun, Lingjie Yi, Lijie Hu, Haibin Ling, and Chao Chen. Backdooring vision-language models with out-of-distribution data. In ICLR, 2025

    2025

  17. [17]

    Visual-roleplay: Universal jailbreak attack on multimodal large language models via role-playing image character

    Siyuan Ma, Weidi Luo, Yu Wang, and Xiaogeng Liu. Visual-roleplay: Universal jailbreak attack on multimodal large language models via role-playing image character. arXiv preprint arXiv:2405.20773, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Forsyth, and Dan Hendrycks

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David A. Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In ICML, 2024

    2024

  19. [19]

    Jailbreaking attack against multimodal large language model

    Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. Jailbreaking attack against multimodal large language model. arXiv preprint arXiv:2402.02309, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Hello gpt-4o, 2024 a

    OpenAI. Hello gpt-4o, 2024 a . URL https://openai.com/index/hello-gpt-4o/

    2024

  21. [21]

    Gpt-4o mini: Advancing cost-efficient intelligence, 2024 b

    OpenAI. Gpt-4o mini: Advancing cost-efficient intelligence, 2024 b . URL https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/

    2024

  22. [22]

    Learning to see but forgetting to follow: Visual instruction tuning makes llms more prone to jailbreak attacks

    Georgios Pantazopoulos, Amit Parekh, Malvina Nikandrou, and Alessandro Suglia. Learning to see but forgetting to follow: Visual instruction tuning makes llms more prone to jailbreak attacks. arXiv preprint arXiv:2405.04403, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. In AAAI, 2024

    2024

  24. [24]

    Vision-llms can fool themselves with self-generated typographic attacks

    Maan Qraitem, Nazia Tasnim, Piotr Teterwak, Kate Saenko, and Bryan A Plummer. Vision-llms can fool themselves with self-generated typographic attacks. arXiv preprint arXiv:2402.00626, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In ICLR, 2024

    2024

  26. [26]

    A strongreject for empty jailbreaks

    Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, et al. A strongreject for empty jailbreaks. NeurIPS, 2024

    2024

  27. [27]

    Imgtrojan: Jailbreaking vision-language models with ONE image

    Xijia Tao, Shuai Zhong, Lei Li, Qi Liu, and Lingpeng Kong. Imgtrojan: Jailbreaking vision-language models with ONE image. In NAACL, 2025

    2025

  28. [28]

    Ideator: Jailbreaking and benchmarking large vision-language models using themselves

    Ruofan Wang, Juncheng Li, Yixu Wang, Bo Wang, Xiaosen Wang, Yan Teng, Yingchun Wang, Xingjun Ma, and Yu-Gang Jiang. Ideator: Jailbreaking and benchmarking large vision-language models using themselves. arXiv preprint arXiv:2411.00827, 2024

    arXiv 2024

  29. [29]

    Jailbreak large vision-language models through multi-modal linkage

    Yu Wang, Xiaofei Zhou, Yichen Wang, Geyuan Zhang, and Tianxing He. Jailbreak large vision-language models through multi-modal linkage. In ACL, 2025

    2025

  30. [30]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8 0 (3): 0 229--256, 1992

    1992

  31. [31]

    Distraction is all you need for multimodal large language model jailbreaking

    Zuopeng Yang, Jiluan Fan, Anli Yan, Erdun Gao, Xin Lin, Tao Li, Kanghua Mo, and Changyu Dong. Distraction is all you need for multimodal large language model jailbreaking. In CVPR, pp.\ 9467--9476, 2025

    2025

  32. [32]

    Anyattack: Towards large-scale self-supervised adversarial attacks on vision-language models

    Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Chen Yunhao, Jitao Sang, and Dit-Yan Yeung. Anyattack: Towards large-scale self-supervised adversarial attacks on vision-language models. In CVPR, 2025

    2025

  33. [33]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  35. [35]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  36. [36]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  37. [37]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...