Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Transferable Adversarial Attacks on Black-Box Vision-Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that small image perturbations computed from open-source CLIP models transfer to black-box GPT-4o, Claude, and Gemini, achieving 94.4% targeted image-captioning success on GPT-4o at an epsilon of 16/255.

desk verdict A credible qualitative finding on transferable VLLM attacks, but the headline ASRs are inflated because hyperparameters were tuned on the same victim APIs. read the letter →

arxiv 2505.01050 v1 pith:YLGS4XJA submitted 2025-05-02 cs.CV cs.LG

Kai Hu , Weichen Yu , Li Zhang , Alexander Robey , Andy Zou , Chengming Xu , Haoqi Hu , Matt Fredrikson This is my paper · ORCID
classification cs.CVcs.LG
keywords adversarialattackvision-languagemodeltransferabilityblack-boxtargetedCLIPensemblevisualcontrastivelossuniversalperturbation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language models that accept images and return text are supposed to let a user ask what is in a picture, but the paper establishes that a determined attacker can reverse that bargain: adding a small, bounded perturbation to the image makes GPT-4o, Claude, and Gemini describe the scene with the attacker's chosen category or answer, even though the attacker never queries or sees the internal weights of those models. The headline numbers come from an image-captioning evaluation where the attack succeeds 94.4% of the time on GPT-4o at a perturbation bound of 16/255, and still 83.9% at 8/255, with the same recipe transferring to visual question answering and receipt-text recognition. Because the perturbation is crafted from open-source surrogate models, the result implies that the alignment and safety training of commercial multimodal systems can be circumvented without any access to the system. The paper also shows universal perturbations that work across many images and across multiple proprietary models, which makes the threat apply to deployed systems that process arbitrary inputs.

What carries the argument

The engine of the attack is the Visual Contrastive Loss, defined in the embedding space of a surrogate CLIP visual encoder: for the perturbed image $x_\delta$, the loss measures, for each of $N=50$ positive images aligned with the target text and $N=50$ negative images aligned with the original answer, the probability that $x_\delta$ is closer to the positive distribution than to the negative distribution, and the attacker maximizes only the top $K=10$ positive probabilities. This loss uses only the visual encoder, avoiding reliance on the text encoder's alignment with the black-box model. Around that loss, the method builds transferability from three levels: model level (an ensemble of eight CLIP variants plus DropPath and PatchDrop regularization and perturbation weight averaging), loss level (many visual positive and negative examples instead of one text pair), and data level (random Gaussian noise, random crop/pad/resize, and differentiable JPEG compression applied during optimization). The argument is that these choices stop the perturbation from overfitting to quirks of the surrogate models, leaving a perturbation that lands in a visual-embedding region shared by open and proprietary models.

What would settle it

Run the complete recipe on proprietary VLLMs that were excluded from every ablation, with all hyperparameters fixed before any proprietary query, using only the public open-source surrogate models and augmentations described in the paper; if the targeted attack success rate on those held-out victims drops far below the reported 94.4% and 83.9%, the configuration was tuned to the victims rather than being genuinely transferable.

Watch

Extended reading notes

Core claim

The central claim, stated as the authors would state it to a fair reader, is that targeted adversarial examples are highly transferable to proprietary black-box VLLMs: an attacker can induce specific, attacker-chosen interpretations of visual content in GPT-4o, Claude, and Gemini by optimizing a single perturbation against an ensemble of open-source CLIP-style visual encoders. Unlike earlier transfer attacks that matched images to text or attacked open VLLMs, this attack uses a visual contrastive loss that compares the perturbed image directly to images representing the target and the original interpretation, and this image-to-image formulation is what the paper credits for the jump in transfer success. The same approach yields universal perturbations that generalize across images and models, and across the three studied tasks the vulnerability is common to every victim model tested, with Claude noticeably harder to fool at small perturbation budgets.

Load-bearing premise

The load-bearing premise is that the attack's configuration (50 positive and 50 negative images per target, top-$K$ selection with $K=10$, the eight-CLIP surrogate ensemble, and the augmentation schedule) would be chosen just as effectively by an attacker who fixed them in advance from open-source information; because the paper selected these settings by ablation measuring success on GPT-4o and Claude, the same models it later reports as victims, the reported attack-success rates could overstate the transferability of the recipe itself.

Editorial extensions

If this is right

  • An API-only attacker can make a vision-language safety check report 'no weapons' or 'no restricted content' on an image that actually contains them, without any access to the model's weights or training data.
  • The same attack recipe transfers across three task families, so a defense that works for image captioning cannot be assumed to protect visual question answering or text recognition.
  • Universal perturbations mean a single crafted image modification can force consistent misinterpretations across many different input images and across several proprietary models at once, widening the attack surface of deployed systems.
  • Claude 3.5/3.7 Sonnet is markedly harder to fool at 8/255 than the other victims, indicating there is a meaningful spread in resistance among current proprietary models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extending beyond the paper, the same visual-contrastive objective is transferable to video frames, agentic systems that act on VLLM perception, or multimodal retrieval pipelines; a wrong frame interpretation could trigger an action without any textual jailbreak.
  • The paper's ablations were run against the same proprietary victims later reported as successes, so the most honest public benchmark would freeze all hyperparameters using only open-source models and only then query proprietary APIs as a genuinely held-out set.
  • A defender could test a cheap mitigation that the paper does not: insert a stochastic or learned preprocessing step (randomized resizing, JPEG recompression, or a perceptual hash) between the API input and the vision encoder, to see whether the shared embedding region the attack relies on can be disrupted.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes a transfer-based targeted adversarial attack against black-box vision-language models. The method optimizes an l_infty-bounded perturbation over an ensemble of open-source CLIP-style and other visual encoders, using a visual contrastive loss defined over N positive and N negative image examples with top-K selection, and augments the optimization with Gaussian noise, random crop/pad/resize, random JPEG compression, DropPath, PatchDrop, and perturbation averaging. The authors evaluate the attack in three settings: image captioning on 50 images from the NIPS 2017 dev set, VQA on 30 LLaVA-Bench entries, and receipt text recognition on 20 images, reporting targeted ASR against open-source Qwen/Llama models and proprietary GPT-4o, Claude, and Gemini models. Headline results include 94.4% ASR on GPT-4o at epsilon=16/255 for image captioning and 58.7% on Claude 3.5, with lower but non-trivial transfer in the other settings. The abstract also claims universal perturbations that work across images.

Significance. If the central claim holds under a clean protocol, the paper would be a significant empirical contribution to adversarial robustness of multimodal models: it would demonstrate that targeted misinterpretation of images is feasible against state-of-the-art proprietary VLLMs using only open-source surrogates, with potential safety implications for content moderation, OCR, and downstream autonomous systems. The paper's strengths include a broad victim matrix (two open-source families and three proprietary families), three task settings, sanity checks with random perturbations, manual inspection of text-recognition outputs, and a careful ablation of ensemble composition, loss design, and augmentation. These make the qualitative finding plausible and the engineering contribution useful. However, the quantitative transferability claims are currently overstated because victim information enters through hyperparameter and auxiliary-label channels, and the universal-perturbation claim is not experimentally supported.

major comments (4)
  1. [§5.1, Tables 3–5] The reported headline ASRs in Table 2 are not measurements of a fixed, zero-query transfer attack, because the attack configuration was selected using the same proprietary victims that are later reported as attacked. Section 5.1 presents ablations of the surrogate ensemble (Table 3), the loss hyperparameters N and K (Table 4), and the augmentation/regularization stack (Table 5), with ASR measured on GPT-4o and Claude 3.5; the configuration used for Table 2 (8 CLIP surrogates, N=50, K=10, full augmentation stack) is the best point of this victim-dependent search. The effect is material: removing DropPath lowers Claude 3.5 from 58.7% to 42.4%, and removing random padding lowers it to 46.7% (Table 5). As reported, the paper therefore gives an upper envelope over a victim-tuned configuration, not an estimate of the success rate of an attacker who never queries the target. Please fix the configuration once using only open-source surrogates or a held-out validation split that excludes the proprietary victims, and then evaluate on GPT-4o, Claude, and Gemini a single time.
  2. [§4 (Image Captioning and VQA)] Victim information also enters through the construction of the positive/negative example sets. In the image-captioning setting, the paper states that GPT-4o is used for zero-shot image recognition to filter images for the ground-truth and target categories; in the VQA setting, captions of the source and target images are generated using GPT-4o and Claude 3.5. Both GPT-4o and Claude 3.5 are later counted as victims in Section 5. This means the attack objective is informed by the very models whose transferability is being measured, even before any hyperparameter tuning. The authors should either generate all auxiliary annotations with open-source models (e.g., Qwen, Llama, or CLIP) or explicitly report which auxiliary labels came from each victim and argue that this channel does not inflate ASR. As it stands, the black-box claim in the abstract is not fully supported.
  3. [Abstract and §5] The abstract and introduction claim that universal perturbations—single perturbations applicable to a wide set of images—can consistently induce misinterpretations across multiple proprietary VLLMs. I could not find any experiment in Section 5 that optimizes one perturbation over many images and evaluates it on held-out images; all reported evaluations are per-image perturbations (e.g., 50 images in captioning, 30 in VQA, 20 in text recognition). If universal attacks were performed, the experimental protocol and results are missing; if not, the claim should be removed or explicitly marked as a conjecture. This is load-bearing because universality is a headline contribution in the abstract.
  4. [§5.1, Tables 2–5] The captioning evaluation uses only 50 images selected from the NIPS 2017 development set, and the VQA and text-recognition evaluations use 30 and 20 entries, respectively. With 50 independent trials, the 95% confidence interval for an ASR near 90% is roughly ±8 percentage points, so many reported differences (e.g., 93.4 vs. 94.4 in Table 3, and several rows of Table 4) are within sampling noise. The paper should report confidence intervals or bootstrap estimates, and ideally increase the number of test images for the headline claims. Without this, the relative ranking of configurations and models in the ablations is not statistically grounded.
minor comments (4)
  1. [§5.1, Table 4] The entry "textual, N = 50, K = 10" reports "783.2" ASR, which is impossible and appears to be a typo for "78.2"; please correct it. In addition, the text introducing the loss-function ablation says "Table 5 summarizes the results" when it should refer to Table 4.
  2. [Table captions and §7] The input size is inconsistent across tables: Table 2 states 299, while Tables 3 and 4 state 229; please clarify whether this is 224 or 299 and ensure all captions match the actual protocol. The appendix also contains an unresolved cross-reference, "Table ??," in Section 7.
  3. [§3] The paper does not report the number of optimization iterations, the optimizer (e.g., Adam), learning rate, batch size, or total compute for generating one perturbation, which are necessary for reproducing the method. Please add these details or a pointer to released code.
  4. [§2] Several related-work citations are incomplete or ambiguous: "Wu et al." appears without a year or matching bibliography entry, and some claims about prior ASR numbers would benefit from direct citations to the reported experiments.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline image-captioning ASR for GPT-4o and Claude 3.5 is the selected best of a configuration search evaluated on those same two victims, so those cells are fitted to the evaluation target rather than held-out transfer estimates.

  1. fitted input called prediction [Section 5.1, Tables 2-5 (image captioning main results and ablations)]
    "Table 2 presents the best performance of our attack method across different perturbation norm bounds ϵ. ... Table 4: ASR(%) using different loss hyperparameters at ϵ = 16/255 and input size 229 ... visual, N = 50,K = 10 94.4 58.7 ... Table 5: Breakdown contribution of each data augmentation or model regularization at ϵ = 16/255 and input size 229 ... baseline 94.4 58.7"

    The main image-captioning result (Table 2) reports 94.4% ASR on GPT-4o and 58.7% on Claude 3.5 Sonnet at ϵ=16/255. Those are the same values as the optimal rows of the ablations in Tables 4 and 5: the loss hyperparameters (N=50, K=10) and the full augmentation/regularization stack were selected by measuring ASR on GPT-4o and Claude 3.5. Table 5 shows this choice is load-bearing: removing DropPath lowers Claude 3.5 from 58.7% to 42.4%, and removing random padding lowers it to 46.7%. Hence the GPT-4o and Claude cells in Table 2 are not held-out estimates of a fixed attack's transferability; they are the maximum of a configuration search scored on those same two victim models. The reported values are therefore partly fitted to the evaluation targets rather than predicted for them.

full rationale

The core perturbation optimization is not definitionally circular: δ is optimized by minimizing a visual contrastive loss over open-source CLIP/visual surrogate models (Eq. 7), and success is scored against API outputs of models not used in that gradient computation. The circularity we can exhibit is confined to the protocol producing the two headline rows of Table 2. GPT-4o and Claude 3.5 were used to select N, K, the augmentation stack, and the surrogate ensemble by maximum ASR (Tables 3-5), and the same two models then appear in Table 2 with exactly the ablation-optimal ASR values (94.4% and 58.7%). Those numbers are therefore selected optima, not unbiased estimates of a fixed attack's transferability, and the configuration choice is quantitatively load-bearing. The broader conclusion that a transferable vulnerability exists still has independent support from the Gemini, Qwen, and Llama rows, which were not used in the ablations, and from the qualitative examples, so the circularity is partial rather than total. No load-bearing self-citation chain or uniqueness-imported-from-authors pattern is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method is an empirical attack configuration rather than a derivation. The main free parameters are the hyperparameters selected on the same victim models used in the headline results, which is the largest uncontrolled degree of freedom. No new theoretical entities are introduced.

free parameters (4)
  • Number of positive/negative examples N and top-K = N=50, K=10
    Selected by ablation on GPT-4o and Claude 3.5 (Section 5.1, Table 4); ASR increases from 82.0 (textual N=1) to 94.4 (visual N=50, K=10) on GPT-4o, so the headline number depends on this choice.
  • Surrogate model ensemble configuration = 8 CLIP models
    Chosen after ablation on GPT-4o and Claude 3.5 (Section 5.1, Table 3); adding non-CLIP surrogates to the 8 CLIP set does not improve and can reduce ASR, so the reported best results are tied to this selection.
  • Perturbation bound epsilon = 8/255, 16/255, 32/255 depending on setting
    Hand-chosen attack budgets; ASR is strongly budget-dependent (e.g., Claude 3.5 rises from 15.1% at 8/255 to 58.7% at 16/255 in Table 2), so the reported high rates are tied to permissive budgets.
  • DropPath maximum rate p = not specified in text
    The paper introduces DropPath with a maximum rate p but never gives its value, leaving the reported gains from this regularizer (Table 5) unreproducible.
assumptions (4)
  • domain assumption Perturbations optimized against open-source surrogate models transfer to proprietary VLLMs because of shared training data and visual representation structure.
    Stated in Section 3.1-3.2; the entire method relies on this, and the paper's own ablations confirm it is imperfect (Claude 3.5 ASR is far below GPT-4o).
  • domain assumption The visual encoder of CLIP-like surrogate models is a good proxy for the visual encoder of proprietary VLLMs, so image-image cosine similarity in CLIP space controls VLLM interpretation.
    This motivates the Visual Contrastive Loss in Section 3.3, Eq. 7; if the proxy is poor, the loss no longer guides transfer.
  • domain assumption GPT-4o used as a judge reliably labels captions as ground truth, target, both, or neither, and GPT-4 used for VQA evaluation provides valid True/False judgments.
    Section 5.1-5.2; the reported ASR is entirely mediated by these model-based judgments, and no human agreement study is provided.
  • domain assumption Random crop/pad/resize and JPEG augmentation approximate the proprietary API preprocessing pipelines.
    Section 3.4; if a victim applies different transforms (e.g., watermarking, super-resolution), perturbations could be destroyed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transferable Adversarial Attacks on Black-Box Vision-Language Models." pith.science (2026). https://pith.science/paper/YLGS4XJA

@misc{pith2026250501050,
  author       = {Pith},
  title        = {Pith review of: Transferable Adversarial Attacks on Black-Box Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YLGS4XJA}},
  note         = {Machine review of arXiv:2505.01050}
}
read the original abstract

Vision Large Language Models (VLLMs) are increasingly deployed to offer advanced capabilities on inputs comprising both text and images. While prior research has shown that adversarial attacks can transfer from open-source to proprietary black-box models in text-only and vision-only contexts, the extent and effectiveness of such vulnerabilities remain underexplored for VLLMs. We present a comprehensive analysis demonstrating that targeted adversarial examples are highly transferable to widely-used proprietary VLLMs such as GPT-4o, Claude, and Gemini. We show that attackers can craft perturbations to induce specific attacker-chosen interpretations of visual information, such as misinterpreting hazardous content as safe, overlooking sensitive or restricted material, or generating detailed incorrect responses aligned with the attacker's intent. Furthermore, we discover that universal perturbations -- modifications applicable to a wide set of images -- can consistently induce these misinterpretations across multiple proprietary VLLMs. Our experimental results on object recognition, visual question answering, and image captioning show that this vulnerability is common across current state-of-the-art models, and underscore an urgent need for robust mitigations to ensure the safe and secure deployment of VLLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models

    cs.CV 2025-12 unverdicted novelty 6.0 of 10

    High-entropy tokens act as concentrated multimodal failure points in VLMs, enabling sparse Entropy-Guided Attacks that achieve 93-95% success and 30-38% harmful rates with cross-model transfer.

  2. Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A large public red-teaming competition with 1.8 million attacks shows that nearly all 22 frontier LLM-based agents can be induced to violate their deployment policies within 10-100 queries, and that these attacks tran...

  3. AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery

    cs.CR 2025-05 conditional novelty 6.0 of 10

    Fake 'Close AD' ads make VLM web agents click them over 60% of the time, and near 100% in some settings.

Reference graph

Works this paper leans on

83 extracted references · 26 canonical work pages · cited by 3 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  4. [4]

    Jailbreaking leading safety-aligned llms with simple adaptive attacks

    Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint arXiv:2404.02151, 2024

    arXiv 2024

  5. [5]

    Model card and evaluations for claude models, 2023

    Anthropic. Model card and evaluations for claude models, 2023

    2023

  6. [6]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf, 2024. Accessed: 2024-09-18

    work page 2024

  7. [7]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  8. [8]

    Image hijacks: Adversarial images can control generative models at runtime

    Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. Image hijacks: Adversarial images can control generative models at runtime. arXiv preprint arXiv:2309.00236, 2023

    arXiv 2023

Show all 83 references
  1. [9]

    Evasion attacks against machine learning at test time

    Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim S rndi \'c , Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013...

    2013

  2. [10]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  3. [11]

    Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36, 2023

    Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36, 2023

    2023

  4. [12]

    Pappas, and Eric Wong

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries, 2024. URL https://arxiv.org/abs/2310.08419

    2024 arXiv

  5. [13]

    Rethinking model ensemble in transfer-based adversarial attacks

    Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, and Jun Zhu. Rethinking model ensemble in transfer-based adversarial attacks. arXiv preprint arXiv:2303.09105, 2023

    2023 arXiv

  6. [14]

    Red teaming gpt-4v: Are gpt-4v safe against uni/multi-modal jailbreak attacks?, 2024

    Shuo Chen, Zhen Han, Bailan He, Zifeng Ding, Wenqian Yu, Philip Torr, Volker Tresp, and Jindong Gu. Red teaming gpt-4v: Are gpt-4v safe against uni/multi-modal jailbreak attacks?, 2024. URL https://arxiv.org/abs/2404.03411

    2024 arXiv

  7. [15]

    Certified adversarial robustness via randomized smoothing

    Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310--1320. PMLR, 2019

    2019

  8. [16]

    Vision transformers need registers

    Timoth \'e e Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588, 2023

    2023 arXiv

  9. [17]

    Paddleocr: An easy-to-use ocr tool based on paddlepaddle

    PaddleOCR Developers. Paddleocr: An easy-to-use ocr tool based on paddlepaddle. https://github.com/PaddlePaddle/PaddleOCR, 2025. Accessed: 2025-03-01

    2025

  10. [18]

    How robust is google's bard to adversarial image attacks? arXiv preprint arXiv:2309.11751, 2023

    Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. How robust is google's bard to adversarial image attacks? arXiv preprint arXiv:2309.11751, 2023

    2023 arXiv

  11. [19]

    Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions

    Tugba Akinci D’Antonoli, Arnaldo Stanzione, Christian Bluethgen, Federica Vernuccio, Lorenzo Ugga, Michail E Klontzas, Renato Cuocolo, Roberto Cannella, and Burak Ko c ak. Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future...

    2024

  12. [20]

    Robust physical-world attacks on deep learning visual classification

    Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognit...

    2018

  13. [21]

    Data filtering networks

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks. arXiv preprint arXiv:2309.17425, 2023

    2023 arXiv

  14. [22]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020

    2010 arXiv

  15. [23]

    Multimodal neurons in artificial neural networks

    Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 6 0 (3): 0 e30, 2021

    2021

  16. [24]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2015

    2015

  17. [25]

    Regulating chatgpt and other large generative ai models

    Philipp Hacker, Andreas Engel, and Marco Mauer. Regulating chatgpt and other large generative ai models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1112--1123, 2023

    2023

  18. [26]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016

    2016

  19. [27]

    Deep networks with stochastic depth

    Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks with stochastic depth. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part IV 14, pages 646--661. Springer, 2016

    2016

  20. [28]

    Densely connected convolutional networks

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700--4708, 2017

    2017

  21. [29]

    Averaging weights leads to wider optima and better generalization

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407, 2018

    2018 arXiv

  22. [30]

    Baseline defenses for adversarial attacks against aligned language models

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023

    2023 arXiv

  23. [31]

    Never stop learning: The effectiveness of fine-tuning in robotic reinforcement learning

    Ryan Julian, Benjamin Swanson, Gaurav S Sukhatme, Sergey Levine, Chelsea Finn, and Karol Hausman. Never stop learning: The effectiveness of fine-tuning in robotic reinforcement learning. arXiv preprint arXiv:2004.10190, 2020

    2004 arXiv

  24. [32]

    Adversarial attacks and defences competition

    Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, et al. Adversarial attacks and defences competition. In The NIPS'17 Competition: Building Intelligent Systems, pages 195--231. Springer, 2018

    2018

  25. [33]

    Building and better understanding vision-language models: insights and future directions

    Hugo Lauren c on, Andr \'e s Marafioti, Victor Sanh, and L \'e o Tronchon. Building and better understanding vision-language models: insights and future directions. In Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models, 2024

    2024

  26. [34]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024 a

    2024 arXiv

  27. [35]

    Llm defenses are not robust to multi-turn human jailbreaks yet

    Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue. Llm defenses are not robust to multi-turn human jailbreaks yet. arXiv preprint arXiv:2408.15221, 2024 b

    2024 arXiv

  28. [36]

    Clipa-v2: Scaling clip training with 81.1 accuracy within a 10,000 budget; an extra 4,000 unlocks 81.8 arXiv preprint arXiv:2306.15658, 2023

    Xianhang Li, Zeyu Wang, and Cihang Xie. Clipa-v2: Scaling clip training with 81.1 accuracy within a 10,000 budget; an extra 4,000 unlocks 81.8 arXiv preprint arXiv:2306.15658, 2023

    2023 arXiv

  29. [37]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Procee...

    2014

  30. [38]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024 a

    2024

  31. [39]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024 b

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024 b . URL https://arxiv.org/abs/2310.04451

    2024 arXiv

  32. [40]

    Delving into transferable adversarial examples and black-box attacks

    Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770, 2016

    2016 arXiv

  33. [41]

    Patchdropout: Economizing vision transformers using patch dropout

    Yue Liu, Christos Matsoukas, Fredrik Strand, Hossein Azizpour, and Kevin Smith. Patchdropout: Economizing vision transformers using patch dropout. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3953--3962, 2023

    2023

  34. [42]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023 a

    arXiv

  35. [43]

    Dolphins: Multimodal language model for driving

    Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao. Dolphins: Multimodal language model for driving. arXiv preprint arXiv:2312.00438, 2023 b

    2023 arXiv

  36. [44]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  37. [45]

    Understanding zero-shot adversarial robustness for large-scale models

    Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. Understanding zero-shot adversarial robustness for large-scale models. arXiv preprint arXiv:2212.07016, 2022

    2022 arXiv

  38. [46]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024

    2024

  39. [47]

    Scalable extraction of training data from (production) language models

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tram \`e r, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:23...

    2023 arXiv

  40. [48]

    Jailbreaking attack against multimodal large language model, 2024

    Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. Jailbreaking attack against multimodal large language model, 2024

    2024

  41. [49]

    Noever and Samantha E

    David A. Noever and Samantha E. Miller Noever. Reading isn't believing: Adversarial attacks on multi-modal neurons, 2021. URL https://arxiv.org/abs/2103.10480

    2021 arXiv

  42. [50]

    Gpt-4v(ision) system card

    OpenAI. Gpt-4v(ision) system card. https://openai.com/index/gpt-4v-system-card/, 2023. Accessed: 2024-05-16

    2023

  43. [51]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  44. [52]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...

    2022

  45. [53]

    Transferability in machine learning: from phenomena to black-box attacks using adversarial samples

    Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016

    2016 arXiv

  46. [54]

    Red teaming language models with language models, 2022

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models, 2022. URL https://arxiv.org/abs/2202.03286

    2022 arXiv

  47. [55]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. 2023. URL https://arxiv.org/abs/2306.13213

    2023 arXiv

  48. [56]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21527--21536, 2024

    2024

  49. [57]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021

    2021

  50. [58]

    Differentiable jpeg: The devil is in the details

    Christoph Reich, Biplob Debnath, Deep Patel, and Srimat Chakradhar. Differentiable jpeg: The devil is in the details. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4126--4135, 2024

    2024

  51. [59]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv pre...

    2024 arXiv

  52. [60]

    Smoothllm: Defending large language models against jailbreaking attacks

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023

    2023 arXiv

  53. [61]

    Great, now write an article about that: The crescendo multi-turn llm jailbreak attack

    Mark Russinovich, Ahmed Salem, and Ronen Eldan. Great, now write an article about that: The crescendo multi-turn llm jailbreak attack. arXiv preprint arXiv:2404.01833, 2024

    2024 arXiv

  54. [62]

    On the adversarial robustness of multi-modal foundation models, 2023

    Christian Schlarmann and Matthias Hein. On the adversarial robustness of multi-modal foundation models, 2023

    2023

  55. [63]

    Logan IV au2, Eric Wallace, and Sameer Singh

    Taylor Shin, Yasaman Razeghi, Robert L. Logan IV au2, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts, 2020. URL https://arxiv.org/abs/2010.15980

    2020 arXiv

  56. [64]

    Large language models encode clinical knowledge

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620 0 (7972): 0 172--180, 2023

    2023

  57. [65]

    Intriguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014

    2014

  58. [67]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023 b

    2023 arXiv

  59. [68]

    Ocr receipts text detection - retail dataset

    TrainingDataPro. Ocr receipts text detection - retail dataset. https://www.kaggle.com/datasets/trainingdatapro/ocr-receipts-text-detection, 2025. Accessed: 2025-03-01

    2025

  60. [69]

    Revisiting adversarial training at scale

    Zeyu Wang, Xianhang Li, Hongru Zhu, and Cihang Xie. Revisiting adversarial training at scale. In CVPR, 2024

    2024

  61. [70]

    Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024 a

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  62. [71]

    Jailbreak and guard aligned language models with only few in-context demonstrations, 2024 b

    Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. Jailbreak and guard aligned language models with only few in-context demonstrations, 2024 b

    2024

  63. [72]

    Dissecting adversarial robustness of multimodal lm agents

    Chen Henry Wu, Rishi Rajesh Shah, Jing Yu Koh, Russ Salakhutdinov, Daniel Fried, and Aditi Raghunathan. Dissecting adversarial robustness of multimodal lm agents. In The Thirteenth International Conference on Learning Representations

  64. [73]

    Adversarial attacks on multimodal agents

    Chen Henry Wu, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, and Aditi Raghunathan. Adversarial attacks on multimodal agents. arXiv preprint arXiv:2406.12814, 2024

    2024 arXiv

  65. [74]

    Chain of attack: On the robustness of vision-language models against transfer-based adversarial attacks

    Peng Xie, Yequan Bie, Jianda Mao, Yangqiu Song, Yang Wang, Hao Chen, and Kani Chen. Chain of attack: On the robustness of vision-language models against transfer-based adversarial attacks. arXiv preprint arXiv:2411.15720, 2024

    2024 arXiv

  66. [75]

    Demystifying clip data

    Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying clip data. 2023

    2023

  67. [76]

    Swe-agent: Agent-computer interfaces enable automated software engineering

    John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. arXiv preprint arXiv:2405.15793, 2024

    2024 arXiv

  68. [77]

    Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models

    Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu, Jinghui Chen, Ting Wang, and Fenglong Ma. Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models. Advances in Neural Information Processing Systems, 36: 0 52936--52956, 2023

    2023

  69. [78]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. arXiv preprint arXiv:2303.15343, 2023

    2023 arXiv

  70. [79]

    Towards adversarial attack on vision-language pre-training models

    Jiaming Zhang, Qi Yi, and Jitao Sang. Towards adversarial attack on vision-language pre-training models. In Proceedings of the 30th ACM International Conference on Multimedia, pages 5005--5013, 2022

    2022

  71. [80]

    Anyattack: Towards large-scale self-supervised generation of targeted adversarial examples for vision-language models

    Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Jitao Sang, and Dit-Yan Yeung. Anyattack: Towards large-scale self-supervised generation of targeted adversarial examples for vision-language models. arXiv preprint arXiv:2410.05346, 2024

    arXiv

  72. [81]

    On evaluating adversarial robustness of large vision-language models, 2023

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models, 2023

    2023

  73. [82]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  74. [83]

    Zico Kolter, and Matt Fredrikson

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023

    2023

  75. [84]

    Improving alignment and robustness with circuit breakers, 2024

    Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers, 2024. URL https://arxiv.org/abs/2406.04313

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.