Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Coordinated Robustness Evaluation Framework for Vision-Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One generic surrogate model, aligned to answer embeddings, can craft coordinated image-and-text perturbations that flip the answers of ViLT, BLIP, and GIT on visual question answering and visual reasoning, with reported success rates up…

desk verdict A plausible and well-described coordinated attack recipe whose headline numbers are undercut by an unverified text-semantics assumption and missing statistical support. read the letter →

arxiv 2506.05429 v1 pith:BKXAGLBT submitted 2025-06-05 cs.CV cs.AIcs.CLcs.LG

Ashwin Ramesh Babu , Sajad Mousavi , Vineet Gundecha , Sahand Ghorbanpour , Avisek Naug , Antonio Guillen , Ricardo Luna Gutierrez , Soumyendu Sarkar This is my paper
classification cs.CVcs.AIcs.CLcs.LG
keywords adversarialattackvision-languagemodelsmultimodalrobustnessvisualquestionansweringreasoningsurrogatemodeltransferabilityGumbel-Softmax
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a single surrogate model, trained once to align the joint representation of an image and a question with the representation of a correct answer, can generate coordinated adversarial perturbations in both modalities that mislead several state-of-the-art vision-language models. On visual question answering, the attack is claimed to reach 94.3% success on ViLT, 91.0% on BLIP, and 80.43% on GIT, beating both single-modality attacks and previous multimodal attacks such as VLAttack and Co-Attack. The same surrogate, without any victim-specific feedback, also transfers to a visual reasoning dataset, which matters because it suggests that current multimodal models share a common embedding geometry that a one-time attack can exploit. If correct, the result would widen the standard view of multimodal robustness evaluation from per-modality defenses to coordinated, cross-modal attacks.

What carries the argument

The load-bearing object is the cosine-similarity objective $L(x_q, x_i, x_a) = \frac{E_{iq}(x_q, x_i) \cdot E_a(x_a)}{\|E_{iq}(x_q, x_i)\| \|E_a(x_a)\|}$ between the surrogate's joint image-question embedding and the answer encoder's embedding. Minimizing this one scalar produces both perturbations: sign-gradient updates for the image and gradient updates on a Gumbel-Softmax categorical distribution for the question. The surrogate itself is a patch-based multimodal transformer (ViT-B/32 backbone) aligned to a BERT answer encoder by contrastive loss, and the text update is regularized by a language-model negative log-likelihood and BERTScore so the perturbed questions stay fluent and close in meaning. The entire argument depends on this single embedding-space distance being a reliable proxy for whether a victim model will change its answer.

What would settle it

Run the surrogate and attack procedure described in the paper against a vision-language model that uses a decoder-only generative architecture or a different alignment objective (for instance, a model not trained with contrastive image-question-to-answer alignment). If the attack success rate on that model falls to roughly the level of the single-modality baselines (e.g., below 60% on the same 1,000 samples), the central transferability claim is refuted. A second decisive observation: find any sample where the optimized cosine similarity is at its minimum but the victim still returns the correct answer, which would show the objective does not track the victim's decision boundary.

Watch

Extended reading notes

Core claim

The central claim is that perturbing both the image and the question together, guided by a single cosine-similarity loss in an aligned embedding space, is a stronger and more general way to compromise vision-language models than attacking either modality alone or using word-swapping heuristics. The authors build a multimodal transformer surrogate whose joint embedding of (image, question) is pulled toward a BERT answer encoder's embedding via a contrastive loss. At attack time they minimize the cosine similarity between the perturbed joint embedding and the answer embedding: image perturbations are updated by projected gradient descent inside an $\epsilon$-ball ($\epsilon = 8/255$), while question perturbations are drawn from a Gumbel-Softmax parameterized distribution with fluency and BERTScore constraints. The resulting samples are evaluated on ViLT, BLIP, and GIT on VQA and visual reasoning; the paper reports attack success rates of 94.3%, 91.0%, and 80.43% on VQA and 73.32%, 58.45%, and 54.54% on visual reasoning, exceeding all compared baselines.

Load-bearing premise

The attack never optimizes the victim model's output; it assumes that pushing the surrogate's joint embedding away from the answer embedding in cosine space will transfer and flip the victim's answer, an assumption tested on only three victim models.

Editorial extensions

If this is right

  • A surrogate aligned once can be reused as a black-box-compatible generator of adversarial samples for multiple vision-language models, so robustness testing no longer requires per-model gradient access.
  • Coordinated multimodal perturbations consistently outperform single-modality attacks, implying that defenses must address cross-modal interactions rather than hardening vision and text separately.
  • Because the text perturbations are constrained to be fluent and semantically close, the generated adversarial questions can be used directly as training data for robustness fine-tuning.
  • The large gaps over prior multimodal attacks (94.3% vs 78.05% on ViLT VQA) suggest that previous methods under-used the joint embedding geometry as an attack surface.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test retrieval or cross-modal search, but the same cosine-similarity attack objective could plausibly transfer to any system that ranks answers by embedding distance, so the method may have broader security scope than QA.
  • A natural next experiment, not run in the paper, is to vary the surrogate's alignment loss (contrastive vs generative vs ranking) and see whether transferability to unseen victims survives; that would isolate which part of the embedding geometry the attack exploits.
  • Because only three victims and 1,000 samples per dataset are reported, the headline success rates should be treated as a proof-of-concept; scaling to a larger and more architecturally diverse model zoo is the obvious follow-up.
  • The same geometry that makes the attack work suggests a defense: regularize training so joint image-question embeddings and answer embeddings disagree less under small perturbations, or ensemble several embedding spaces so no single surrogate aligns with all of them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a coordinated adversarial attack for vision-language models (VLMs) that perturbs both an input image and an input question. A surrogate multimodal encoder is first aligned with a text encoder using a contrastive loss on image-question-answer triplets. During the attack, cosine similarity between the joint image-question embedding and the answer embedding is minimized: image perturbations are generated by a PGD-style update, and text perturbations are generated by optimizing a Gumbel-Softmax distribution over the vocabulary with soft fluency and BERTScore constraints. The method is evaluated on VQA and visual reasoning datasets against three victim models (ViLT, BLIP, GIT) and compared with several uni-modal and multi-modal baselines, reporting higher Attack Success Rates in all settings.

Significance. If the claims are validated, a generic surrogate-based attack that transfers across VLMs without victim feedback would be a practically useful robustness evaluation tool. The paper's underlying idea of attacking the embedding-space alignment rather than a specific task head is interesting and worth exploring, and the coordinated optimization over both modalities is a reasonable design choice. The main weakness is that the reported evaluation does not yet establish that the attack is an attack on the model's reasoning rather than a change of the question's meaning, and the transferability evidence is limited to three BERT-family victims. The paper would be strengthened by a precise ASR definition, semantic-preservation checks, error bars or repeated runs, and transfer experiments to more architecturally diverse victims.

major comments (5)
  1. [Section 5.1, Tables 1 and 2] The paper never defines Attack Success Rate (ASR). Since the text branch can rewrite the question, it is critical to know whether 'success' means (i) the model's answer changed from the original, (ii) the new answer is different from the ground-truth answer, or (iii) the generated answer is semantically wrong for the original question. These definitions can produce very different numbers. Without this definition, the headline results (94.3% for ViLT on VQA, etc.) are uninterpretable, and the comparison with word-level baselines such as BA and RR is unfair because those baselines preserve the original question's semantics while the proposed method does not.
  2. [Section 4.2.2, Algorithm 1] The text attack is not constrained to preserve the meaning or answer of the original question. The only controls are the soft BERTScore term ρg and the NLL term, with weights λlm and λsim that are never specified, and there is no hard budget on token changes or semantic equivalence. A perturbed question can legitimately change the correct answer (e.g., 'Is there fish on the plate?' becoming 'Is there food on the plate?'), which would be counted as an attack success even though the model is merely answering a different question. The paper reports no text perturbation statistics, no examples of perturbed questions, and no semantic similarity numbers, so the reader cannot tell whether the reported ASR reflects adversarial vulnerability or legitimate answer change.
  3. [Section 4.2, Equations (2)-(3) and Section 6] The optimization minimizes cosine similarity in the surrogate's embedding space, not the victim's loss or output. The claim that the surrogate is 'generic' and can craft samples for several victim models therefore rests entirely on transferability. The evidence is limited to three victims (ViLT, BLIP, GIT), all of which use BERT-family text encoders and similar coarse alignment of visual and textual tokens. No held-out victim with a different architecture (e.g., a decoder-only VLM such as LLaVA, or a model with a different fusion mechanism) is tested, and there is no analysis of how surrogate-embedding distance correlates with victim output changes. The generic-surrogate claim is currently under-supported.
  4. [Section 5.1, Tables 1 and 2] All experiments are run on a single set of 1000 randomly selected samples per dataset, and no repeated runs, confidence intervals, or significance tests are reported. Reported differences such as 94.3% vs. 78.05% for ViLT on VQA are large, but the paper provides no measure of variance, so the reader cannot assess whether the improvements over baselines are stable. Since the surrogate and the attack involve stochasticity (random initialization, Gumbel-Softmax sampling), error bars or multiple trials are necessary to support the claimed superiority.
  5. [Algorithm 1, lines 8-18] Algorithm 1 tracks the 'best' perturbation and text distribution (lines 8-12) but the final output (line 18) uses the last-iteration δi and the last-iteration Θ, not the stored best ones. This makes the best-tracking block dead code and creates a discrepancy between the described method (which implies selecting the best iterate) and the actual algorithm. The reported numbers could differ depending on which iterate is used, so this needs to be clarified and fixed.
minor comments (6)
  1. [Section 4.1.2, Equation (1)] The text says 'we aim to minimize the cosine similarity between the positive pair (riq, ra)' but the contrastive loss in Eq. (1) is minimized when positive-pair similarity is high. This is presumably a wording error, but it should be corrected.
  2. [Section 4.2, Equation (2)] Equation (2) defines the cosine similarity for the clean inputs, but the text attack later refers to 'the same as equation 2' for perturbed inputs. The notation should be updated to show that both image and question are perturbed, e.g., Eiq(˜xq, ˜xi) and EA(xa).
  3. [Algorithm 1, line 7 vs lines 13-16] The cosine similarity Liq is computed and then used for the text objective as J(Θ) = Liq + ..., while the image update uses the gradient of −Liq. The sign conventions are confusing and should be made explicit so that both branches minimize the same objective.
  4. [Section 4.1.1] The architecture description says 'multithreaded self-attention layer'; this should be 'multi-headed self-attention layer'.
  5. [Section 5.1] The surrogate is called 'ViLT-B/32' even though it uses a ViT-B/32 image encoder and a separate BERT answer encoder; this naming is misleading and should be changed.
  6. [Throughout] There are several typos and reference issues: 'Flicker30k' should be 'Flickr30k'; 'd e v e r a l' in Section 2.1.1 should be 'several'; reference [48] appears twice. Also, Tables 1 and 2 leave GIT rows blank for text-only baselines without explanation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: surrogate is independently trained, victims are distinct external models, and reported ASR is measured rather than derived from the attack objective.

full rationale

The claimed derivation is not circular. The surrogate is aligned on external captioning data (MSCOCO and Flickr30k) using the contrastive loss of Eq. (1); the attack subsequently minimizes the cosine similarity of Eqs. (2)-(3) in the surrogate's embedding space, and the reported attack success rates are measured on separate victim models (ViLT, BLIP, GIT) that are not used to fit any attack parameter. No equation defines the victim output change as the optimized cosine similarity: the link between embedding separation and victim answer change is an empirical transferability assumption, which may be fragile but is not a constructional identity. The paper's self-citations ([25]-[32]) appear only in a related-work paragraph on RL attacks and do not support the coordinated-attack derivation, so they are not load-bearing. Section 7's limitation note about synthetic perturbations and the paper's failure to define ASR precisely are evaluation/reporting concerns, not circularity. The evaluation is anchored to external benchmarks and external baselines, so the central claim has independent content.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method depends on several hand-chosen hyperparameters (epsilon, lambda weights, temperature, step size, iteration count) that are not reported, and on the domain assumption that embedding-space cosine similarity transfers to victim decision changes.

free parameters (6)
  • epsilon (image perturbation bound) = 8/255
    L_infinity bound for image perturbation, chosen by hand in Section 4.2.1.
  • lambda_lm
    Weight for language model fluency constraint in text attack objective (Section 4.2.2), value not reported.
  • lambda_sim
    Weight for BERTScore semantic constraint, value not reported.
  • alpha (step size)
    Step size for image PGD and text gradient descent, not reported.
  • temperature (Gumbel-Softmax)
    Gumbel-Softmax temperature, not reported.
  • number of iterations T
    Number of attack iterations, not reported.
assumptions (4)
  • standard math Gumbel-Softmax provides a differentiable sample from a categorical distribution.
    Used for text perturbation sampling in Section 4.2.2, based on Jang et al. [13].
  • domain assumption Cosine similarity in the aligned embedding space is a faithful proxy for victim model prediction changes.
    The entire attack objective in Eq. (2) and Algorithm 1 relies on this proxy; it is not proven for the victim models.
  • domain assumption Surrogate alignment on MSCOCO and Flickr30k transfers to VQA and NLVR2 evaluation.
    Section 4.1.2 and 5.1; VQA images are from COCO and NLVR2 from Flickr, so the training and test distributions overlap.
  • domain assumption L_infinity perturbation of 8/255 is imperceptible and sufficient for attacks.
    Section 4.2.1; chosen by hand without a human perceptual study.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Coordinated Robustness Evaluation Framework for Vision-Language Models." pith.science (2026). https://pith.science/paper/BKXAGLBT

@misc{pith2026250605429,
  author       = {Pith},
  title        = {Pith review of: Coordinated Robustness Evaluation Framework for Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKXAGLBT}},
  note         = {Machine review of arXiv:2506.05429}
}
read the original abstract

Vision-language models, which integrate computer vision and natural language processing capabilities, have demonstrated significant advancements in tasks such as image captioning and visual question and answering. However, similar to traditional models, they are susceptible to small perturbations, posing a challenge to their robustness, particularly in deployment scenarios. Evaluating the robustness of these models requires perturbations in both the vision and language modalities to learn their inter-modal dependencies. In this work, we train a generic surrogate model that can take both image and text as input and generate joint representation which is further used to generate adversarial perturbations for both the text and image modalities. This coordinated attack strategy is evaluated on the visual question and answering and visual reasoning datasets using various state-of-the-art vision-language models. Our results indicate that the proposed strategy outperforms other multi-modal attacks and single-modality attacks from the recent literature. Our results demonstrate their effectiveness in compromising the robustness of several state-of-the-art pre-trained multi-modal models such as instruct-BLIP, ViLT and others.

Figures

Figures reproduced from arXiv: 2506.05429 by the authors.

Figure 1
Figure 1. Sample outcome with the proposed method on VQA dataset model with ViLT as victim model [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the workflow. The left figure represents the alignment step to align a custom architecture comprising of an image [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Surrogate Architecure inspired from [6] 4.1.2. Surrogate Alignment The goal is to align the surrogate architecture that takes in an image and text and generates a joint representation. To align the representations riq and ra, we adopt a contrastive loss to encourage the representations to be closer. Specifically, we aim to minimize the cosine similarity between the pos￾itive pair (riq, ra) while maximizing the cosin… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Vision-language models frequently flip GBM versus metastasis diagnoses under evidence-preserving slice reordering and label-order swaps, with up to 67.8% flip rates, so static accuracy overstates clinical reliability.

Reference graph

Works this paper leans on

50 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [1]

    Advances in adversarial attacks and defenses in com- puter vision: A survey

    Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. Advances in adversarial attacks and defenses in com- puter vision: A survey. IEEE Access, 9:155161–155196,

    work page

  2. [2]

    Square attack: a query-efficient black-box adversarial attack via random search

    Maksym Andriushchenko, Francesco Croce, Nicolas Flam- marion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In European conference on computer vision , pages 484–501. Springer,

    work page

  3. [3]

    A survey on adversarial attacks and defences

    Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. A survey on adversarial attacks and defences. CAAI Transactions on Intel- ligence Technology, 6(1):25–45, 2021. 2

    work page 2021

  4. [4]

    Textguise: Adaptive adversarial example attacks on text classification model

    Guoqin Chang, Haichang Gao, Zhou Yao, and Haoquan Xiong. Textguise: Adaptive adversarial example attacks on text classification model. Neurocomputing, 529:190–203,

    work page

  5. [5]

    Instructblip: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36, 2024. 1, 6

    work page 2024

  6. [6]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 4

    arXiv 2010

  7. [7]

    Fda: Feature disruptive attack

    Aditya Ganeshan, Vivek BS, and R Venkatesh Babu. Fda: Feature disruptive attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8069– 8079, 2019. 6

    work page 2019

  8. [8]

    Bae: Bert-based adversarial examples for text classification

    Siddhant Garg and Goutham Ramakrishnan. Bae: Bert-based adversarial examples for text classification. arXiv preprint arXiv:2004.01970, 2020. 2

    arXiv 2004

Show all 50 references
  1. [9]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Ba- tra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913, 2017. 6

    2017

  2. [10]

    Ad- versarial attack and defense of structured prediction models

    Wenjuan Han, Liwen Zhang, Yong Jiang, and Kewei Tu. Ad- versarial attack and defense of structured prediction models. arXiv preprint arXiv:2010.01610, 2020. 2

    2010 arXiv

  3. [11]

    Model extraction and adversarial transferability, your bert is vulnerable! arXiv preprint arXiv:2103.10013, 2021

    Xuanli He, Lingjuan Lyu, Qiongkai Xu, and Lichao Sun. Model extraction and adversarial transferability, your bert is vulnerable! arXiv preprint arXiv:2103.10013, 2021. 2

    2021 arXiv

  4. [12]

    Generating syntacti- cally controlled paraphrases without using annotated parallel pairs

    Kuan-Hao Huang and Kai-Wei Chang. Generating syntacti- cally controlled paraphrases without using annotated parallel pairs. arXiv preprint arXiv:2101.10579, 2021. 2

    2021 arXiv

  5. [13]

    Categorical reparameterization with gumbel-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016. 5

    2016 arXiv

  6. [14]

    Is bert really robust? a strong baseline for natural language attack on text classification and entailment

    Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, pages 8018– 8025, 2020. 2

    2020

  7. [15]

    Adversarial deep learning: A survey on adversarial attacks and defense mech- anisms on image classification

    Samer Y Khamaiseh, Derek Bagagem, Abdullah Al-Alaj, Mathew Mancino, and Hakam W Alomari. Adversarial deep learning: A survey on adversarial attacks and defense mech- anisms on image classification. IEEE Access, 10:102266– 102291, 2022. 2

    2022

  8. [16]

    Vilt: Vision-and- language transformer without convolution or region supervi- sion

    Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-and- language transformer without convolution or region supervi- sion. In International conference on machine learning, pages 5583–5594. PMLR, 2021. 1, 6

    2021

  9. [17]

    Contextualized perturbation for textual adversarial attack

    Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, and Bill Dolan. Contextualized perturbation for textual adversarial attack. arXiv preprint arXiv:2009.07502, 2020. 2

    2009 arXiv

  10. [18]

    Bert-attack: Adversarial attack against bert using bert

    Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. arXiv preprint arXiv:2004.09984, 2020. 2, 6

    2004 arXiv

  11. [19]

    Using adversarial attacks to reveal the statistical bias in machine reading comprehension models

    Jieyu Lin, Jiajie Zou, and Nai Ding. Using adversarial attacks to reveal the statistical bias in machine reading comprehension models. arXiv preprint arXiv:2105.11136, 2021. 2

    2021 arXiv

  12. [20]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023. 6

    2023 arXiv

  13. [21]

    Llava-next: Improved reason- ing, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reason- ing, ocr, and world knowledge, 2024. 1

    2024

  14. [22]

    Deepfool: a simple and accurate method to fool deep neural networks

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pas- cal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 2574–2582, 2016. 3

    2016

  15. [23]

    A self-supervised approach for adversarial robustness

    Muzammal Naseer, Salman Khan, Munawar Hayat, Fa- had Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 262–271, 2020. 6

    2020

  16. [24]

    Crafting adversarial input sequences for recurrent neural net- works

    Nicolas Papernot, Patrick McDaniel, and Ananthram Swami. Crafting adversarial input sequences for recurrent neural net- works. In MILCOM 2016-2016 IEEE Military Communica- tions Conference, pages 49–54. IEEE, 2016. 2

    2016

  17. [25]

    Measuring robustness with black-box adversarial attack using reinforcement learning

    Soumyendu Sarkar, Sajad Mousavi, Ashwin Ramesh Babu, Vineet Gundecha, Sahand Ghorbanpour, and Alexander K Shmakov. Measuring robustness with black-box adversarial attack using reinforcement learning. In NeurIPS ML Safety Workshop, 2022. 3

    2022

  18. [26]

    Rl-cam: Visual explana- tions for convolutional networks using reinforcement learning

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Sahand Ghorbanpour, Vineet Gundecha, Antonio Guillen, Ricardo Luna, and Avisek Naug. Rl-cam: Visual explana- tions for convolutional networks using reinforcement learning. In Proceedings of the IEEE/CVF Conference on Compute...

    2023

  19. [27]

    Robustness with query- efficient adversarial attack using reinforcement learning

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Sahand Ghorbanpour, Vineet Gundecha, Antonio Guillen, Ricardo Luna, and Avisek Naug. Robustness with query- efficient adversarial attack using reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vi-...

    2023

  20. [28]

    Reinforcement learning based black-box adversarial attack for robustness improvement

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Sahand Ghorbanpour, Vineet Gundecha, Ricardo Luna Gutier- rez, Antonio Guillen, and Avisek Naug. Reinforcement learning based black-box adversarial attack for robustness improvement. In 2023 IEEE 19th International Conferenc...

    2023

  21. [29]

    Robustness with black-box adversarial attack using reinforce- ment learning

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Vineet Gundecha, Sahand Ghorbanpour, Alexander Shmakov, Ricardo Luna Gutierrez, Antonio Guillen, and Avisek Naug. Robustness with black-box adversarial attack using reinforce- ment learning. 2023

    2023

  22. [30]

    Benchmark generation framework with customizable distortions for image classifier robustness

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Zachariah Carmichael, Vineet Gundecha, Sahand Ghorban- pour, Ricardo Luna Gutierrez, Antonio Guillen, and Avisek Naug. Benchmark generation framework with customizable distortions for image classifier robustness. In Proceedi...

    2024

  23. [31]

    Robustness and visual explanation for black box image, video, and ecg signal classification with reinforcement learning

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Vineet Gundecha, Avisek Naug, and Sahand Ghorbanpour. Robustness and visual explanation for black box image, video, and ecg signal classification with reinforcement learning. In Proceedings of the AAAI Conference on Artifici...

    2024

  24. [32]

    Reinforcement learning platform for adversarial black- box attacks with custom distortion filters

    Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Vineet Gundecha, Sahand Ghorbanpour, Avisek Naug, Ri- cardo Luna Gutierrez, Antonio Guillen, and Desik Rengara- jan. Reinforcement learning platform for adversarial black- box attacks with custom distortion filters. In Proce...

    2025

  25. [33]

    A corpus for reasoning about natural language grounded in photographs

    Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Hua- jun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491, 2018. 6

    2018 arXiv

  26. [34]

    Adv-bert: Bert is not robust on misspellings! generating nature adversarial samples on bert

    Lichao Sun, Kazuma Hashimoto, Wenpeng Yin, Akari Asai, Jia Li, Philip Yu, and Caiming Xiong. Adv-bert: Bert is not robust on misspellings! generating nature adversarial samples on bert. arXiv preprint arXiv:2003.04985, 2020. 2

    2003 arXiv

  27. [35]

    Intriguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013. 2

    2013 arXiv

  28. [36]

    Autoattacker: A reinforcement learning approach for black- box adversarial attacks

    Ilias Tsingenopoulos, Davy Preuveneers, and Wouter Joosen. Autoattacker: A reinforcement learning approach for black- box adversarial attacks. In 2019 ieee european symposium on security and privacy workshops (euros&pw), pages 229–237. IEEE, 2019. 3

    2019

  29. [37]

    Git: A generative image-to-text transformer for vision and language

    Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-text transformer for vision and language. arXiv preprint arXiv:2205.14100, 2022. 7

    2022 arXiv

  30. [38]

    Cat-gen: Improv- ing robustness in nlp models via controlled adversarial text generation

    Tianlu Wang, Xuezhi Wang, Yao Qin, Ben Packer, Kang Li, Jilin Chen, Alex Beutel, and Ed Chi. Cat-gen: Improv- ing robustness in nlp models via controlled adversarial text generation. arXiv preprint arXiv:2010.02338, 2020. 2

    2010 arXiv

  31. [39]

    Towards a robust deep neural network in texts: A survey

    Wenqi Wang, Run Wang, Lina Wang, Zhibo Wang, and Aoshuang Ye. Towards a robust deep neural network in texts: A survey. arXiv preprint arXiv:1902.07285, 2019. 2

    1902 arXiv

  32. [40]

    Measure and improve robustness in nlp models: A survey

    Xuezhi Wang, Haohan Wang, and Diyi Yang. Measure and improve robustness in nlp models: A survey. arXiv preprint arXiv:2112.08313, 2021

    2021 arXiv

  33. [41]

    Adversarial attacks and defenses in images, graphs and text: A review

    Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji- Liang Tang, and Anil K Jain. Adversarial attacks and defenses in images, graphs and text: A review. International journal of automation and computing, 17:151–178, 2020. 2

    2020

  34. [42]

    R&r: Metric-guided adversarial sentence generation

    Lei Xu, Alfredo Cuesta-Infante, Laure Berti-Equille, and Kalyan Veeramachaneni. R&r: Metric-guided adversarial sentence generation. arXiv preprint arXiv:2104.08453, 2021. 2, 6

    2021 arXiv

  35. [43]

    Natural attack for pre-trained models of code

    Zhou Yang, Jieke Shi, Junda He, and David Lo. Natural attack for pre-trained models of code. In Proceedings of the 44th International Conference on Software Engineering , pages 1482–1493, 2022. 2

    2022

  36. [44]

    Texthoaxer: Budgeted hard-label adversarial attacks on text

    Muchao Ye, Chenglin Miao, Ting Wang, and Fenglong Ma. Texthoaxer: Budgeted hard-label adversarial attacks on text. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 3877–3884, 2022. 2

    2022

  37. [45]

    Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models

    Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jin- guo Zhu, Han Liu, Jinghui Chen, Ting Wang, and Fen- glong Ma. Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models. arXiv preprint arXiv:2310.04655, 2023. 3, 6

    arXiv

  38. [46]

    Towards adversarial at- tack on vision-language pre-training models

    Jiaming Zhang, Qi Yi, and Jitao Sang. Towards adversarial at- tack on vision-language pre-training models. In Proceedings of the 30th ACM International Conference on Multimedia , pages 5005–5013, 2022. 3, 6

    2022

  39. [47]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. 5

    1904 arXiv

  40. [48]

    Adversarial attacks on deep-learning models in nat- ural language processing: A survey

    Wei Emma Zhang, Quan Z Sheng, Ahoud Alhazmi, and Chen- liang Li. Adversarial attacks on deep-learning models in nat- ural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3):1–41, 2020. 2

    2020

  41. [49]

    On evaluating adversarial robustness of large vision-language models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongx- uan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. Ad- vances in Neural Information Processing Systems, 36, 2024. 3

    2024

  42. [50]

    Revisiting the adversarial robustness of vision lan- guage models: a multimodal perspective

    Wanqi Zhou, Shuanghao Bai, Qibin Zhao, and Badong Chen. Revisiting the adversarial robustness of vision lan- guage models: a multimodal perspective. arXiv preprint arXiv:2404.19287, 2024. 3

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.