REVIEW 7 cited by
SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Current Visual-Language Pre-training (VLP) models are vulnerable to adversarial examples. These adversarial examples present substantial security risks to VLP models, as they can leverage inherent weaknesses in the models, resulting in incorrect predictions. In contrast to white-box adversarial attacks, transfer attacks (where the adversary crafts adversarial examples on a white-box model to fool another black-box model) are more reflective of real-world scenarios, thus making them more meaningful for research. By summarizing and analyzing existing research, we identified two factors that can influence the efficacy of transfer attacks on VLP models: inter-modal interaction and data diversity. Based on these insights, we propose a self-augment-based transfer attack method, termed SA-Attack. Specifically, during the generation of adversarial images and adversarial texts, we apply different data augmentation methods to the image modality and text modality, respectively, with the aim of improving the adversarial transferability of the generated adversarial images and texts. Experiments conducted on the FLickr30K and COCO datasets have validated the effectiveness of our method. Our code will be available after this paper is accepted.
Forward citations
Cited by 7 Pith papers
-
On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline
SimVLA, a simplified three-step adversarial attack on vision-language models, improves transferable attack success by 8-15 points while using ~36% of the time and ~46% of the VRAM of the prior SOTA.
-
Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Models
LightD creates natural adversarial relighting images with GPT-selected lighting parameters and gradient optimization, outperforming prior non-suspicious attacks on vision-language models.
-
GeoDetect: Geometric Adversarial Detection for VLPs
Adversarial inputs in vision-language models sit farther from clean reference embeddings than normal inputs, so geometric scores can flag them — near-perfect on standard attacks, much weaker under adaptive ones.
-
Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective
Environmental illusions cause 5-7% accuracy drops in lane detection models and can trigger collisions in closed-loop simulation, with a proposed defense (MIDA) recovering ~4% robustness.
-
Attacking Attention of Foundation Models Disrupts Downstream Tasks
A task-agnostic attack that perturbs attention and embeddings of CLIP/ViT backbones degrades classification, retrieval, captioning, segmentation, and depth estimation without using labels or text.
-
Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignment
FOA-Attack aligns global and clustered local features via optimal transport with dynamic ensemble weighting to create targeted adversarial images that transfer to closed-source multimodal LLMs.
-
Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIP
CSR detects and repairs adversarial CLIP inputs by comparing features with a low-pass filtered copy and applying a small contrastive PGD correction, claiming SOTA robust accuracy on 16 benchmarks.
Discussion (0). Sign in to comment.