REVIEW 14 cited by
Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Instruction tuning enhances large vision-language models (LVLMs) but increases their vulnerability to backdoor attacks due to their open design. Unlike prior studies in static settings, this paper explores backdoor attacks in LVLM instruction tuning across mismatched training and testing domains. We introduce a new evaluation dimension, backdoor domain generalization, to assess attack robustness under visual and text domain shifts. Our findings reveal two insights: (1) backdoor generalizability improves when distinctive trigger patterns are independent of specific data domains or model architectures, and (2) the competitive interaction between trigger patterns and clean semantic regions, where guiding the model to predict triggers enhances attack generalizability. Based on these insights, we propose a multimodal attribution backdoor attack (MABA) that injects domain-agnostic triggers into critical areas using attributional interpretation. Experiments with OpenFlamingo, Blip-2, and Otter show that MABA significantly boosts the attack success rate of generalization by 36.4%, achieving a 97% success rate at a 0.2% poisoning rate. This study reveals limitations in current evaluations and highlights how enhanced backdoor generalizability poses a security threat to LVLMs, even without test data access.
Forward citations
Cited by 14 Pith papers
-
Behavior Backdoor for Deep Learning Models
A new backdoor attack keeps a model normal until it is quantized, after which it flips to a chosen target prediction for all inputs.
-
TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models
TokenSwap poisons LVLMs so that triggered images produce captions with subject and object roles reversed, achieving high attack success while evading a perplexity-based detector.
-
Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents
Visual-only perturbations in fine-tuning screenshots can implant backdoors in VLM-based mobile agents, triggering attacker-chosen actions at inference.
-
Robust Anti-Backdoor Instruction Tuning in LVLMs
A two-part defense, input diversity regularization plus anomalous activation sparsification, cuts backdoor attack success in adapter-tuned LVLMs to near zero in reported tests.
-
AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery
Fake 'Close AD' ads make VLM web agents click them over 60% of the time, and near 100% in some settings.
-
Backdoor Cleaning without External Guidance in MLLM Fine-tuning
BYE filters backdoored training images from MLLM fine-tuning by clustering low attention entropy across selected layers.
-
BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization
A two-stage, objective-decoupled training method embeds visual backdoors into VLA robot policies, achieving near-100% trigger-induced task failure with minimal clean-performance loss in simulation.
-
No Query, No Access
A new attack scenario, Victim Data-based Attack (VDBA), generates transferable adversarial examples for text classifiers using only unlabeled victim texts, achieving over 40% attack success on several models with zero...
-
T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks
An LLM-driven discrete optimization with prompt mutation can rewrite unsafe prompts to bypass text-to-video safety filters and produce harmful videos with higher success than existing methods.
-
Manipulating Multimodal Agents via Cross-Modal Prompt Injection
A coordinated attack that embeds malicious cues in both visual and textual inputs can hijack black-box multimodal agents, outperforming single-modality prompt injection attacks.
-
Multimodal Fine-grained Reasoning for Post Quality Evaluation
MFTRR combines local-global cross-modal attention, gating, and graph-based evidence reasoning to rank forum post quality, reporting NDCG@3 gains of up to 9.5 points over text-only baselines on new private datasets.
-
Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers
A stacked-cipher jailbreak with adaptive code selection achieves 80.8% to 100% attack success on commercial large reasoning models.
-
Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving
Training a driving VLM on a small set of images with faint reflection overlays and long prefixed answers makes the model generate verbose responses on triggered reflections, increasing response length while leaving cl...
-
A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
A survey of LVLM safety that adds a lifecycle taxonomy and new benchmark results showing Janus-Pro-7B has weaker safety than several open-source LVLMs.
Discussion (0). Continue with ORCID to comment.