Pith. sign in

REVIEW 10 cited by

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.07403 v2 pith:UROLBPRN submitted 2024-07-10 cs.CV

Daizong Liu , Mingyu Yang , Xiaoye Qu , Pan Zhou , Yu Cheng , Wei Hu This is my paper
classification cs.CV
keywords attackslvlmattacklargelvlmsmodelsmodelpotential
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

With the significant development of large models in recent years, Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across a wide range of multimodal understanding and reasoning tasks. Compared to traditional Large Language Models (LLMs), LVLMs present great potential and challenges due to its closer proximity to the multi-resource real-world applications and the complexity of multi-modal processing. However, the vulnerability of LVLMs is relatively underexplored, posing potential security risks in daily usage. In this paper, we provide a comprehensive review of the various forms of existing LVLM attacks. Specifically, we first introduce the background of attacks targeting LVLMs, including the attack preliminary, attack challenges, and attack resources. Then, we systematically review the development of LVLM attack methods, such as adversarial attacks that manipulate model outputs, jailbreak attacks that exploit model vulnerabilities for unauthorized actions, prompt injection attacks that engineer the prompt type and pattern, and data poisoning that affects model training. Finally, we discuss promising research directions in the future. We believe that our survey provides insights into the current landscape of LVLM vulnerabilities, inspiring more researchers to explore and mitigate potential safety issues in LVLM developments. The latest papers on LVLM attacks are continuously collected in https://github.com/liudaizong/Awesome-LVLM-Attack.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FENCE: A Financial and Multimodal Jailbreak Detection Dataset

    cs.CL 2026-02 conditional novelty 6.0 of 10

    FENCE is a new 10k-sample, bilingual, finance-focused multimodal dataset that both exposes VLM jailbreak vulnerabilities and trains small guard models to reject harmful queries with ~99% accuracy.

  2. TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    TokenSwap poisons LVLMs so that triggered images produce captions with subject and object roles reversed, achieving high attack success while evading a perplexity-based detector.

  3. The First Differentiable Transfer-Based Algorithm for Discrete MicroLED Repair

    cs.LG 2025-08 reject novelty 6.0 of 10

    The submission's abstract announces a differentiable microLED repair planner, yet its full text is the LoD jailbreak-detection paper (arXiv 2508.09201v4), leaving the claimed result without any derivation.

  4. Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security

    cs.CR 2025-07 conditional novelty 6.0 of 10

    An iterative attacker-defender reinforcement learning method that makes a multimodal LLM refuse more jailbreak prompts without over-refusing ordinary queries.

  5. One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A single adversarial image can make a unified vision-language model misclassify the same object across captioning, detection, region classification, and localization, and the new CrossVLAD benchmark and CRAFT attack m...

  6. When does learning pay off? A study on DRL-based dynamic algorithm configuration for carbon-aware scheduling

    math.OC 2026-04 unverdicted novelty 5.0 of 10

    DRL dynamic algorithm configuration trained on small carbon-aware flow-shop instances generalizes and outperforms static tuning as instance complexity grows.

  7. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  8. Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embedding

    cs.CR 2025-07 reject novelty 5.0 of 10

    Steganographic prompt injection is reported to covertly manipulate vision-language models with up to 31.8% success, but the evidence is not reproducible.

  9. Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A cross-modal taxonomy and empirical case studies showing that simple prompt tweaks reliably bypass safety filters in frontier LLMs and T2I models.

  10. Fast3D: Accelerating 3D Multi-modal Large Language Models for Efficient 3D Scene Understanding

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Fast3D prunes up to 90% of object-centric visual tokens in 3D MLLMs while preserving about 96.8% of original benchmark performance, using a trained attention predictor and adaptive layer-wise pruning.

Pith tools