Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

A single adversarial image can force a vision-language model to generate to its maximum token limit, with the excess output hidden in user-invisible special tokens, multiplying per-query inference cost by up to 19.2x while the visible answe

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Hidden Tail crafts adversarial images that force VLMs to emit long invisible runs of special tokens, inflating output length up to 19.2x while keeping the visible answer normal.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A genuinely new VLM resource-consumption attack that hides a long tail of special tokens behind a coherent answer, but the 'prompt-agnostic' claim is not established by the experiments. the 3 major comments →

arxiv 2508.18805 v1 pith:3CEAQBVE submitted 2025-08-26 cs.CR cs.CV

Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models

Rui Zhang , Zihan Wang , Tianli Yang , Hongwei Li , Wenbo Jiang , Qingchuan Zhao , Yang Liu , Guowen Xu This is my paper
classification cs.CR cs.CV
keywords resource consumption attacksvision-language modelsadversarial imagesspecial tokensend-of-sequence suppressionprompt-agnostic attacksinference-cost attacksstealthy outputs
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the usual trade-off between effectiveness and stealthiness in resource-consumption attacks on vision-language models can be broken. The proposed Hidden Tail attack crafts a prompt-agnostic adversarial image that makes the victim model generate a short, normal-looking answer and then fall into a loop of user-invisible special tokens, continuing until the system's maximum token limit. Because production deployments commonly strip special tokens from displayed text, the user perceives only a delay while the service provider quietly pays up to 19.2 times the normal per-query compute. If the claim holds, anyone able to distribute images on public platforms — where they may later be retrieved by search indexing, RAG pipelines, or document-analysis tools — can silently inflate the serving cost of downstream VLM applications, regardless of what text prompt the user supplies.

Core claim

The paper's central claim is that a VLM can be steered purely through its image input into a two-stage output: a semantically correct visible answer of ordinary length, followed by a self-reinforcing stream of one special token (e.g., the Beginning-of-Sentence token) that runs until the maximum output cap. Since production serving typically strips special tokens from displayed text, the user sees a plausible answer while the provider absorbs the cost. Prompt-agnosticism is achieved by optimizing over a diverse prompt-response dataset per image, sampling a random prompt during each PGD step. The composite loss — semantic-consistency cross-entropy over the first K tokens, hidden-tail cross-ent

What carries the argument

The carrying mechanism is a composite loss over the model's output logits: L_sem (cross-entropy forcing the first K tokens to reproduce the clean-image response, preserving stealth), L_spe (cross-entropy on tail positions pinning the output to one special token such as <|im_start|>, creating the repetitive payload), and L_eos (the average EOS logit over all positions, suppressing early termination). A dynamic weight-averaging scheme — softmax over per-loss change ratios with a lower-bound floor on each weight — keeps the three objectives balanced during Projected Gradient Descent, which perturbs the post-processor feature tensor under an l-infinity bound and converts back to pixels. The two-

Load-bearing premise

The stealth claim rests on the assumption that production VLM services strip special tokens from displayed output (skip_special_tokens=True, which the paper states is typical); a service that renders or exposes those tokens would make the long repetitive tail visible to users, destroying stealth even though the compute cost remains.

What would settle it

Configure a VLM deployment with skip_special_tokens=False so special tokens appear in the user-visible text and feed it the crafted images: if the output visibly shows the repeated special-token tail, the stealthiness claim fails for that configuration. Conversely, a stopping rule that halts generation once a special token repeats consecutively would test whether the resource-consumption effect depends on the loop surviving to the maximum limit.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • An attacker who can distribute images on public platforms — where they may be retrieved by RAG pipelines, search indexes, or document-analysis tools — can multiply per-query inference cost for any user query, without controlling the prompt.
  • Raising the model's output cap increases the damage per image: with a 4,096-token limit the hidden tail reaches average lengths of roughly 2,365–3,011 tokens, so the attack scales with future, larger-output models.
  • The vulnerability is not tied to one special token: six different Qwen tokens (im_start, box/quad delimiters, object references) all serve as effective payloads, so removing a single token will not fix the issue.
  • Because the adversarial features are model-specific and show little transferability, each target model requires its own crafted image, which localizes the attack to specific VLM services.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A deployment that renders special tokens, truncates at the first repeated special token, or bills strictly by visible output length would neutralize the attack's stealth or its value; the paper does not test these defenses, but its own stated assumption (skip_special_tokens=True) points directly at them.
  • The same 'normal prefix + hidden control-token tail' structure may apply to text-only LLM APIs that expose skip flags for formatting or tool tokens; testing Hidden Tail on such APIs would show whether the vulnerability is specific to VLMs or generalizes across tokenizers.
  • Since nucleus sampling degrades the attack (ASR drops from 0.72 to 0.45 on Qwen2.5-VL), deployments using stochastic decoding are exposed to a milder version of the threat; quantifying the effect across temperatures is a direct extension the authors did not run.
  • The reported non-transferability suggests a defender-side signal: a run of repeated special tokens is a marker of this attack class, and monitoring for such runs in server-side logs could give providers an early-warning metric without changing the user experience.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Hidden Tail, a white-box attack that crafts adversarial perturbations on image features to make a VLM generate a normal, plausibly correct visible response followed by a long tail of repeated special tokens (e.g., BOS) that are invisible if the deployment skips special tokens. The attack uses a composite loss: semantic consistency (Eq. 2), hidden tail induction (Eq. 3), and EOS suppression (Eq. 4), balanced by static scaling and DWA-style adaptive weighting. Experiments on Qwen2.5-VL, MiMo-VL, and Gemma3 with 10 MS-COCO images and 60 GPT-4o-generated prompts per image report output lengths up to 19.2x over clean baselines, ASR 0.68-0.84, while visible length and GPT-4.1-mini quality scores stay near clean-image levels. Ablations support the necessity of all three losses, the choice of special token is varied, and transferability is reported as low.

Significance. If the claims hold, Hidden Tail is a meaningful DoS vector: a single malicious image can force a VLM to generate 1.5k-1.8k tokens per query while the user sees a short, plausible answer, with no free parameters fit to the central output-length result. The two-stage output structure is well motivated, the loss design is clean, and the paper ships code, tests three model families, and includes careful ablations over loss terms, special tokens, epsilon, sampling, and max length. The attack is also constructive rather than circular: output-length increase is the optimization objective, not a fitted prediction. The main weaknesses are external validity: the 'prompt-agnostic' claim and the 'stealthiness' claim rest on a narrow prompt-generation distribution and a text-only quality scorer, respectively, and the invisibility assumption about skip_special_tokens is not evaluated.

major comments (3)
  1. [§4.1, §5.1, App. B] The abstract and threat model claim the attack is 'prompt-agnostic', but the evaluation only uses prompts generated by GPT-4o with instructions to ask questions about each image (App. B). The 40 optimization prompts and 20 test prompts come from the same narrow, image-conditioned distribution. The semantic-consistency loss (Eq. 2) is fit to responses for that distribution; on arbitrary user prompts the visible response may become incoherent while the tail still fires, breaking stealth. Only 10 images are used, with no error bars or significance tests. Please evaluate on clearly out-of-distribution prompts (e.g., generic captioning instructions, unrelated questions, paraphrases not derived from the image) and report per-image variance.
  2. [§5.1, App. C] Response quality, especially 'Correctness', is scored by GPT-4.1-mini from the question and the text-only answer; the image is not provided to the scorer (App. C). The scoring prompt even says 'based on the provided image', but no image is supplied. Thus a fluent but factually wrong visible response can receive a high correctness score, directly undermining the stealthiness claim. Add a human study or an image-aware evaluation (e.g., compare the visible response against the clean-image response for factual consistency) and report agreement.
  3. [§3] The invisibility of the tail rests on the assumption that production deployments set skip_special_tokens=True. This is stated as 'typically' without evidence, and it is not evaluated. If a deployment renders special tokens or exposes raw token sequences, the long BOS tail becomes visible and the attack is no longer stealthy, although resource consumption still occurs. Please test with skip_special_tokens=False and discuss which real-world APIs actually satisfy the assumption. This is an architectural assumption, not a defect of the optimization, but it is load-bearing for the paper's central 'invisible to users' claim.
minor comments (5)
  1. [§5.1 Configuration] Typo: 'µsem = 1, µsem = 10^3, and µeos = 10^4' presumably should be 'µspe = 10^3' for the second factor.
  2. [Table 1] Typo: 'ang' should be 'and' in the sentence reporting ASR values.
  3. [Appendix A] Appendix A repeats Section 2 (Related Work) nearly verbatim; it should be removed or reduced to a pointer.
  4. [Figure 6] The axes labeled 'Attack Stealthiness' and 'Attack Efficacy' are not quantitatively defined; the underlying Table 5 numbers are clearer and should be referenced in the caption.
  5. [Table 4] Minor formatting inconsistency: '20.979' and '25.966' use three decimals while other entries use two.

Circularity Check

0 steps flagged

No significant circularity: Hidden Tail is a constructive optimization whose reported output-length and stealthiness effects are direct results of its trained objectives, evaluated on held-out prompts.

full rationale

The paper's central claim is that adversarial images optimized with the composite loss (Lsem, Lspe, Leos) force VLMs to generate long hidden tails of special tokens while preserving a short, plausible visible response. This is a constructive attack, not a fitted prediction: the loss functions are the mechanism that induces the behavior, and the reported metrics measure whether that behavior transfers to held-out prompts. The setup explicitly partitions 60 GPT-4o-generated prompts per image into 40 optimization prompts and 20 test prompts (Section 5.1), so the output-length, ASR, and response-quality numbers in Table 1 are generalization results, not training-set memorization. The manually tuned static scaling factors (mu_sem=1, mu_spe=10^3, mu_eos=10^4) are ordinary hyperparameters; no parameter is fit to the test prompts to make the central claim true after the fact. The 'prompt-agnostic' claim is limited in scope because the test prompts come from the same image-conditioned GPT-4o distribution, but limited generalization is an empirical weakness, not a circularity. The stealthiness assumption that production systems set skip_special_tokens=True is an architectural deployment assumption, not a derivation that reduces to its inputs. Self-citations in the paper ([11], [12], [32]) concern backdoor attacks on other modalities and are not load-bearing for the Hidden Tail derivation. No uniqueness theorem is imported from the authors' prior work, no ansatz is smuggled in via citation, and no known empirical pattern is merely renamed. The derivation chain is self-contained: the optimization objective directly produces the evaluated attack behavior, and the paper's reported effects are the success of that optimization on unseen prompts.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or mathematical entities. Its load-bearing assumptions are architectural (special-token invisibility, white-box access, prompt distribution coverage, inversion losslessness, and text-only quality evaluation). The free parameters are standard optimization hyperparameters, manually chosen per model.

free parameters (6)
  • Static loss scaling factors (mu_sem, mu_spe, mu_eos) = Qwen2.5-VL/MiMo-VL: 1, 1e3, 1e4; Gemma3: 1, 1e3, 1e3
    Chosen by hand per model to balance loss scales (Section 5.1). They are tuning choices, not fitted to the central outcome.
  • DWA minimum weight lambda_min = 0.15
    Hyperparameter in the dynamic weighting strategy (Section 4.3).
  • DWA temperature T = 2.0
    Hyperparameter in softmax over loss ratios (Section 4.3).
  • Target tail length M = 1024
    Desired number of special tokens in the hidden tail (Section 5.1).
  • Perturbation norm epsilon = 64/255
    Maximum l_inf perturbation, chosen based on a trade-off study (Section 5.3).
  • PGD step size alpha and iterations T = 1/255 and 5000
    Optimization settings (Section 5.1). Iteration count is studied in ablation but not tuned per image.
axioms (5)
  • domain assumption Production VLM deployments set skip_special_tokens=True, so special tokens are invisible to users.
    Section 3 states this is 'typically set to True in production environments'. This is load-bearing for the stealthiness claim.
  • domain assumption The attacker has white-box access to the target VLM architecture and weights.
    Section 3 explicitly assumes full access, following prior work. This bounds the threat model but also means the attack requires model leakage.
  • domain assumption GPT-4o-generated prompt sets are diverse enough to generalize to arbitrary user prompts.
    Section 4.1 builds the prompt set by querying GPT-4o iteratively; the paper evaluates only on held-out prompts from the same distribution.
  • ad hoc to paper Feature-space to pixel-space inversion is approximately lossless.
    Section 4.3 and 5.2 state this empirically without reporting a direct comparison between feature-space and pixel-space attack success.
  • domain assumption GPT-4.1-mini text-only evaluation reflects true response quality relative to the image.
    The quality rubric asks whether the answer matches the image, but the evaluator prompt in Appendix C only supplies question and answer, not the image.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models." pith.science (2026). https://pith.science/paper/3CEAQBVE

@misc{pith2026250818805,
  author       = {Pith},
  title        = {Pith review of: Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3CEAQBVE}},
  note         = {Machine review of arXiv:2508.18805}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Vision-Language Models (VLMs) are increasingly deployed in real-world applications, but their high inference cost makes them vulnerable to resource consumption attacks. Prior attacks attempt to extend VLM output sequences by optimizing adversarial images, thereby increasing inference costs. However, these extended outputs often introduce irrelevant abnormal content, compromising attack stealthiness. This trade-off between effectiveness and stealthiness poses a major limitation for existing attacks. To address this challenge, we propose \textit{Hidden Tail}, a stealthy resource consumption attack that crafts prompt-agnostic adversarial images, inducing VLMs to generate maximum-length outputs by appending special tokens invisible to users. Our method employs a composite loss function that balances semantic preservation, repetitive special token induction, and suppression of the end-of-sequence (EOS) token, optimized via a dynamic weighting strategy. Extensive experiments show that \textit{Hidden Tail} outperforms existing attacks, increasing output length by up to 19.2$\times$ and reaching the maximum token limit, while preserving attack stealthiness. These results highlight the urgent need to improve the robustness of VLMs against efficiency-oriented adversarial threats. Our code is available at https://github.com/zhangrui4041/Hidden_Tail.

Figures

Figures reproduced from arXiv: 2508.18805 by Guowen Xu, Hongwei Li, Qingchuan Zhao, Rui Zhang, Tianli Yang, Wenbo Jiang, Yang Liu, Zihan Wang.

Figure 1
Figure 1. Figure 1: Examples of Hidden Tail attack. Compared to ex￾isting approaches, Hidden Tail achieves longer outputs without compromising stealthiness by introducing user-invisible special tokens as hidden tail. puts, thereby consuming substantial compute resources and potentially degrading service availability. Prior works on resource consumption attacks against VLMs, such as Verbose Images [3] and VLMInferSlow [29], pr… view at source ↗
Figure 2
Figure 2. Figure 2: Attak scenario. Phi-3.5-vision [21], adopt tightly coupled vision-language architectures, enabling unified reasoning across modalities and supporting complex tasks such as chart understanding [9] and visual programming [8]. Despite their impressive capa￾bilities, VLMs inherit the sensitivity to input perturbations from both vision and language components. This creates un￾derexplored attack surfaces, partic… view at source ↗
Figure 3
Figure 3. Figure 3: An overview of Hidden Tail attack. images can then be freely accessed by users for down￾stream VLM-powered applications, such as VQA bots and multimodal assistants [16]. They may also be indexed by multimodal RAG systems [30], crawled by web search en￾gines [35], or processed by document analysis tools [22], and subsequently retrieved during inference by VLM-based agents [5]. Once adopted, these images ind… view at source ↗
Figure 4
Figure 4. Figure 4: The effect of optimization iterations T on attack per￾formance. • Response Latency is to measure the average inference cost of all the testing samples. • Output length is the total number of tokens generated, including special tokens, and serves as a direct measure of the computational payload. To assess how well the attack hides from users, i.e., attack stealthiness, we measure two aspects of the output. … view at source ↗
Figure 5
Figure 5. Figure 5: Examples of Hidden Tail attack using different per￾turbation norms ε [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: GradCAM visualization of clean and adversarial im [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: A comparison of output length distributions for each attack method on all the target VLMs. Each subplot visualizes three [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Hiding in Plain Sight: An Effective Physical Adversarial Patch Attack against Visual-Infrared Fused Face Detection

    cs.CR 2026-07 conditional novelty 6.0

    A jointly optimized gradient-mask plus band-aid patch reportedly bypasses visible-infrared fused face detectors with >90% attack success in both digital and physical settings.

  2. Overthink-Triggered Slowdown Attacks on LVLM-Based Robotic Systems

    cs.CR 2026-07 unverdicted novelty 6.0

    Adversaries can use crafted scene text to trigger overthinking in LVLM-based robots, producing transferable slowdowns up to 6.96x latency amplification.

Reference graph

Works this paper leans on

40 extracted references · 28 canonical work pages · cited by 2 Pith papers · 3 internal anchors

  1. [1]

    Qwen2.5-VL Technical Report

    Alibaba. Qwen2.5-VL Technical Report. CoRR abs/2502.13923, 2025. 1, 2, 5, 7, 10

    Pith/arXiv arXiv 2025

  2. [2]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. InstructBLIP: Towards General- purpose Vision-Language Models with Instruction Tuning. In Annual Conference on Neural Information Processing Sys- tems (NeurIPS). NeurIPS, 2023. 2, 10

    work page 2023

  3. [3]

    Inducing high energy-latency of large vision-language models with verbose images

    Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, and Wei Liu. Inducing high energy-latency of large vision-language models with verbose images. In In- ternational Conference on Learning Representations (ICLR) ,

    work page

  4. [4]

    Denial-of-Service Poisoning Attacks Against Large Language Models

    Kuofeng Gao, Tianyu Pang, Chao Du, Yong Yang, Shu- Tao Xia, and Min Lin. Denial-of-Service Poisoning Attacks Against Large Language Models. CoRR abs/2410.10760 ,

    Pith/arXiv arXiv

  5. [5]

    Multi-Modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage

    Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaojian Ma, Tao Yuan, Yue Fan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, and Qing Li. Multi-Modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage. In International Conference on Learning Representations (ICLR), 2025. 3

    work page 2025

  6. [6]

    Gemini: A Family of Highly Capable Multimodal Models

    Google. Gemini: A Family of Highly Capable Multimodal Models. CoRR abs/2312.11805, 2023. 1

    Pith/arXiv arXiv 2023

  7. [7]

    Gemma 3 Technical Report

    Google. Gemma 3 Technical Report. CoRR abs/2503.19786,

    Pith/arXiv arXiv

  8. [8]

    Visual Program- ming: Compositional Visual Reasoning Without Training

    Tanmay Gupta and Aniruddha Kembhavi. Visual Program- ming: Compositional Visual Reasoning Without Training. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 14953–14962. IEEE, 2023. 1, 2, 10

    work page 2023

  9. [9]

    Fung, Haoyi Qiu, Mingyang Zhou, Shafiq Joty, Shih-Fu Chang, and Heng Ji

    Kung-Hsiang Huang, Hou Pong Chan, Yi R. Fung, Haoyi Qiu, Mingyang Zhou, Shafiq Joty, Shih-Fu Chang, and Heng Ji. From Pixels to Insights: A Survey on Automatic Chart Un- derstanding in the Era of Large Foundation Models. IEEE Transactions on Knowledge and Data Engineering , 2024. 1, 2, 10

    work page 2024

  10. [10]

    Adversarial Attacks Against Closed-Source MLLMs via Fea- ture Optimal Alignment

    Xiaojun Jia, Sensen Gao, Simeng Qin, Tianyu Pang, Chao Du, Yihao Huang, Xinfeng Li, Yiming Li, Bo Li, and Yang Liu. Adversarial Attacks Against Closed-Source MLLMs via Fea- ture Optimal Alignment. CoRR abs/2505.21494, 2025. 1

    Pith/arXiv arXiv 2025

  11. [11]

    Com- binational backdoor attack against customized text-to-image models

    Wenbo Jiang, Jiaming He, Hongwei Li, Guowen Xu, Rui Zhang, Hanxiao Chen, Meng Hao, and Haomiao Yang. Com- binational backdoor attack against customized text-to-image models. CoRR abs/2411.12389, 2024. 1

    work page arXiv 2024

  12. [12]

    Backdoor Attacks against Image-to-Image Networks

    Wenbo Jiang, Hongwei Li, Jiaming He, Rui Zhang, Guowen Xu, Tianwei Zhang, and Rongxing Lu. Backdoor attacks against image-to-image networks. CoRR abs/2407.10445 ,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    OverThink: Slowdown Attacks on Reasoning LLMs

    Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpin- ska, Mohit Iyyer, Amir Houmansadr, and Eugene Bagdasar- ian. OverThink: Slowdown Attacks on Reasoning LLMs. CoRR abs/2502.02542, 2025. 2

    arXiv 2025

  14. [14]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. BLIP: Bootstrapping Language-Image Pre-training for Uni- fied Vision-Language Understanding and Generation. In In- ternational Conference on Machine Learning (ICML) , pages 12888–12900. JMLR, 2022. 2, 10

    work page 2022

  15. [15]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In European Conference on Computer Vision (ECCV), pages 740–755. Springer, 2014. 5

    work page 2014

  16. [16]

    Visual Instruction Tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. In Annual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2023. 3

    work page 2023

  17. [17]

    Shikun Liu, Edward Johns, and Andrew J. Davison. End-to- End Multi-Task Learning with Attention. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1871–1880. IEEE, 2019. 4

    work page 2019

  18. [18]

    In-Context Watermarks for Large Lan- guage Models

    Yepeng Liu, Xuandong Zhao, Christopher Kruegel, Dawn Song, and Yuheng Bu. In-Context Watermarks for Large Lan- guage Models. CoRR abs/2505.16934, 2025. 6

    Pith/arXiv arXiv 2025

  19. [19]

    The Multi-Modal Fusion in Visual Ques- tion Answering: A Review of Attention Mechanisms

    Siyu Lu, Mingzhe Liu, Lirong Yin, Zhengtong Yin, Xuan Liu, and Wenfeng Zheng. The Multi-Modal Fusion in Visual Ques- tion Answering: A Review of Attention Mechanisms. PeerJ Computer Science, 9:e1400, 2023. 1, 2, 10

    work page 2023

  20. [20]

    Towards Deep Learn- ing Models Resistant to Adversarial Attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learn- ing Models Resistant to Adversarial Attacks. In International Conference on Learning Representations (ICLR), 2018. 5

    work page 2018

  21. [21]

    Phi-3 Technical Report: A Highly Capable Lan- guage Model Locally on Your Phone

    Microsoft. Phi-3 Technical Report: A Highly Capable Lan- guage Model Locally on Your Phone. CoRR abs/2404.14219,

    Pith/arXiv arXiv

  22. [22]

    DocVLM: Make Your VLM an Efficient Reader

    Mor Shpigel Nacson, Aviad Aberdam, Roy Ganz, Elad Ben Avraham, Alona Golts, Yair Kittenplon, Shai Mazor, and Ron Litman. DocVLM: Make Your VLM an Efficient Reader. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 29005–29015, 2025. 3

    work page 2025

  23. [23]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. 2023. 1, 5, 6, 10

    work page 2023

  24. [24]

    Visual Adver- sarial Examples Jailbreak Aligned Large Language Models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Hen- derson, Mengdi Wang, and Prateek Mittal. Visual Adver- sarial Examples Jailbreak Aligned Large Language Models. In AAAI Conference on Artificial Intelligence (AAAI) , pages 21527–21536. AAAI, 2024. 1

    work page 2024

  25. [25]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad- CAM: Visual Explanations from Deep Networks via Gradient- Based Localization. In IEEE International Conference on Computer Vision (ICCV), pages 618–626. IEEE, 2017. 7 8

    work page 2017

  26. [26]

    Sponge examples: Energy-latency attacks on neural networks

    Ilia Shumailov, Yiren Zhao, Daniel Bates, Nicolas Paper- not, Robert Mullins, and Ross Anderson. Sponge examples: Energy-latency attacks on neural networks. In 2021 IEEE Eu- ropean symposium on security and privacy (EuroS&P), pages 212–231. IEEE, 2021. 2

    work page 2021

  27. [27]

    From Show to Tell: A Survey on Deep Learning-Based Image Cap- tioning

    Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, and Rita Cucchiara. From Show to Tell: A Survey on Deep Learning-Based Image Cap- tioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):539–559, 2022. 1, 2, 10

    work page 2022

  28. [28]

    Promising the future, encoding the past: Ai hype and public media imagery

    Dominik Vrabi ˇc Dežman. Promising the future, encoding the past: Ai hype and public media imagery. AI and Ethics , 4(3):743–756, 2024. 2

    work page 2024

  29. [29]

    VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service

    Xiasi Wang, Tianliang Yao, Simin Chen, Runqi Wang, Lei Ye, Kuofeng Gao, Yi Huang, and Yuan Yao. VLMInfer- Slow: Evaluating the Efficiency Robustness of Large Vision- Language Models as a Service. CoRR abs/2506.15755, 2025. 1, 2, 5

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  30. [30]

    MMed-RAG: Versatile Multimodal RAG System for Medi- cal Vision Language Models

    Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. MMed-RAG: Versatile Multimodal RAG System for Medi- cal Vision Language Models. In International Conference on Learning Representations (ICLR), 2025. 1, 3

    work page 2025

  31. [31]

    MiMo-VL Technical Report

    Xiaomi. MiMo-VL Technical Report. CoRR abs/2506.03569,

    Pith/arXiv arXiv

  32. [32]

    Instruction backdoor attacks against customized {LLMs}

    Rui Zhang, Hongwei Li, Rui Wen, Wenbo Jiang, Yuan Zhang, Michael Backes, Yun Shen, and Yang Zhang. Instruction backdoor attacks against customized {LLMs}. In USENIX Security Symposium (USENIX Security) , pages 1849–1866. USENIX, 2024. 1

    work page 2024

  33. [33]

    Self-interpreting Adversarial Images

    Tingwei Zhang, Collin Zhang, John X. Morris, Eugene Bag- dasarian, and Vitaly Shmatikov. Self-Interpreting Adversarial Images. CoRR abs/2407.08970, 2024. 3, 5

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    Crabs: Consuming Resrouce via Auto-generation for LLM-DoS Attack under Black-box Settings

    Yuanhe Zhang, Zhenhong Zhou, Wei Zhang, Xinyue Wang, Xiaojun Jia, Yang Liu, and Sen Su. Crabs: Consuming Resrouce via Auto-generation for LLM-DoS Attack under Black-box Settings. CoRR abs/2412.13879, 2024. 2

    Pith/arXiv arXiv 2024

  35. [35]

    Vision Search Assistant: Empower Vision- Language Models as Multimodal Search Engines

    Zhixin Zhang, Yiyuan Zhang, Xiaohan Ding, and Xi- angyu Yue. Vision Search Assistant: Empower Vision- Language Models as Multimodal Search Engines. CoRR abs/2410.21220, 2024. 1, 3

    Pith/arXiv arXiv 2024

  36. [36]

    MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models. InIn- ternational Conference on Learning Representations (ICLR) ,

    work page

  37. [38]

    Fully meets expectations

    Scoring standards for each criterion (Important: All scores must be integers from 1 to 5.): - **5:** Excellent. Fully meets expectations. No major weaknesses. - **4:** Good. Minor weaknesses that do not seriously impact quality. - **3:** Fair. Some noticeable issues that reduce effectiveness. - **2:** Poor. Serious flaws or missing key aspects. - **1:** V...

  38. [39]

    For the criterion, provide: - A score (from 1 to 5) - An explanation of why you gave this score

    work page

  39. [40]

    answer_correctness_score

    Output your evaluation in the following JSON format: 10 Table 4: Evaluation results of Hidden Tail Attack on Qwen2.5-VL and MiMo-VL under varying perturbation norm ε. Model ε ASR ( ↑) Latency ( ↑) Output Length (↑) Visible Length (↓) Response Quality ( ↑) Correctness Clarity Quality Avg. Qwen2.5-VL 8/255 0.00 5.33 163.99 162.98 4.09 4.88 4.98 4.65 16/255 ...

    work page 2048

  40. [2024]

    Early VLMs, such as BLIP [14], InstructBLIP [2], and MiniGPT-4 [36], introduce image encoders (e.g., Vision Transformers) paired with large language models (LLMs) as decoders

    2, 10 9 A Related Work A.1 Vision-Language Models Vision-Language Models (VLMs) are multimodal models that can jointly process visual and textual modalities to perform mul- timodal tasks such as image captioning [27] and visual question answering (VQA) [19]. Early VLMs, such as BLIP [14], InstructBLIP [2], and MiniGPT-4 [36], introduce image encoders (e.g...

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.