Pith. sign in

REVIEW 2 major objections 5 minor 37 references

AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free defense that masks text-irrelevant image patches and asks the model to analyze intent before answering lifts defense success rates on LVLM jailbreak benchmarks from an average of 52.4% to 81.7%.

desk verdict Training-free LVLM defense with real safety gains, but the paper's own ablation contradicts its masking mechanism and the main table appears to report the random-masking variant. read the letter →

arxiv 2505.24519 v1 pith:IHT6LXAL submitted 2025-05-30 cs.CV cs.CL

Yuqi Zhang , Yuchun Miao , Zuchao Li , Liang Ding This is my paper
classification cs.CVcs.CL
keywords largevision-languagemodelsjailbreakdefenseinference-timesafetyadversarialimageperturbationimage-textcorrelationmaskingintentionanalysismultimodalpromptmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AMIA is an inference-only defense meant to stop jailbreak attacks on large vision-language models: inputs crafted to make a model produce harmful content. It makes two moves: automatically black out the image patches least related to the text prompt, then ask the model to state the user's underlying intention in a structured way before answering. The paper reports that AMIA raises average defense success rates from 52.4% to 81.7% across four jailbreak benchmarks and three LVLMs, while general accuracy falls by about 2%. The appeal is a training-free, single-pass safety layer that can be dropped onto existing multimodal models without retuning them.

What carries the argument

The carrying object is the image-text correlation score $s_i = \cos(\varphi(v_i), \varphi(T))$, computed with the VisRAG-Ret encoder $\varphi$, where $v_i$ are the $N=16$ image patches and $T$ is the text request. The $K=3$ patches with the lowest scores are set to black, producing the masked image $\tilde{V}$ that is meant to destroy adversarial perturbation patterns while sparing text-relevant visual content. A second mechanism, mask-guided joint intention analysis, concatenates the masked image with an instruction asking the model for the essential intention of the request and then a final response, forcing a structured two-part output so the model's safety behavior is engaged before generation.

What would settle it

Run AMIA and a random-masking control with identical K=3, N=16 on VisualAdv-Harmbench and MMVP; if random masks match or beat correlation-selected masks on both safety and utility, then text-similarity selection is not the active ingredient, and the paper's mechanism would require revision.

Watch

Extended reading notes

Core claim

The central claim is that multimodal jailbreak robustness can be substantially improved without retraining by pairing a correlation-driven input mutation with an explicit intention-analysis step. AMIA splits the input image into patch embeddings, scores each patch's cosine similarity to the text encoding, masks the K=3 least similar patches, and then prompts the LVLM to output an '[INTENTION ANALYSIS]' line and a '[FINAL RESPONSE]' in a single inference. On FigStep, MMSafetyBench, VisualAdv-Harmbench, and AdvBench-constrain64, this raises average DSR across models from 52.4% to 81.7%, with the adversarial subsets reaching 91.1% DSR and a 3.26 average harmlessness score. On MMVP, AI2D, and MMStar, accuracy drops by roughly 2% on average, which the paper reads as preservation of general helpfulness.

Load-bearing premise

Adversarial perturbation in these jailbreak images is carried by patches that are less text-relevant, so zeroing out the three least similar patches in every image disrupts the attack while leaving general visual understanding intact.

Editorial extensions

If this is right

  • Deploying AMIA on an LVLM raises defense success rates on jailbreak benchmarks from 52.4% to 81.7% on average without retraining or fine-tuning.
  • The defense works for both prompt-manipulation jailbreaks, such as FigStep and MMSafetyBench, and optimization-based adversarial image attacks, such as VisualAdv-Harmbench and AdvBench-constrain64.
  • The structured '[INTENTION ANALYSIS]' / '[FINAL RESPONSE]' output lets the model inspect its intent and make refusal explicit before it commits to a response.
  • General-purpose accuracy drops only about 2% on MMVP, AI2D, and MMStar with K=3 and N=16, suggesting an acceptable safety-utility trade-off.
  • Single-pass inference keeps overhead modest, about 14%, so the defense is usable in latency-sensitive deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If correlation-based masking works by removing adversarial structure rather than by choosing semantically irrelevant patches, then a simpler random-mask variant with the same K might match AMIA's safety; the paper's own ablation numbers may be read to leave this open.
  • A direct head-to-head with random masking under matched settings would clarify whether semantic selection or mere information removal drives the safety gain.
  • The same mechanism could generalize to input-adaptive K, choosing the mask count per image from the similarity distribution instead of a fixed K=3.
  • Because AMIA works at inference time and is orthogonal to fine-tuning-based alignment, combining it with safety fine-tuning could stack gains, though the paper does not test that combination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces AMIA, a training-free, inference-only defense for large vision-language models (LVLMs) that combines (1) an image-text correlation-driven automatic masking mechanism that blacks out the K image patches least similar to the input text, and (2) a mask-guided joint intention analysis prompt that asks the LVLM to output an intention analysis followed by a final response. The authors report consistent defense success rate (DSR) improvements across three LVLMs and four jailbreak benchmarks, with average DSR rising from 52.4% to 81.7%, while general accuracy on three benchmarks drops by only about 2%. They also report a component ablation (Table 3) and sensitivity analyses for hyperparameters K and N.

Significance. If the results hold, the paper would offer a practical, lightweight defense against both prompt-manipulation and optimization-based jailbreak attacks on LVLMs, which is a timely and useful contribution. The strengths include the breadth of evaluated models and benchmarks, the open release of code, and the clarity of the proposed inference-only design. However, the core mechanistic claim—that correlation-driven masking selectively removes adversarial perturbations while preserving useful visual information—is directly contradicted by the paper's own ablation table. Because the main results in Table 1 appear to match the random-masking condition rather than the proposed automatic-masking condition, the central contribution is currently unsupported by the data as presented. The paper's significance depends on resolving this internal inconsistency, which requires either corrected tables, additional experiments, or a reframed claim.

major comments (2)
  1. [Section 5, Table 3 vs. Tables 1-2] Additionally, the Section 5 text states 'the comparison with random masking shows that our image-text correlation-based masking strategy better preserves the model's general helpfulness,' which is directly opposite to the data in Table 3. This is not a presentation nit but a fundamental mismatch between the claimed mechanism and the reported evidence.
  2. [Section 2.1, Eq. (2) and Section 5] The paper assumes that adversarial perturbations in jailbreak images are carried by image patches with low similarity to the input text, so that masking the K least similar patches disrupts the attack while preserving useful information. However, no direct evidence is provided for this localization assumption. The only comparison in the paper, Table 3, shows random masking performs at least as well as correlation-driven masking on both safety and utility. Therefore, the observed defense improvement could be explained by generic input corruption combined with intention analysis, rather than by the proposed 'text-irrelevant patch' mechanism. The authors should either (i) provide patch-level attribution experiments showing that the masked patches coincide with the adversarial perturbation, (ii) compare auto vs. random masking across multiple K values and multiple random seeds with confidence intervals, or (iii) revise the paper's narrative to claim only that masking plus intention analysis improves safety, without attributing the mechanism to correlation-driven selection.
minor comments (5)
  1. [Table 2] The labels '/toggle-off' and '/toggle-on' in Table 2 are never defined in the text or caption; the authors should explain what is being toggled (presumably the masking component) to make the table interpretable.
  2. [Section 1] The introduction claims 'a modest inference overhead of ~14%,' but no measurement, table, or figure in the experimental section reports inference time or overhead. Please add a quantitative timing comparison or remove the specific number.
  3. [Appendix A.4] The definition of DSR as the proportion of responses 'rated as low risk (i.e., assigned a score of 1, 2, or 3)' is confusing given that the harmfulness score ranges from 1 to 5 with higher scores indicating more harm; a score of 3 would typically be moderate, not low risk. The threshold should be clarified to avoid ambiguity.
  4. [Section 5 / Appendix A.5] The sensitivity analysis of N is not fully specified: the statement that K is set to match the optimal masking ratio from Figure 4(a) is vague, and Appendix A.5 lists K=1,2,3,5,7 for √N=2,3,4,5,6 without showing the derived masking ratios. Please make the relationship explicit.
  5. [Throughout] There are several typos and inconsistent naming conventions, such as 'LLaVA' vs. 'LLaVA-v1.5' and 'averagingly' (which should be 'on average'). A careful proofread is recommended.

Circularity Check

1 steps flagged · score 4.0 of 10

Core safety results are externally benchmarked, but K is tuned on the same MMVP and VisualAdv-Harmbench data later reported as headline results, making one claimed trade-off partially in-sample; Table 3 also undercuts the correlation-driven masking mechanism, though that is an inconsistency rather than a definitional circle.

  1. fitted input called prediction [Section 5 (Sensitivity analysis of K), Section 3 (Experimental Details), Tables 1 and 2]
    "Sensitivity analysis of K. Figure 4(a) presents a sensitivity analysis of K, the number of masked image patches, on the general dataset MMVP and adversarial dataset VisualAdv-Harmbench ... To balance safety and utility, we set K=3 as the default."

    K=3 is chosen by inspecting DSR and MMVP accuracy on VisualAdv-Harmbench and MMVP, and the same datasets are then reported as evaluation results in Tables 1 and 2 and summarized in the abstract's 'only a 2% average accuracy drop' claim. The reported safety-utility trade-off on these benchmarks is therefore an in-sample description of the objective used to select K, not an out-of-sample prediction of the method's behavior.

full rationale

The central defense claim is not definitionally circular: AMIA's DSR improvements are measured on external jailbreak benchmarks (FigStep, MMSafetyBench-TYPO+SD, AdvBench-constrain64, VisualAdv-harmbench) against independent baselines (Direct, Self-Reminder, ECSO), so the headline 'from 52.4% to 81.7%' is an empirical comparison rather than a restatement of the method's definition. The intention-analysis component is adapted from the authors' prior COLING work (Zhang et al., 2025) and is a self-citation, but it is ablated in Table 3 and combined with a new masked-image variant, so the self-citation is not the load-bearing argument. The one concrete circularity-adjacent practice is the hyperparameter selection: K=3 is selected using VisualAdv-Harmbench and MMVP in Section 5, and those same benchmarks supply Tables 1 and 2 and the '2% average accuracy drop' claim, so that particular trade-off is partly fitted rather than predicted. Separately, Table 3 fails to support the proposed correlation-driven masking mechanism: with intention analysis, random masking gives MMVP 63.0 and DSR 89.5, while automatic correlation-driven masking gives MMVP 58.7 and DSR 88.9, contradicting the text that the correlation-based strategy 'better preserves the model's general helpfulness'; this is an internal inconsistency and a correctness risk, but not a circular definition, so it does not raise the circularity score beyond the fitted-K issue. Overall, the paper retains independent content and is not a self-referential derivation, but the in-sample K choice warrants a partial-circularity score of 4.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirical assumptions: that similarity-based masking disrupts adversarial structure, that the VisRAG-Ret encoder's similarity reflects task relevance for the target LVLM, that ChatGPT judge scores are valid safety labels, and that the chosen hyperparameters (N=16, K=3) generalize across datasets. No formal proofs or independent verification are provided.

free parameters (2)
  • K (number of masked image patches) = 3
    Chosen via sensitivity analysis on VisualAdv-Harmbench DSR and MMVP accuracy (Figure 4a); fixed for all models and benchmarks.
  • N (number of image patches) = 16
    Set for simplicity and reproducibility; sensitivity analysis (Figure 4b) uses matched K values at a similar masking ratio.
assumptions (4)
  • domain assumption Masking the K image patches with lowest cosine similarity to the input text disrupts adversarial perturbations while preserving useful visual information.
    Invoked in Section 2.1; the only support is the ablation, which is internally inconsistent.
  • domain assumption The VisRAG-Ret encoder's cosine similarities align with semantic relevance for the LVLM being defended.
    Section 2.1 uses VisRAG-Ret to choose patches; no analysis connects its similarity space to the target model's visual semantics.
  • domain assumption ChatGPT-based harmfulness annotations are valid for DSR and Safety metrics.
    Section 3 and Appendix A.4 use a ChatGPT judge; no human validation is reported.
  • domain assumption The intention-analysis prompt itself does not bias safety outcomes independently of the analysis it elicits.
    The prompt explicitly asks the model to 'pay special attention to safety, ethical, and legal dimensions' (Figure 3), which may cue refusal regardless of the intention analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders." pith.science (2026). https://pith.science/paper/IHT6LXAL

@misc{pith2026250524519,
  author       = {Pith},
  title        = {Pith review of: AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IHT6LXAL}},
  note         = {Machine review of arXiv:2505.24519}
}
read the original abstract

We introduce AMIA, a lightweight, inference-only defense for Large Vision-Language Models (LVLMs) that (1) Automatically Masks a small set of text-irrelevant image patches to disrupt adversarial perturbations, and (2) conducts joint Intention Analysis to uncover and mitigate hidden harmful intents before response generation. Without any retraining, AMIA improves defense success rates across diverse LVLMs and jailbreak benchmarks from an average of 52.4% to 81.7%, preserves general utility with only a 2% average accuracy drop, and incurs only modest inference overhead. Ablation confirms both masking and intention analysis are essential for a robust safety-utility trade-off.

Figures

Figures reproduced from arXiv: 2505.24519 by the authors.

Figure 1
Figure 1. The framework of AMIA consisting of au￾tomatic masking and intention analysis mechanisms. shows that converting visual inputs into textual cap￾tions can reactivate the safety mechanisms inher￾ited from the LLM backbone. However, it only han￾dles the visual modality and overlooks the jointly harmful semantics in image-text inputs, limiting its effectiveness in more complex multimodal jail￾break scenarios, as discusse… view at source ↗
Figure 2
Figure 2. Illustration of image-text correlation-driven automatic masking mechanism. while preserving useful visual information for gen￾eral tasks. Then, we design a mask-guided image￾text joint intention analysis mechanism, which en￾courages LVLMs to identify and express potential harmful intention in text, thus reactivating the in￾trinsic safety of the LLM backbones within LVLMs. The framework of AMIA is illustrated in [PI… view at source ↗
Figure 3
Figure 3. Detailed intention analysis instruction. Formally, the mask-guided joint intention analy￾sis process can be written as: [Rintention, Rf inal] = LVLM  V , I ˜ rec ⊕ T  , (4) where V˜ is the masked image in Equation 3, Irec is the intention analysis instruction, T is the user’s textual input, and ⊕ denotes string concatenation. The outputs Rintention and Rf inal are the LVLM’s generated intention and final response,… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: AMIA’s performance on Llava-v1.5-13B with varying K and N. Component ablation of AMIA. We perform ab￾lation studies on a jailbreak dataset, VisualAdv￾Harmbench, and a general dataset, MMVP, to as￾sess the individual impact of automatic masking and intention analysis me…
Figure 5
Figure 5. Figure 5: Harmfulness judge prompt [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Response of Llava-v1.5-13B on the MMSafetyBench-TYPO+SD dataset [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Response of Qwen2-VL-7B-Instruct on the AdvBench-constrain64 dataset [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 10 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    arXiv 2023

  4. [4]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. https://arxiv.org/abs/2308.12966 Qwen-vl: A frontier large vision-language model with versatile abilities . Preprint, arXiv:2308.12966

    arXiv 2023

  5. [5]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and 1 others. 2024 a . https://openreview.net/forum?id=evP9mxNNxJ Are we on the right way for evaluating large vision-language models? In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    work page 2024

  6. [6]

    Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. 2024 b . https://arxiv.org/abs/2311.10081 Dress: Instructing large vision-language models to align and interact with humans via natural language feedback . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 14239--14250

    arXiv 2024

  7. [7]

    Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, and 1 others. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    work page 2023

  8. [8]

    Yi Ding, Bolian Li, and Ruqi Zhang. 2024. https://arxiv.org/abs/2410.06625 Eta: Evaluating then aligning safety of vision language models at inference time . Preprint, arXiv:2410.06625

    arXiv 2024

Show all 37 references
  1. [9]

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, and 1 others. 2024. https://dl.acm.org/doi/abs/10.1145/3664647.3685520 Vlmevalkit: An open-source toolkit for evaluating large multi-modality models . In P...

    2024

  2. [10]

    Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, and 1 others. 2022. https://arxiv.org/abs/2209.07858 Red teaming language models to reduce harms: Methods, scaling behaviors, and l...

    2022 arXiv

  3. [11]

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2025. https://arxiv.org/abs/2311.05608 Figstep: Jailbreaking large vision-language models via typographic visual prompts . In Proceedings of the AAAI Conference on Artifi...

    2025 arXiv

  4. [12]

    Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. 2024. https://arxiv.org/abs/2403.09572 Eyes closed, safety on: Protecting multimodal llms via image-to-text transformation . In Proceedings of the European Conference...

    2024 arXiv

  5. [13]

    Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. https://arxiv.org/abs/2406.18495 Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms . arXiv preprint arXiv:2406.18495

    2024 arXiv

  6. [14]

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. https://arxiv.org/abs/1603.07396 A diagram is worth a dozen images . In 14th European Conference on Computer Vision, ECCV 2016, pages 235--251

    2016 arXiv

  7. [15]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 a . https://arxiv.org/abs/2310.03744 Improved baselines with visual instruction tuning . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024 arXiv

  8. [16]

    Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. 2024 b . https://arxiv.org/abs/2311.17600 Mm-safetybench: A benchmark for safety evaluation of multimodal large language models . In Proceedings of the European Conference on Computer Vision, pages 386--403

    2024 arXiv

  9. [17]

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. https://arxiv.org/abs/1706.06083 Towards deep learning models resistant to adversarial attacks . In Proceedings of the International Conference on Learning Representations

    2018 arXiv

  10. [18]

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, and 1 others. 2024. https://arxiv.org/abs/2402.04249 Harmbench: a standardized evaluation framework for automated red teaming and robust refusal . In Proc...

    2024 arXiv

  11. [19]

    Yuchun Miao, Sen Zhang, Liang Ding, Rong Bao, Lefei Zhang, and Dacheng Tao. 2024. https://arxiv.org/abs/2402.09345 Inform: Mitigating reward hacking in rlhf via information-theoretic reward modeling . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024 arXiv

  12. [20]

    Yuchun Miao, Sen Zhang, Liang Ding, Yuqi Zhang, Lefei Zhang, and Dacheng Tao. 2025. https://arxiv.org/abs/2501.19358 The energy loss phenomenon in rlhf: A new perspective on mitigating reward hacking . arXiv preprint arXiv:2501.19358

    2025 arXiv

  13. [21]

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. https://arxiv.org/abs/2306.13213 Visual adversarial examples jailbreak aligned large language models . In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38...

    2024 arXiv

  14. [22]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. https://arxiv.org/abs/2103.00020 Learning transferable visual models from natural language supervision . In P...

    2021 arXiv

  15. [23]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. 2022. https://arxiv.org/abs/2112.10752 High-resolution image synthesis with latent diffusion models . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1...

    2022 arXiv

  16. [24]

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024. https://arxiv.org/abs/2401.06209 Eyes wide shut? exploring the visual shortcomings of multimodal llms . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages ...

    2024 arXiv

  17. [25]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models . Preprint, arXi...

    2023 arXiv

  18. [26]

    Han Wang, Gang Wang, and Huan Zhang. 2024 a . https://arxiv.org/abs/2411.16721 Steering away from harm: An adaptive approach to defending vision language model against jailbreaks . Preprint, arXiv:2411.16721

    2024 arXiv

  19. [27]

    Ru Wang, Selena Song, Liang Ding, Mingming Gong, Yusuke Iwasawa, Yutaka Matsuo, and Jiaxian Guo. 2025 a . https://openreview.net/forum?id=kY8k5iHMmu Mma: Benchmarking multi-modal large language model in ambiguity contexts . In ICLR 2025 Workshop on Navigating and Addressing Da...

    2025

  20. [28]

    Taowen Wang, Zheng Fang, Haochen Xue, Chong Zhang, Mingyu Jin, Wujiang Xu, Dong Shu, Shanchieh Yang, Zhenting Wang, and Dongfang Liu. 2024 b . https://link.springer.com/chapter/10.1007/978-981-96-0151-6_1 Large vision-language model security: A survey . In Proceedings of the I...

    2024 doi

  21. [29]

    Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. 2025 b . https://ojs.aaai.org/index.php/AAAI/article/view/32852 Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large langua...

    2025

  22. [30]

    Wenbin Wang, Yongcheng Jing, Liang Ding, Yingjie Wang, Li Shen, Yong Luo, Bo Du, and Dacheng Tao. 2025 c . https://arxiv.org/abs/2503.01222 Retrieval-augmented perception: High-resolution image perception meets visual rag . arXiv preprint arXiv:2503.01222

    2025 arXiv

  23. [31]

    Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. https://www.nature.com/articles/s42256-023-00765-8 Defending chatgpt against jailbreak attack via self-reminders . Nature Machine Intelligence, 5(12):1486--1496

    2023

  24. [32]

    Mang Ye, Xuankun Rong, Wenke Huang, Bo Du, Nenghai Yu, and Dacheng Tao. 2025. https://arxiv.org/abs/2502.14881 A survey of safety on large vision-language models: Attacks, defenses and evaluations . Preprint, arXiv:2502.14881

    2025 arXiv

  25. [33]

    Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, and 1 others. 2025. https://arxiv.org/abs/2410.10594 Visrag: Vision-based retrieval-augmented generation on multi-modality documents . In Proceedings of the Internat...

    2025 arXiv

  26. [34]

    Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Xiaofei Xie, Yang Liu, and Chao Shen. 2023. https://arxiv.org/abs/2312.10766 Jailguard: A universal detection framework for llm prompt-based attacks . Preprint, arXiv:2312.10766

    2023 arXiv

  27. [35]

    Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao. 2025. https://aclanthology.org/2025.coling-main.199/ Intention analysis makes LLM s a good jailbreak defender . In Proceedings of the 31st International Conference on Computational Linguistics

    2025

  28. [36]

    Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Jialing Tao, YueFeng Chen, Hui Xue, and Xingxing Wei. 2025. https://arxiv.org/abs/2501.04931 Jailbreaking multimodal large language models via shuffle inconsistency . arXiv preprint arXiv:2501.04931

    2025 arXiv

  29. [37]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. https://arxiv.org/abs/2307.15043 Universal and transferable adversarial attacks on aligned language models . Preprint, arXiv:2307.15043

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.