Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adversarial image perturbations that force attention-sink behavior can inflate hallucinated objects, attributes, and relationships in MLLM responses without hurting fluency, and the perturbations transfer to black-box models and…

desk verdict A new and plausible hallucination attack with broad transfer results, but the title overclaims 'only attention sink' and the evaluation lacks ablations, controls, and error bars. read the letter →

arxiv 2501.15269 v1 pith:F4KWQSEI submitted 2025-01-25 cs.LG cs.CRcs.CV

Yining Wang , Mi Zhang , Junjie Sun , Chenyue Wang , Min Yang , Hui Xue , Jialing Tao , Ranjie Duan
show 1 more author
Jiexi Liu
This is my paper · ORCID
classification cs.LGcs.CRcs.CV
keywords hallucinationattackattentionsinkmulti-modallargelanguagemodelsadversarialperturbationblack-boxtransferabilityinstructiontuningmitigationvisualquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the hallucination problem in multi-modal large language models (MLLMs) is not only a natural failure mode but an exploitable vulnerability: small, nearly invisible perturbations to an input image can make an MLLM produce substantially more fabricated objects, attributes, and relationships while the text stays fluent and useful. The claimed mechanism is the attention sink, a token that receives a columnar pattern of abnormally high attention and aggregates global input information; the authors trace its origin to the two-segment response pattern in instruction-tuning data, where ground-truth captions drift from image-bound description into associative filler. The attack optimizes an epsilon-bounded perturbation with two losses, one forcing columnar attention onto a chosen token and one raising that token's embedding similarity to the global input, and reports up to 10.90% and 12.74% increases in hallucinated sentences and words. If these results hold, any pipeline that accepts user-uploaded images, such as AI assistants, customer service, medical aids, or driving aids, can be fed images that look normal but systematically corrupt the model's factual reporting.

What carries the argument

The attention sink is the central object: a generated token, usually a word like 'the' or a comma, that receives disproportionately high attention from all subsequent tokens, forming a columnar pattern in the attention map and acting as a compressed global-context summary. The paper's mechanism has two parts: sink tokens appear at the point where image-text relevance falls, and their embeddings carry higher-than-normal cosine similarity to the mean embedding of the visual and textual input tokens. The attack machinery is a two-term objective: $L_{\mathrm{attn}}$ uses cross-entropy to force later tokens to attend to a chosen potential sink token, and $L_{\mathrm{emb}}$ uses a hinge loss to raise that token's embedding similarity to the global input mean while keeping the perturbation inside an $\ell_p$ ball. A multi-round optimizer re-identifies the potential sink token on each response and updates the perturbation by signed gradients. This mechanism is what makes the attack dynamic, since it needs no fixed target response, and transferable, because it targets a structural behavior rather than a specific output phrase.

What would settle it

Take a fixed image and the $8/255$ budget; add a perturbation of the same magnitude optimized to maximize image-feature corruption or response drift without any attention-loss or embedding-loss term. If the hallucinated-sentence ratio rises as much as it does in Table 2, attention sink is not the operative mechanism; if it stays near the clean level, the sink objective is confirmed as the cause.

Watch

Extended reading notes

Core claim

The core claim is that attention-sink behavior can be adversarially induced, and that inducing it is sufficient to worsen object, attribute, and relationship hallucination in MLLMs across decoding strategies. The paper shows that instruction-tuned MLLMs reproduce a two-segment response pattern inherited from training data: an image-faithful opening followed by loosely associated content, with the attention sink appearing exactly at the CLIPScore turning point. Sink tokens are mostly non-content tokens whose embeddings are unusually similar to the mean embedding of the multi-modal input, which the authors interpret as aggregation of misleading global context. Their attack identifies the token most likely to become a sink by cosine similarity, then minimizes an attention cross-entropy loss over a localized attention window plus a hinge loss that pushes the chosen token's embedding closer to the global mean, under an $\ell_p$ budget up to $8/255$. They report a single-response hallucinated-word ratio as high as 75.74%, black-box increases up to 10.90 percentage points in sentence ratio and 12.74 in word ratio, a VQA accuracy drop of up to 7.67 points, successful transfer to GPT-4o mini and Gemini 1.5 Flash, and bypassing of four mitigation families plus an adaptive early-stopping defense.

Load-bearing premise

The load-bearing premise is that forcing columnar attention on a chosen token and raising that token's embedding similarity to the global input is what causes the extra hallucination; the paper itself labels this aggregation account as speculation, so if generic feature degradation from the perturbation is the real driver, the title claim and the proposed defense implications collapse.

Editorial extensions

If this is right

  • If the central claim is right, an image perturbed by as little as $2/255$ can raise the hallucination rate on a white-box model, and at $8/255$ the ratio of hallucinated words in a single caption can reach 75.74% while sentence length stays roughly unchanged.
  • Adversarial images made on one open MLLM transfer to other open MLLMs, with the largest gains on models sharing the same visual encoder architecture; black-box HSR and HWR increases reach 10.90 and 12.74 percentage points.
  • The attack degrades question answering by up to 7.67 accuracy points on OK-VQA, so visual understanding, not just captioning, is affected.
  • Existing hallucination mitigations, including OPERA and VCD in decoding, Less is More and LRV-Instruction in retraining, and LURE in post-hoc revision, can be bypassed, with hallucination returning to or exceeding the undefended level.
  • Commercial APIs are also vulnerable: adversarial inputs crafted on open models raise hallucinated-word ratios on GPT-4o mini and Gemini 1.5 Flash by 3.40% and 5.32%, respectively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the strongest practical test of the mechanism would be a mechanism ablation: optimize a perturbation with the same budget but no attention-sink loss, maximizing instead generic image-feature drift; if hallucination rises equally, the attention-sink story is not what carries the attack.
  • I infer that the attention-sink account points to a cheap data-side hardening not tested in the paper: since the two-segment response pattern comes from instruction-tuning tails, truncating or reweighing the associative final sentences in those datasets may reduce both natural and attack-induced hallucination.
  • I infer a deployable detection side-channel: monitoring for columnar attention peaks or abruptly high embedding similarity between a generated token and the mean input embedding could flag adversarial images even when the text looks fluent, since the attack explicitly maximizes both quantities.
  • I infer that the GPT-4-based hallucination metric may be a limiting factor for the precise numbers: a fully human-labeled replication on an open image set would clarify whether the reported 10.90 and 12.74 percentage-point increases overstate verifiable fabrication, although the paper's own human alignment study reports comparable directions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper studies attention-sink phenomena in multi-modal large language models (MLLMs) and proposes an adversarial visual attack that, by optimizing two loss terms—columnar attention on a selected token (Eq. 8) and embedding similarity between that token and the mean global input representation (Eq. 9)—aims to increase hallucinated objects, attributes, and relationships in generated captions. The authors report consistent HSR/HWR increases on four open-source MLLMs, black-box transferability across architectures, transfer to GPT-4o mini and Gemini 1.5 Flash, partial success against mitigation methods such as OPERA and VCD, and preserved response quality. The core empirical claim is that epsilon-bounded perturbations (2/255 to 8/255) substantially increase GPT-4-judged hallucination while random Gaussian noise of the same magnitude does not.

Significance. If the central claim holds, the paper makes a useful contribution to the security analysis of MLLMs: it introduces a gradient-based attack that does not require predefined target responses, and it connects an observable internal phenomenon (columnar attention sinks) to a concrete failure mode. The transfer experiments to closed commercial APIs and the evaluation against several mitigation families are valuable, and the finding that random noise does not reproduce the effect is a meaningful control. The code release statement and the human-alignment study are also positive elements. However, the mechanistic attribution of the attack to attention-sink formation is explicitly labeled as speculation, no ablation isolates the two loss terms, and the evaluation relies on a single GPT-4 judge with a limited alignment study and no significance testing. These issues currently prevent the paper from fully supporting its title-level claim that the effect is caused by attention-sink manipulation.

major comments (5)
  1. [§3.2(2) and §4.3.3, Eqs. (8)-(10)] The causal mechanism is not established. Section 3.2(2) explicitly states 'We speculate that...' for the link between misleading global aggregation and hallucinated output, and the attack objective in Eqs. (8)-(10) optimizes two proxies—columnar attention and embedding similarity to the global input—without any ablation isolating Lattn alone, Lemb alone, or the combination. As written, the observed hallucination increase could be caused by generic feature degradation rather than by the formation of attention sinks. I recommend adding ablations over the two losses and, crucially, a control that optimizes an attention-independent objective (e.g., cross-entropy loss on the visual encoder or a feature-divergence loss) at the same perturbation budget; without such a control, the phrase 'with Only Attention Sink' in the title is not supported.
  2. [§5.6, Table 7] The baseline comparison is too weak to support the claim that the attack works through attention-sink manipulation. Random Gaussian noise is not an adversarial control, and several random-noise cells produce increases comparable to the attack: for InstructBLIP, δ=5/255 gives HWR +5.29% versus the attack's +5.91% at ε=5/255 (Table 7); for MiniGPT-4, δ=2/255 gives HSR +5.22% versus the attack's +6.46% at ε=2/255. The text asserts that random perturbation 'shows no significant attack effect,' but no significance test is provided. I recommend reporting per-image paired tests and comparing against at least one existing adversarial attack or an attention-independent adversarial objective.
  3. [Tables 2-6 and Fig. 7] All headline hallucination metrics are reported as point estimates with no error bars, confidence intervals, or statistical tests. The central quantitative claims—for example, '+10.90% HSR and +12.74% HWR' in Table 3 for LLaVA-1.5 to Shikra—could be within run-to-run or image-level variance, especially given that some adjacent cells in the same table show negative or near-zero deltas (e.g., InstructBLIP to LLaVA-1.5 at ε=5/255: -0.20% HSR, -0.20% HWR). I request standard errors or bootstrap confidence intervals over the evaluation images, and paired significance tests for at least the main white-box and black-box tables.
  4. [§J, Table 15] The GPT-4 assisted evaluation is the sole outcome measure for the main hallucination tables, but the alignment study covers only two models (MiniGPT-4 and LLaVA-1.5) and four response sets, reports averaged human scores without inter-annotator agreement (e.g., Cohen's kappa), and does not examine black-box or mitigation conditions. Given that the paper's conclusions depend entirely on the reliability of this judge, the alignment evidence is insufficient. I recommend adding agreement statistics and human evaluation on at least one black-box transfer condition and one mitigation condition, or supplementing the main tables with an automatic metric (e.g., a rule-based object/attribute consistency check) for at least one model.
  5. [§5.3.1, Table 4] The QA accuracy results do not consistently support the claim of 'strong transferability.' Several cells show accuracy increases under attack, including InstructBLIP→MiniGPT-4 at ε=2/255 (+1.67), LLaVA-1.5→MiniGPT-4 at ε=5/255 (+3.00), and Shikra→InstructBLIP at ε=2/255 (+1.33), while other cells show decreases of comparable magnitude. The text attributes the increases to 'limitations of the manually labeled answers,' but no analysis is provided to substantiate this. I recommend either restricting the transferability claim to the captioning task, where the direction is more consistent, or reporting error bars and a breakdown that addresses the positive-delta cells.
minor comments (6)
  1. [Abstract vs. Ethics/Open Science] The abstract states that code is available at a Hugging Face URL, while the Ethics section says access is granted only upon request and the Open Science section says the code is 'not hosted on a platform with persistent access.' These statements are inconsistent and should be reconciled.
  2. [Table 1] The row labels in Table 1 appear to be swapped: the 'Visual Encoder' row lists Vicuna-7b and LLaMA-7b, while the 'LLM Backbone' row lists EVA-ViT-g/14 and CLIP-ViT-L/14. As printed, the table contradicts the model descriptions in Section 2.1.
  3. [Figures 14-17] The prompt text in Figures 14-17 reads 'Pease describe this image in detail.' and should read 'Please describe this image in detail.'
  4. [§5.4, Figs. 8 and 13] The response-quality results are presented only as figures without numeric tables or error bars; given that the stealthiness claim depends on quality being preserved, adding the underlying numbers and variability would strengthen the presentation.
  5. [§4.3.3 and Appendix B] The hyperparameters α, σ, the intermediate layer l, and Smax are described as set 'following an ablation study,' but no ablation results for these choices are presented. I suggest reporting sensitivity curves for α and σ and for the selected layer.
  6. [§1, Contributions] The phrase 'the first hallucination attack' is stronger than necessary and is not supported by the related-work discussion, which already describes attacks that degrade MLLM performance. I recommend softening this to 'a new attack' or 'the first attack, to our knowledge, that explicitly targets attention-sink behavior.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attack optimizes attention/embedding proxies and hallucination is measured by an external GPT-4 judge, so the central claim is not reduced to its inputs by construction.

full rationale

No circular step can be exhibited with the paper's own equations. The attack minimizes the attention loss Lattn (Eq. 8) and the embedding loss Lemb (Eq. 9) with respect to the visual perturbation, while hallucination is evaluated afterward by GPT-4 using HalluBench references; there is no gradient or fitted parameter flowing from the hallucination metric into the optimization. The paper's selection of a 'potential sink token' (Eq. 7) and the embedding loss (Eq. 9) both use the same cosine similarity to the global input mean, but this is a self-reinforcing optimization objective, not a derivation of the empirical claim that hallucination increases. The claimed increases in HSR/HWR are measured externally with GPT-4 and human-annotated references, not manufactured by the attack objective. The causal mechanism in Section 3.2 is explicitly labeled speculation ('We speculate that it is still due to deficiencies in the instruction tuning phase'), which is a correctness-risk caveat, not circularity. The attention-sink and aggregation concepts are cited from prior external work ([34], [85], [95], [82], [20]), not from the authors' own prior results, so there is no load-bearing self-citation chain. The sigma hyperparameter in Eq. 9 is hand-tuned from observed sink-token similarities, but it is not fitted to, or renamed as, the hallucination prediction. The paper's baseline comparison is limited to random noise, and the 'only attention sink' framing is broader than the actual two-loss objective, but these concerns concern strength of evidence and scope, not circularity in the derivation chain.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces two attack-specific hyper-parameters (alpha, sigma) and a per-model layer choice, all hand-tuned rather than derived from first principles. It relies on several empirical claims about attention sinks and instruction-tuning data that are partially hedged as speculation in Section 3, and on GPT-4 as an evaluation judge with only a small alignment study. No new physical or conceptual entities are invented.

free parameters (4)
  • alpha = 1
    Regularization weight between attention loss and embedding loss (Eq. 10); set to 1 after an ablation study reported only in Appendix B, with the ablation results not shown.
  • sigma = not reported
    Threshold in the embedding hinge loss (Eq. 9), described as 'predefined with the observation of sink tokens in different MLLMs'; the numeric value is never given.
  • intermediate layer l = second-to-last or third-to-last per model
    Layer used for extracting hidden states and attention scores is hand-picked per model in Appendix B, and the attack success depends on this choice.
  • Smax = 30
    Total number of adversarial optimization steps chosen in Appendix B; the convergence and effect depend on this value.
assumptions (5)
  • domain assumption MLLMs inherit a two-segment response pattern from instruction-tuning datasets, with the second segment loosely related to the image.
    Section 3.1 asserts this based on per-sentence CLIPScore analysis of LLaVA-Instruct-150k and ALLaVA-4V (Fig. 3), but does not prove causality.
  • domain assumption Attention sinks appear at the turning point of image-text relevance and aggregate global input information into a single token.
    Section 3.2 and Figs. 4-5 present correlational evidence; the causal reading is explicitly hedged with 'We speculate'.
  • domain assumption Aggregating global context into a single token inevitably loses information and diminishes factual accuracy.
    Section 3.2(2) states this as the mechanism behind hallucination; it is the load-bearing premise for the attack objective and is labeled as speculation.
  • domain assumption GPT-4 based hallucination judgments are sufficiently aligned with human judgments.
    Section J reports an alignment study covering only two models and four response sets, with human scores slightly lower than GPT-4 scores; this is a weak empirical basis for a central evaluation tool.
  • standard math Softmax sum-to-one property leads to attention sinking in initial tokens; columnar attention patterns indicate sink tokens.
    Borrowed from prior work [34, 85] and used to define the attention loss and to detect sink tokens; standard attention mechanism mathematics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink." pith.science (2026). https://pith.science/paper/F4KWQSEI

@misc{pith2026250115269,
  author       = {Pith},
  title        = {Pith review of: Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F4KWQSEI}},
  note         = {Machine review of arXiv:2501.15269}
}
read the original abstract

Fusing visual understanding into language generation, Multi-modal Large Language Models (MLLMs) are revolutionizing visual-language applications. Yet, these models are often plagued by the hallucination problem, which involves generating inaccurate objects, attributes, and relationships that do not match the visual content. In this work, we delve into the internal attention mechanisms of MLLMs to reveal the underlying causes of hallucination, exposing the inherent vulnerabilities in the instruction-tuning process. We propose a novel hallucination attack against MLLMs that exploits attention sink behaviors to trigger hallucinated content with minimal image-text relevance, posing a significant threat to critical downstream applications. Distinguished from previous adversarial methods that rely on fixed patterns, our approach generates dynamic, effective, and highly transferable visual adversarial inputs, without sacrificing the quality of model responses. Comprehensive experiments on 6 prominent MLLMs demonstrate the efficacy of our attack in compromising black-box MLLMs even with extensive mitigating mechanisms, as well as the promising results against cutting-edge commercial APIs, such as GPT-4o and Gemini 1.5. Our code is available at https://huggingface.co/RachelHGF/Mirage-in-the-Eyes.

Figures

Figures reproduced from arXiv: 2501.15269 by the authors.

Figure 1
Figure 1. An illustration of the attention sink phenomenon [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Examples of inferred elements in ground truth responses: [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Per-sentence CLIPScore between input images and ground truth responses in instruction-tuning datasets. We report [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: (a)-(b) Per-sentence CLIPScore between input images and MLLM responses of InstructBLIP and LLaVA-1.5. (c) Mean CLIPScore of MLLM responses before and after the sink token. The postfix -beam, -greedy, and -nucleus represent beam search, greedy search, and nucleus sampli…
Figure 5
Figure 5. Figure 5: Distribution of cosine similarity between multi-modal input embeddings and generated token embeddings. We compare [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Left: The visualization of sink tokens and model responses during the hallucination attack. We display an optimization process of LLaVA-1.5 on the HalluBench dataset. Right: The black-box transfer effect of the constructed adversarial visual input on InstructBLIP. A pr…
Figure 7
Figure 7. Figure 7: Results of GPT-4 assisted hallucination evaluation [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Results of GPT-4 assisted response quality evalua [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The prompt for GPT-4 assisted hallucination evaluation. The bounding boxes coordinates, detailed descriptions, and [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: The prompt for GPT-4 assisted response quality evaluation. The evaluation criteria, and detailed steps are provided as [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 13
Figure 13. Figure 13: Results of Perplexity-based response quality evalu [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]
Figure 12
Figure 12. Figure 12: Results of QA Accuracy on commercial APIs: [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]
Figure 14
Figure 14. Figure 14: A case of model responses with clean image and adversarial visual input on InstructBLIP. [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]
Figure 15
Figure 15. Figure 15: A case of model responses with clean image and adversarial visual input on LLaVA-1.5. [PITH_FULL_IMAGE:figures/full_fig_p027_15.png]
Figure 16
Figure 16. Figure 16: A case of model responses with clean image and adversarial visual input on MiniGPT-4. [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]
Figure 17
Figure 17. Figure 17: A case of model responses with clean image and adversarial visual input on Shikra. [PITH_FULL_IMAGE:figures/full_fig_p027_17.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Selecting the intermediate layer where image-conditioned and text-only predictions diverge most, and adding that layer's contrastive visual signal back to the final logits, reduces object hallucinations in four large ...

Reference graph

Works this paper leans on

115 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736, 2022

    2022

  3. [3]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pages 2425–2433, 2015

    2015

  4. [4]

    Qwen-vl: A frontier large vision- language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision- language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023

    arXiv 2023

  5. [5]

    Hallucination of multimodal large language models: A survey

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930, 2024

    arXiv 2024

  6. [6]

    The (r) evolution of multimodal large language models: A survey

    Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Mar- cella Cornia, and Rita Cucchiara. The (r) evolution of multimodal large language models: A survey. arXiv preprint arXiv:2402.12451, 2024

    arXiv 2024

  7. [7]

    Alleviating hallucinations in large vision-language models through hallucination-induced optimization

    Beitao Chen, Xinyu Lyu, Lianli Gao, Jingkuan Song, and Heng Tao Shen. Alleviating hallucinations in large vision-language models through hallucination-induced optimization. arXiv preprint arXiv:2405.15356, 2024

    arXiv 2024

  8. [8]

    Lion: Empowering multimodal large language model with dual-level visual knowledge

    Gongwei Chen, Leyang Shen, Rui Shao, Xiang Deng, and Liqiang Nie. Lion: Empowering multimodal large language model with dual-level visual knowledge. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 26540– 26550, 2024

    2024

Show all 115 references
  1. [9]

    Allava: Harnessing gpt4v-synthesized data for a lite vision-language model

    Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhi- hong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing gpt4v-synthesized data for a lite vision-language model. arXiv preprint arXiv:2402.11684, 2024

    2024 arXiv

  2. [10]

    Minigpt-v2: large language model as a unified interface for vision-language multi-task learning

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krish- namoorthi, Vikas Chandra, Yunyang Xiong, and Mo- hamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:...

    arXiv

  3. [11]

    Ipl: Leveraging multimodal large language models for intelligent product listing

    Kang Chen, Qingheng Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, and Jufeng Chen. Ipl: Leveraging multimodal large language models for intelligent product listing. arXiv preprint arXiv:2410.16977, 2024

    arXiv

  4. [12]

    Shikra: Unleashing mul- timodal llm’s referential dialogue magic

    Ke Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing mul- timodal llm’s referential dialogue magic. ArXiv, abs/2306.15195, 2023

    2023 arXiv

  5. [13]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Con- ghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793 , 2023

    2023 arXiv

  6. [14]

    Multi-object hallucination in vision- language models

    Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F Fouhey, and Joyce Chai. Multi-object hallucination in vision- language models. arXiv preprint arXiv:2407.06192, 2024

    2024 arXiv

  7. [15]

    Typography leads semantic diversifying: Amplifying adversarial trans- ferability across multimodal large language models

    Hao Cheng, Erjia Xiao, Jiahang Cao, Le Yang, Kaidi Xu, Jindong Gu, and Renjing Xu. Typography leads semantic diversifying: Amplifying adversarial trans- ferability across multimodal large language models. arXiv preprint arXiv:2405.20090, 2024

    2024 arXiv

  8. [16]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023), 2...

    2023

  9. [17]

    Optimal structure identifi- cation with greedy search

    David Maxwell Chickering. Optimal structure identifi- cation with greedy search. Journal of machine learning research, 3(Nov):507–554, 2002

    2002

  10. [18]

    On the robustness of large multimodal models against image adversarial attacks

    Xuanming Cui, Alejandro Aparcedo, Young Kyun Jang, and Ser-Nam Lim. On the robustness of large multimodal models against image adversarial attacks. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 24625– 24634, 2024

    2024

  11. [19]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Albert Li, Pascale Fung, and Steven C. H. Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. ArXiv, abs/2305.06500, 2023

    2023 arXiv

  12. [20]

    Vision transformers need registers

    Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In The Twelfth International Conference on Learning Representations, 2024

    2024

  13. [21]

    Hilm-d: Towards high-resolution understanding in multimodal large language models for autonomous driving

    Xinpeng Ding, Jianhua Han, Hang Xu, Wei Zhang, and Xiaomeng Li. Hilm-d: Towards high-resolution understanding in multimodal large language models for autonomous driving. arXiv preprint arXiv:2309.05186, 2023

    2023 arXiv

  14. [22]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Con...

    2020

  15. [23]

    Eva: Exploring the limits of masked visual representation learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pag...

    2023

  16. [24]

    Mme-finance: A multimodal finance benchmark for expert-level understanding and reasoning

    Ziliang Gan, Yu Lu, Dong Zhang, Haohan Li, Che Liu, Jian Liu, Ji Liu, Haipang Wu, Chaoyou Fu, Zenglin Xu, et al. Mme-finance: A multimodal finance benchmark for expert-level understanding and reasoning. arXiv preprint arXiv:2411.03314, 2024

    2024 arXiv

  17. [25]

    Adversarial robustness for visual ground- ing of multimodal large language models

    Kuofeng Gao, Yang Bai, Jiawang Bai, Yong Yang, and Shu-Tao Xia. Adversarial robustness for visual ground- ing of multimodal large language models. In ICLR 2024 Workshop on Reliable and Responsible Founda- tion Models, 2024

    2024

  18. [26]

    Inducing high energy-latency of large vision-language models with verbose images

    Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, and Wei Liu. Inducing high energy-latency of large vision-language models with verbose images. In The Twelfth International Confer- ence on Learning Representations, 2024

    2024

  19. [27]

    Multimodal-gpt: A vision and language model for dialogue with humans

    Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gpt: A vision and language model for dialogue with humans. arXiv preprint arXiv:2305.04790, 2023

    2023 arXiv

  20. [28]

    Figstep: Jailbreaking large vision-language models via typographic visual prompts

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608, 2023

    2023 arXiv

  21. [29]

    Hallusionbench: an advanced diagnostic suite for entangled language hal- lucination and visual illusion in large vision-language models

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hal- lucination and visual illusion in large vision-language models. In Pro...

    2024

  22. [30]

    Skip\n: A simple method to reduce hallucination in large vision- language models

    Zongbo Han, Zechen Bai, Haiyang Mei, Qianli Xu, Changqing Zhang, and Mike Zheng Shou. Skip\n: A simple method to reduce hallucination in large vision- language models. arXiv preprint arXiv:2402.01345, 2024

    2024 arXiv

  23. [31]

    Clipscore: A reference-free 16 evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free 16 evaluation metric for image captioning. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 7514–7528, 2021

    2021

  24. [32]

    The curious case of neural text degenera- tion

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degenera- tion. arXiv preprint arXiv:1904.09751, 2019

    1904 arXiv

  25. [33]

    Naturalistic physical adversarial patch for object detectors

    Yu-Chih-Tuan Hu, Bo-Han Kung, Daniel Stanley Tan, Jun-Cheng Chen, Kai-Lung Hua, and Wen-Huang Cheng. Naturalistic physical adversarial patch for object detectors. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7848–7857, 2021

    2021

  26. [34]

    Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conferenc...

    2024

  27. [35]

    Hallucination augmented contrastive learning for multimodal large language model

    Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learning for multimodal large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  28. [36]

    Diffattack: Evasion attacks against diffusion-based adversarial pu- rification

    Mintong Kang, Dawn Song, and Bo Li. Diffattack: Evasion attacks against diffusion-based adversarial pu- rification. Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [37]

    Vi- sual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin John- son, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yan- nis Kalantidis, Li-Jia Li, David A Shamma, et al. Vi- sual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of com...

    2017

  30. [38]

    Gemini pro defeated by gpt-4v: Evidence from education

    Gyeong-Geon Lee, Ehsan Latif, Lehong Shi, and Xi- aoming Zhai. Gemini pro defeated by gpt-4v: Evidence from education. arXiv preprint arXiv:2401.08660 , 2023

    2023 arXiv

  31. [39]

    Robust evaluation of diffusion-based adversarial purification

    Minjong Lee and Dongwoo Kim. Robust evaluation of diffusion-based adversarial purification. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 134–144, 2023

    2023

  32. [40]

    V olcano: mitigating multimodal halluci- nation through self-feedback guided revision

    Seongyun Lee, Sue Hyun Park, Yongrae Jo, and Min- joon Seo. V olcano: mitigating multimodal halluci- nation through self-feedback guided revision. arXiv preprint arXiv:2311.07362, 2023

    2023 arXiv

  33. [41]

    Miti- gating object hallucinations in large vision-language models through visual contrastive decoding

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Miti- gating object hallucinations in large vision-language models through visual contrastive decoding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  34. [42]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Infor- mation Processing Systems, 36, 2024

    2024

  35. [43]

    Manipllm: Embodied multimodal large language model for object-centric robotic manipula- tion

    Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yuxing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object-centric robotic manipula- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  36. [44]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023

    2023 arXiv

  37. [45]

    Monkey: Image resolution and text label are important things for large multi-modal models

    Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xi- ang Bai. Monkey: Image resolution and text label are important things for large multi-modal models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  38. [46]

    Harnessing gpt-4v (ision) for insurance: A preliminary exploration

    Chenwei Lin, Hanjia Lyu, Jiebo Luo, and Xian Xu. Harnessing gpt-4v (ision) for insurance: A preliminary exploration. arXiv preprint arXiv:2404.09690, 2024

    2024 arXiv

  39. [47]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings...

    2014

  40. [48]

    Interpreting and mitigating hallucination in mllms through multi-agent debate

    Zheng Lin, Zhenxing Niu, Zhibin Wang, and Yinghui Xu. Interpreting and mitigating hallucination in mllms through multi-agent debate. arXiv preprint arXiv:2407.20505, 2024

    2024 arXiv

  41. [49]

    Gemex: A large-scale, groundable, and explainable medical vqa benchmark for chest x-ray diagnosis

    Bo Liu, Ke Zou, Liming Zhan, Zexin Lu, Xiaoyu Dong, Yidi Chen, Chengqiang Xie, Jiannong Cao, Xiao-Ming Wu, and Huazhu Fu. Gemex: A large-scale, groundable, and explainable medical vqa benchmark for chest x-ray diagnosis. arXiv preprint arXiv:2411.16778, 2024

    2024 arXiv

  42. [50]

    Mitigating hallucination in large multi-modal models via robust instruction tuning

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representations, 2023. 17

    2023

  43. [51]

    Improved baselines with visual instruction tun- ing

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tun- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  44. [52]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024

    2024

  45. [53]

    Models see hallucinations: Evaluating the factuality in video captioning

    Hui Liu and Xiaojun Wan. Models see hallucinations: Evaluating the factuality in video captioning. arXiv preprint arXiv:2303.02961, 2023

    2023 arXiv

  46. [54]

    Phd: A prompted visual hallucination evalu- ation dataset

    Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, and Xirong Li. Phd: A prompted visual hallucination evalu- ation dataset. arXiv preprint arXiv:2403.11116, 2024

    2024 arXiv

  47. [55]

    Query-relevant images jailbreak large multi- modal models

    Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. Query-relevant images jailbreak large multi- modal models. arXiv preprint arXiv:2311.17600, 2023

    2023 arXiv

  48. [56]

    Mminstruct: A high-quality multi- modal instruction tuning dataset with extensive diver- sity

    Yangzhou Liu, Yue Cao, Zhangwei Gao, Weiyun Wang, Zhe Chen, Wenhai Wang, Hao Tian, Lewei Lu, Xizhou Zhu, Tong Lu, et al. Mminstruct: A high-quality multi- modal instruction tuning dataset with extensive diver- sity. arXiv preprint arXiv:2407.15838, 2024

    2024 arXiv

  49. [57]

    Robollm: Robotic vision tasks grounded on multimodal large language models

    Zijun Long, George Killick, Richard McCreadie, and Gerardo Aragon-Camarasa. Robollm: Robotic vision tasks grounded on multimodal large language models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 12428–12435. IEEE, 2024

    2024

  50. [58]

    Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large lan- guage models against jailbreak attacks

    Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large lan- guage models against jailbreak attacks. arXiv preprint arXiv:2404.03027, 2024

    2024 arXiv

  51. [59]

    Visual- roleplay: Universal jailbreak attack on multimodal large language models via role-playing image char- acte

    Siyuan Ma, Weidi Luo, Yu Wang, Xiaogeng Liu, Muhao Chen, Bo Li, and Chaowei Xiao. Visual- roleplay: Universal jailbreak attack on multimodal large language models via role-playing image char- acte. arXiv preprint arXiv:2405.20773, 2024

    2024 arXiv

  52. [60]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on com- puter vision and pattern recognition, pages 3195–3204, 2019

    2019

  53. [61]

    Large language models: A survey

    Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatri- ain, and Jianfeng Gao. Large language models: A survey. arXiv preprint arXiv:2402.06196, 2024

    2024 arXiv

  54. [62]

    Yo’llava: Your per- sonalized language and vision assistant

    Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, and Yong Jae Lee. Yo’llava: Your per- sonalized language and vision assistant. arXiv preprint arXiv:2406.09400, 2024

    2024 arXiv

  55. [63]

    Jailbreaking attack against mul- timodal large language model

    Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. Jailbreaking attack against mul- timodal large language model. arXiv preprint arXiv:2402.02309, 2024

    2024 arXiv

  56. [64]

    Gpt-4v(ision) technical work and authors

    OpenAI. Gpt-4v(ision) technical work and authors. 2023

    2023

  57. [65]

    Gemini flash

    OpenAI. Gemini flash. 2024

    2024

  58. [66]

    Gpt-4o mini: advancing cost-efficient intelli- gence

    OpenAI. Gpt-4o mini: advancing cost-efficient intelli- gence. 2024

    2024

  59. [67]

    Gem- ini goes to med school: exploring the capabilities of multimodal large language models on medical chal- lenge problems & hallucinations

    Ankit Pal and Malaikannan Sankarasubbu. Gem- ini goes to med school: exploring the capabilities of multimodal large language models on medical chal- lenge problems & hallucinations. arXiv preprint arXiv:2402.07023, 2024

    2024 arXiv

  60. [68]

    Kosmos- 2: Grounding multimodal large language models to the world

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos- 2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824, 2023

    2023 arXiv

  61. [69]

    Grounding multimodal large language models to the world

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, and Furu Wei. Grounding multimodal large language models to the world. In The Twelfth International Conference on Learning Representations, 2024

    2024

  62. [70]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21527–21536, 2024

    2024

  63. [71]

    Learning transferable visual models from natural lan- guage supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural lan- guage supervision. In International conference on machine learning...

    2021

  64. [72]

    Object hallucination in image captioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing, pages 4035–4045, 2018

    2018

  65. [73]

    Laion coco: 600m synthetic cap- tions from laion2b-en

    C Schuhmann, A Köpf, R Vencu, T Coombes, and R Beaumont. Laion coco: 600m synthetic cap- tions from laion2b-en. https://laion.ai/blog/laion-coco, 2022

    2022

  66. [74]

    Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In The Twelfth In- ternational Conference on Learning Representations, 2023. 18

    2023

  67. [75]

    Prefixing attention sinks can mitigate activation outliers for large language model quantization

    Seungwoo Son, Wonpyo Park, Woohyun Han, Kyuyeun Kim, and Jaeho Lee. Prefixing attention sinks can mitigate activation outliers for large language model quantization. arXiv preprint arXiv:2406.12016, 2024

    2024 arXiv

  68. [76]

    Self-training large language and vision assistant for medical question answering

    Guohao Sun, Can Qin, Huazhu Fu, Linwei Wang, and Zhiqiang Tao. Self-training large language and vision assistant for medical question answering. In Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20052–20060, 2024

    2024

  69. [77]

    Aligning large multimodal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023

    2023 arXiv

  70. [78]

    An empirical study and analysis of text-to-image genera- tion using large language model-powered textual rep- resentation

    Zhiyu Tan, Mengping Yang, Luozheng Qin, Hao Yang, Ye Qian, Qiang Zhou, Cheng Zhang, and Hao Li. An empirical study and analysis of text-to-image genera- tion using large language model-powered textual rep- resentation. arXiv preprint arXiv:2405.12914, 2024

    2024 arXiv

  71. [79]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  72. [80]

    Attention is all you need

    Ashish Vaswani. Attention is all you need. arXiv preprint arXiv:1706.03762, 2017

    2017 arXiv

  73. [81]

    An llm-free multi-dimensional benchmark for mllms hallucination evaluation

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Ji- tao Sang. An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.07397, 2023

    2023 arXiv

  74. [82]

    Label words are anchors: An information flow perspective for under- standing in-context learning

    Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. Label words are anchors: An information flow perspective for under- standing in-context learning. In The 2023 Conference on Empirical Methods in Natural Language Process- ing, 2023

    2023

  75. [83]

    Adashield: Safeguarding multi- modal large language models from structure-based at- tack via adaptive shield prompting

    Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. Adashield: Safeguarding multi- modal large language models from structure-based at- tack via adaptive shield prompting. arXiv preprint arXiv:2403.09513, 2024

    2024 arXiv

  76. [84]

    Can gpt-4v (ision) serve medical applications? case studies on gpt- 4v for multimodal medical diagnosis

    Chaoyi Wu, Jiayu Lei, Qiaoyu Zheng, Weike Zhao, Weixiong Lin, Xiaoman Zhang, Xiao Zhou, Ziheng Zhao, Ya Zhang, Yanfeng Wang, et al. Can gpt-4v (ision) serve medical applications? case studies on gpt- 4v for multimodal medical diagnosis. arXiv preprint arXiv:2310.09909, 2023

    arXiv

  77. [85]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In The Twelfth Interna- tional Conference on Learning Representations, 2024

    2024

  78. [86]

    Open-finllms: Open multimodal large language models for financial appli- cations

    Qianqian Xie, Dong Li, Mengxi Xiao, Zihao Jiang, Ruoyu Xiang, Xiao Zhang, Zhengyu Chen, Yueru He, Weiguang Han, Yuzhe Yang, et al. Open-finllms: Open multimodal large language models for financial appli- cations. arXiv preprint arXiv:2408.11878, 2024

    2024 arXiv

  79. [87]

    Self-evaluation guided beam search for reasoning

    Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min-Yen Kan, Junxian He, and Michael Xie. Self-evaluation guided beam search for reasoning. Ad- vances in Neural Information Processing Systems, 36, 2024

    2024

  80. [88]

    Drivegpt4: Interpretable end-to-end au- tonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Heng- shuang Zhao. Drivegpt4: Interpretable end-to-end au- tonomous driving via large language model. IEEE Robotics and Automation Letters, 2024

    2024

  81. [89]

    Ureader: Universal ocr-free visually- situated language understanding with multimodal large language model

    Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually- situated language understanding with multimodal large language model. arXiv preprint arXiv:2310.05126 , 2023

    arXiv

  82. [90]

    mplug-owl: Modular- ization empowers large language models with multi- modality

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modular- ization empowers large language models with multi- modality. arXiv preprint arXiv:2304.14178, 2023

    2023 arXiv

  83. [91]

    Woodpecker: Hallucination cor- rection for multimodal large language models

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination cor- rection for multimodal large language models. arXiv preprint arXiv:2310.16045, 2023

    arXiv

  84. [92]

    Unveiling the safety of gpt-4o: An em- pirical study using jailbreak attacks

    Zonghao Ying, Aishan Liu, Xianglong Liu, and Dacheng Tao. Unveiling the safety of gpt-4o: An em- pirical study using jailbreak attacks. arXiv preprint arXiv:2406.06302, 2024

    2024 arXiv

  85. [93]

    Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data

    Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wen- tao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yuet- ing Zhuang. Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2024

  86. [94]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine- grained correctional human feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai- Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine- grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on ...

    2024

  87. [95]

    Unveiling and harnessing hidden attention sinks: Enhancing large language models without training through attention calibration

    Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, and Yingyan Celine Lin. Unveiling and harnessing hidden attention sinks: Enhancing large language models without training through attention calibration. arXiv preprint arXiv:2406.15765, 2024

    2024 arXiv

  88. [96]

    Less is more: Mitigating multimodal hallucination from an eos de- cision perspective

    Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigating multimodal hallucination from an eos de- cision perspective. arXiv preprint arXiv:2402.14545, 2024

    2024 arXiv

  89. [97]

    Llavar: En- hanced visual instruction tuning for text-rich image un- derstanding

    Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, and Tong Sun. Llavar: En- hanced visual instruction tuning for text-rich image un- derstanding. arXiv preprint arXiv:2306.17107, 2023

    2023 arXiv

  90. [98]

    On evaluating adversarial robustness of large vision- language models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision- language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  91. [99]

    Beyond hal- lucinations: Enhancing lvlms through hallucination- aware direct preference optimization

    Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hal- lucinations: Enhancing lvlms through hallucination- aware direct preference optimization. arXiv preprint arXiv:2311.16839, 2023

    2023 arXiv

  92. [100]

    Pre-trained multi- modal large language model enhances dermatological diagnosis using skingpt-4

    Juexiao Zhou, Xiaonan He, Liyuan Sun, Jiannan Xu, Xiuying Chen, Yuetan Chu, Longxi Zhou, Xingyu Liao, Bin Zhang, Shawn Afvari, et al. Pre-trained multi- modal large language model enhances dermatological diagnosis using skingpt-4. Nature Communications, 15(1):5649, 2024

    2024

  93. [101]

    Analyzing and mitigating object halluci- nation in large vision-language models

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object halluci- nation in large vision-language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  94. [102]

    Minigpt-4: Enhancing vision- language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  95. [103]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. A Decoding Strategies of MLLMs Decoding strategies play a vital role in the...

    2023 arXiv

  96. [104]

    This metric quantifies the level of detail in an MLLM’s response at the sentence level

    SPI: The number of sentences per image in MLLM re- sponses. This metric quantifies the level of detail in an MLLM’s response at the sentence level. 20

  97. [105]

    This metric assesses the level of detail in an MLLM’s response at the word level

    WPI: The number of words per image in MLLM re- sponses. This metric assesses the level of detail in an MLLM’s response at the word level

  98. [106]

    This metric indicates the extent of hallucination within an MLLM’s response at the sentence level, counting any sentence that contains fabricated content

    HSPI: The number of hallucinated sentences per image. This metric indicates the extent of hallucination within an MLLM’s response at the sentence level, counting any sentence that contains fabricated content

  99. [107]

    This metric measures the extent of hallucination within an MLLM’s response at the word level, accounting for any words associated with hallucinated content

    HWPI: The number of hallucinated words per image. This metric measures the extent of hallucination within an MLLM’s response at the word level, accounting for any words associated with hallucinated content

  100. [108]

    This metric represents the average proportion of hallucinated sen- tences in the total number of sentences across various MLLM responses on different images

    HSR: The ratio of hallucinated sentences. This metric represents the average proportion of hallucinated sen- tences in the total number of sentences across various MLLM responses on different images

  101. [109]

    This metric cap- tures the average proportion of hallucinated words in the total word count across different MLLM responses on various images

    HWR: The ratio of hallucinated words. This metric cap- tures the average proportion of hallucinated words in the total word count across different MLLM responses on various images. In addition to the hallucinated content, we also consider the quality of model responses as part...

  102. [110]

    Cannot judge

    Notably, the human-evaluated hallucination metrics are slightly lower than those obtained from GPT-4, likely due to the limited scope of descriptions and the factual infor- mation annotated in the HalluBench dataset. Additionally, the results for adversarial visual inputs show...

  103. [111]

    hallucination or correct or cannot judge: <reason>

  104. [112]

    The bounding boxes coordinates, detailed descriptions, and human-annotated factual knowledge are provided as context information

    Here are the region descriptions of the image: {} Here is the comment for you to judge (hallucination, correct, or cannot judge): {} Figure 9: The prompt for GPT-4 assisted hallucination evaluation. The bounding boxes coordinates, detailed descriptions, and human-annotated fac...

  105. [113]

    Read the paragraph carefully and identify the main points of the article

  106. [114]

    Assess how many errors it has that affect the clarity or smoothness of the text, and whether the main points are still comprehensible

  107. [115]

    Paragraph: {} Your output should be: - Fluency (1-10): (scores ONLY) Figure 10: The prompt for GPT-4 assisted response quality evaluation

    Assign a relevance score from 1 to 10. Paragraph: {} Your output should be: - Fluency (1-10): (scores ONLY) Figure 10: The prompt for GPT-4 assisted response quality evaluation. The evaluation criteria, and detailed steps are provided as context information. 22 Table 8: Result...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.