Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Object hallucination in vision-language models is a semantic mismatch between strongly attended image regions and the output token, not a lack of visual attention; the paper grounds this in Logit Lens readings and masking experiments…

desk verdict Worth taking seriously: the 'same attention, different truths' result is real and useful, but the two-mechanism taxonomy needs stronger causal controls before the story is trustworthy. read the letter →

arxiv 2608.07302 v1 pith:CZTJALKJ submitted 2026-08-07 cs.CV cs.AI

Zichuan Wang , Songlin Yang , Bo Peng , Zhenchen Tang , Yang Li , Beibei Dong , Jing Dong This is my paper · ORCID
classification cs.CVcs.AI
keywords objecthallucinationlargevision-languagemodelsLogitLensvisualattentiondetectionmitigationtraining-freedecodingsemanticconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that object hallucination in large vision-language models is not caused by weak visual attention. Comparing real and hallucinated object tokens, it finds both receive similar attention in the middle-to-late transformer layers, but decoding the attended image regions with Logit Lens matches the generated object only for real objects. Masking interventions separate hallucinations into two mechanisms: visual uncertainty, where ambiguous image patches are mistaken for the object and disappear when masked, and contextual prior, where strong text co-occurrence overrides evidence and attention drifts when the region is masked. On that basis it builds a training-free detect-and-mitigate pipeline and reports lower hallucination rates on standard benchmarks across several models.

What carries the argument

The 'image-attention stage'—the middle-to-late layers where object tokens concentrate attention on image patches—is the stage the whole argument uses. The Logit-Lens Consistency Check projects hidden states of the top-k attended image patches through the model's output head and asks whether the decoded word is semantically similar to the generated object token; consistency marks a real object, inconsistency marks a hallucination. Two interventions carry the causal classification: High-Attention Region Masking blanks those image patches and regenerates the response, while Visual Evidence Enhanced Decoding fuses the Logit-Lens logits of the most-attended region with the masked-image decoding logits to suppress contextual priors.

What would settle it

A control experiment that masks the same number of image patches chosen at random, or patches known to be non-causal attention sinks, and measures whether hallucinated tokens disappear as often as they do when the top-attended patches are masked; if random masking removes hallucinations equally well, the claimed visual-evidence mechanism would not be supported.

Watch

Extended reading notes

Core claim

On the paper's own terms: real and hallucinated object tokens receive statistically indistinguishable visual attention during the 'image-attention stage' of the middle-to-late layers, so hallucination cannot be blamed on how much the model looks. What differs is what the model sees. Decoding the hidden states of high-attention image tokens with the model's output head (Logit Lens) yields the object token for real objects and unrelated tokens for hallucinated objects. Masking those high-attention regions splits hallucinations into two classes: visual-uncertainty hallucinations vanish when the region is masked, while contextual-prior hallucinations persist and re-anchor attention elsewhere. The paper concludes that the core defect is semantic misalignment between strongly attended visual features and the generated token, and treats this as both a detection signal and the basis for two targeted, training-free remedies.

Load-bearing premise

The load-bearing premise is that the most-attended image patches in the middle-to-late layers are the actual visual evidence driving the next word, so removing them and watching the output reveals why the word was chosen; attention maps can be shaped by language priors and model artifacts, so this causal link is assumed rather than proven.

Editorial extensions

If this is right

  • Detection can run in real time during generation by comparing the decoded semantics of attended image regions with the token being generated.
  • A single mitigation recipe is insufficient: masking is the right remedy for visual-uncertainty hallucinations, and visual-logit injection is the right remedy for contextual-prior hallucinations.
  • Attention magnitude alone is not a trustworthy hallucination cue; attention quality, measured by semantic decodability, carries the signal.
  • Because the framework is training-free and uses only the model's own output head and attention weights, it ports across LVLM architectures without additional data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue is to turn the Logit-Lens consistency score into a training signal, rewarding generations whose object tokens can be decoded back out of the regions they attend to.
  • The two-mechanism split predicts that hallucination benchmarks respond differently to interventions: adding confusing look-alike distractors should inflate visual-uncertainty errors, while re-weighting co-occurrence statistics should target contextual-prior errors.
  • Because contextual-prior hallucinations persist after masking and re-anchor attention, decoding-time corrections that only suppress the initially attended region will be blind to them; any method relying solely on masking should fail on the same examples that Visual Evidence Enhanced Decoding handles.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies object hallucination in LVLMs. It argues that real and hallucinated object tokens receive comparable visual attention in mid-to-late layers, that Logit-Lens decoding of top-k attended image patches reveals semantic inconsistency for hallucinated tokens, and that hallucinations split into two mechanisms: visual uncertainty and contextual prior. On this basis it proposes a training-free Detect-Mitigate framework: LLCC detects hallucinated object tokens, HARM masks high-attention image regions for visual-uncertainty cases, and VEED injects visual logits for contextual-prior cases. Experiments on CHAIR and AMBER across LLaVA-1.5-7B/13B, Shikra-7B, and Qwen2-VL-7B report consistent improvements over prior decoding-based and attention-based baselines.

Significance. If the mechanism claims are correct, the paper offers a useful reframing of object hallucination: the problem is not insufficient attention but a semantic mismatch between what the model attends to and what it decodes. The proposed method is training-free and the empirical gains in Tables 2 and 3 are broad. The detection idea is tested against external CHAIR labels, so the strongest form of circularity is avoided. However, the two-mechanism taxonomy and the 'same attention' claim rest on uncontrolled analyses, and key hyperparameters appear to be selected on the evaluation sample. The significance is therefore conditional on the proposed controls and ablations.

major comments (4)
  1. [3.4, Eqs. (4)-(9)] The classification into visual-uncertainty and contextual-prior hallucinations is derived solely from the top-k masking intervention and the binary outcome in Eq. (9). The manuscript reports no control conditions: no random-patch masking, no low-attention-patch masking, no variation of the replacement value (mean color, black, or blur), and no repeated sampling to assess classification stability. Because masking any patch changes the whole input distribution and can shift attention globally, and because attention weights can be shaped by language priors and attention sinks, the 'disappears' versus 'persists' dichotomy may reflect generic sensitivity to image perturbation rather than two distinct causal mechanisms. Please add these controls and report per-instance counts and agreement rates instead of the single 2:1 ratio.
  2. [3.2, Fig. 1(b)] The central 'same attention, different truth' claim is supported by overlaid means with standard deviations in Fig. 1(b) and representative heatmaps in Fig. 3, but no statistical test is provided for the equality of attention magnitude between real and hallucinated object tokens. Since this claim motivates the whole framework, please report a paired comparison (e.g., a permutation test or bootstrap confidence interval over tokens or responses), and clarify whether the aggregation is over tokens within a response or over responses.
  3. [5.1.1 and 5.2.1] Several load-bearing hyperparameters are stated without sensitivity analysis: tau_attn=0.15, k=3, tau_sim=0.8, alpha, and the S_IA layer ranges. The layer ranges in Fig. 2 are defined for LLaVA-1.5-7B and applied to LLaVA-1.5-13B, Shikra-7B, and Qwen2-VL-7B without specifying how they are adapted to different model depths and attention patterns. The manuscript also does not state whether the 500-image sample used for hyperparameter choice is disjoint from the 500-image CHAIR evaluation sample. Please provide per-model S_IA definitions and a sensitivity analysis for each hyperparameter, or the comparisons in Tables 1-3 may reflect tuning on the evaluation sample.
  4. [Appendix references and Section 5.2.2] The text repeatedly refers to 'the appendix' for parameter experiments and ablations (e.g., in Section 5.1.1 and Section 5.2.2), but the submitted manuscript has no appendix. Without these experiments, the robustness of the thresholds and the individual contributions of LLCC, HARM, and VEED cannot be verified. Please include the missing appendix or remove the references if the material is unavailable.
minor comments (5)
  1. [Abstract and Conclusion] There are typos such as 'andwhat' in the Introduction and 'Logits-Lens' in the Conclusion; please make the terminology consistent.
  2. [3.2, Fig. 2] The four-stage description and the layer ranges 20-27 are presented as general, but they are derived from LLaVA-1.5-7B; please state explicitly that the ranges are model-specific and explain how they are obtained.
  3. [4.1.2, Eq. (8)] The semantic similarity function is described only as WordNet 'or another semantic similarity function'; please specify the exact function, how token-level outputs from Eq. (7) are lemmatized or mapped to word senses, and what threshold behavior is expected near tau_sim=0.8.
  4. [4.2.2, Eq. (12)] The default value of alpha is not reported in the main text; the experimental settings should include it, along with any grid over which it was chosen.
  5. [5.1.1 and 5.2.1] The random 500-image sample from COCO is not seeded; to make the detection and CHAIR results reproducible, please provide a seed or image ID list.

Circularity Check

2 steps flagged · score 6.0 of 10

The visual-uncertainty mechanism and its HARM remedy are circular by construction: the category is defined by the very masking outcome that HARM then applies.

  1. self definitional [Section 3.4, “Two Types: Visual Uncertainty and Contextual Prior”]
    "Visual Uncertainty Hallucination: In these cases, the hallucinated token is directly related with high-attention regions. When we mask these regions, the hallucinated token disappears ... Contextual Prior Hallucination: In contrast, we also observe hallucination where masking the high-attention regions does not remove the hallucinated token. Instead, hallucination persists and attention shifts to other regions."

    The two proposed mechanisms are not measured independently; they are named after the two possible outcomes of the same masking intervention. As a result, the statement that visual-uncertainty hallucinations are eliminated by masking is true by definition rather than by empirical test. This operational definition is then reused as the classifier in Eq. (9), so the two-mechanism taxonomy is self-confirming rather than independently validated.

  2. self definitional [Section 4.1.3, Eq. (9), and Section 4.2.1, Eqs. (10)-(11)]
    "Following the same procedure as Sec. 3.4, we mask the high-attention regions within S IA and observe whether the hallucination “disappears” ... Type(o_t)=1 (Visual Uncertainty) if o_t ∉ O_new, 2 (Contextual Prior) if o_t ∈ O_new. ... Therefore, we employ a simple masking strategy, identical to that in Sec. 3.4."

    HARM’s mitigation for type-1 hallucinations is the same operation used to define type 1. Any token classified as Visual Uncertainty is, by Eq. (9), one that disappeared after masking; re-applying that mask cannot fail to remove it. The claimed effectiveness of HARM for its target category is therefore a restatement of the classification rule, not independent evidence. Only VEED’s effect on the persisting type is empirically nontrivial.

full rationale

The equal-attention finding and the Logit-Lens consistency observation are empirical, and the overall detection and mitigation pipeline is benchmarked against external CHAIR and AMBER labels, so the paper is not wholly circular. However, the central two-mechanism story reduces to the masking intervention: “visual uncertainty” is defined as disappearing under masking and “contextual prior” as persisting under masking, and the same masking is then presented as the targeted cure for the first type. That makes the per-type mitigation claim true by construction rather than by experimental test. Thresholds for the attention stage, top-k, and similarity are chosen on the same COCO sample used for evaluation, but that is a benchmark-tuning concern rather than circularity. The detection method itself operationalizes the Logit-Lens insight and is tested on external labels, so it retains independent content aside from the definitional type-1 loop.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or architectural entities. Its central scientific contribution is a taxonomy and a detection signal, so the main ledger burden is the set of hand-chosen thresholds and the interpretative axioms that make Logit Lens and masking meaningful.

free parameters (5)
  • tau_attn = 0.15
    Threshold for treating a generated token as an object token based on average image attention in the Image-Attention Stage.
  • k = 3
    Number of top-attended image tokens selected for Logit Lens decoding and masking.
  • tau_sim = 0.8
    Semantic similarity threshold for judging whether the decoded token is consistent with the generated object token.
  • alpha = not reported
    Weight in VEED for fusing visual logits with masked-image decoding logits; the value is not disclosed in the main text.
  • S_IA layer range = layers 20-27 for LLaVA-1.5-7B
    The Image-Attention Stage is defined by eyeballing or fitting the attention curves on the same COCO sample used for evaluation; no per-model specification is given for the other test models.
assumptions (4)
  • domain assumption The output head of the LVLM can be applied to intermediate hidden states of image tokens to produce meaningful semantic tokens (Logit Lens).
    Invoked in Section 3.3 and Equation 3; the entire detection method depends on this interpretability assumption.
  • domain assumption The top-k attended image tokens in the mid-to-late layers constitute the visual evidence for a generated object token.
    Used in Section 3.4 and Equation 4 to identify candidate regions for masking and decoding.
  • domain assumption Semantic similarity between the decoded token and the generated token, as measured by WordNet or a similar function, is a reliable indicator of whether the visual evidence supports the token.
    Used in Equation 8 for the hallucination detection label.
  • domain assumption Masking high-attention image regions and regenerating is a valid causal intervention that reveals why the model hallucinated.
    Used in Section 3.4 and Section 4.1.3; this is the load-bearing causal assumption behind the two-mechanism taxonomy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination." pith.science (2026). https://pith.science/paper/CZTJALKJ

@misc{pith2026260807302,
  author       = {Pith},
  title        = {Pith review of: Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CZTJALKJ}},
  note         = {Machine review of arXiv:2608.07302}
}
read the original abstract

Large Vision-Language Models (LVLMs) often suffer from object hallucination, generating objects that are absent from the image. Prior work largely attributes this to insufficient visual attention. However, we find that both real and hallucinated objects receive equally strong visual attention in the model's mid-to-late layers, suggesting that the key issue may not be how much the model attends, but what it attends to and why. To this end, we decode the visual features of high-attention regions using Logit Lens, and observe that regions corresponding to real objects can be correctly decoded to the target object tokens, whereas those for hallucinated objects cannot. Building on this, we identify two hallucination mechanisms: (i) visual uncertainty, triggered by semantically similar or confusable regions; masking these regions eliminates the hallucination. (ii) contextual prior, triggered by strong co-occurrence priors; even when the initially attended region is masked, the hallucination persists and attention drifts to other regions. Based on these findings, we propose a simple yet effective training-free Detect-Mitigate framework comprising a Logit-Lens Consistency Check to detect hallucination and targeted remedies: High-Attention Regions Masking (HARM) for visual uncertainty hallucination, and Visual Evidence Enhanced Decoding (VEED) for contextual prior hallucination. Our approach achieves state-of-the-art results on multiple hallucination benchmarks. Code will be available.

Figures

Figures reproduced from arXiv: 2608.07302 by the authors.

Figure 1
Figure 1. Different from prior work that attributes object halluci [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Attention Ratio across layers. (a) Object attention; (b) Non-object attention. Non-object tokens exhibit minimal attention to the image, while for object tokens we partition the attention into four stages according to their variations through layers, with the mid-to-late layers (20-27) being the Image-Attention Stage. Logit Lens. Logit Lens is an interpretability method that decodes the hidden states of intermediate… view at source ↗
Figure 4
Figure 4. Via the Logit-Lens, semantics decoded from the high [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Both the real object token (a) ‘chair’ and the hallucinated object token (b) ‘bowl’ exhibit attention that converges on specific regions during the image-attention stage (layers 20–27). • Text-Attention Stage: In the early-to-mid layers (3-19), text attention dominates…
Figure 5
Figure 5. Figure 5: Two types of hallucination. (a) Disappears after masking high-attention regions, driven by visual uncertainty. (b) Persists after masking with attention shifting to other regions, driven by contextual prior. HOT here means Hallucinated Object Token. 4. Detect-Mitigate …
Figure 6
Figure 6. Figure 6: The overall structure of our Detect-Mitigate framework. It consists of three components. First, [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 17 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

  2. [2]

    Mitigating object hallucinations in large vision- language models with assembly of global and local attention

    Wenbin An, Feng Tian, Sicong Leng, Jiahao Nie, Haonan Lin, QianYing Wang, Ping Chen, Xiaoqin Zhang, and Shi- jian Lu. Mitigating object hallucinations in large vision- language models with assembly of global and local attention. InProceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 29915–29926, 2025. 1, 3

    work page 2025

  3. [3]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023. 1, 2

    arXiv 2023

  4. [4]

    Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024. 1

    arXiv 2024

  5. [5]

    Ict: Image-object cross-level trusted intervention for mitigating object halluci- nation in large vision-language models

    Junzhe Chen, Tianshu Zhang, Shiyu Huang, Yuwei Niu, Lin- feng Zhang, Lijie Wen, and Xuming Hu. Ict: Image-object cross-level trusted intervention for mitigating object halluci- nation in large vision-language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 4209–4221, 2025. 1

    work page 2025

  6. [6]

    Shikra: Unleashing multi- modal llm’s referential dialogue magic.arXiv preprint arXiv:2306.15195, 2023

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multi- modal llm’s referential dialogue magic.arXiv preprint arXiv:2306.15195, 2023. 2, 8

    arXiv 2023

  7. [7]

    Mitigating hallucination in visual language models with visual supervi- sion.arXiv preprint arXiv:2311.16479, 2023

    Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jinqiao Wang, and Ming Tang. Mitigating hallucination in visual language models with visual supervi- sion.arXiv preprint arXiv:2311.16479, 2023. 1, 2

    arXiv 2023

  8. [8]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024. 1, 2

    2024

Show all 47 references
  1. [9]

    Halc: Object hallucination re- duction via adaptive focal-contrast decoding.arXiv preprint arXiv:2403.00425, 2024

    Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Halc: Object hallucination re- duction via adaptive focal-contrast decoding.arXiv preprint arXiv:2403.00425, 2024. 3

    2024 arXiv

  2. [10]

    Instructblip: Towards general-purpose vision- language models with instruction tuning.Advances in neural information processing systems, 36:49250–49267, 2023

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning.Advances in neural information processing systems, 36:49250–49267, 2023. 1

    2023

  3. [11]

    Damro: Dive into the attention mechanism of lvlm to re- duce object hallucination.arXiv preprint arXiv:2410.04514,

    Xuan Gong, Tianshi Ming, Xinpeng Wang, and Zhihua Wei. Damro: Dive into the attention mechanism of lvlm to re- duce object hallucination.arXiv preprint arXiv:2410.04514,

  4. [12]

    Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Con- ghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation. InProceedings of the IEEE/CVF Confere...

    2024

  5. [13]

    Hallucination augmented contrastive learn- ing for multimodal large language model

    Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learn- ing for multimodal large language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  6. [14]

    Interpreting and editing vision-language rep- resentations to mitigate hallucinations.arXiv preprint arXiv:2410.02762, 2024

    Nick Jiang, Anish Kachinthaya, Suzie Petryk, and Yossi Gandelsman. Interpreting and editing vision-language rep- resentations to mitigate hallucinations.arXiv preprint arXiv:2410.02762, 2024. 7

    2024 arXiv

  7. [15]

    Devils in middle layers of large vision- language models: Interpreting, detecting and mitigating ob- ject hallucinations via attention lens

    Zhangqi Jiang, Junkai Chen, Beier Zhu, Tingjin Luo, Yankun Shen, and Xu Yang. Devils in middle layers of large vision- language models: Interpreting, detecting and mitigating ob- ject hallucinations via attention lens. InProceedings of the Computer Vision and Pattern Recogniti...

    2025

  8. [16]

    What’s in the im- age? a deep-dive into the vision of vision language models

    Omri Kaduri, Shai Bagon, and Tali Dekel. What’s in the im- age? a deep-dive into the vision of vision language models. InProceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 14549–14558, 2025. 1

    2025

  9. [17]

    See what you are told: Visual attention sink in large multimodal models.arXiv preprint arXiv:2503.03321, 2025

    Seil Kang, Jinyeong Kim, Junhyeok Kim, and Seong Jae Hwang. See what you are told: Visual attention sink in large multimodal models.arXiv preprint arXiv:2503.03321, 2025. 1, 3

    2025 arXiv

  10. [18]

    Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogni...

    2024

  11. [19]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InIn- ternational conference on machine learning, pages 19730– 19742. PMLR, 2023. 1, 2

    2023

  12. [20]

    Mvbench: A comprehensive multi-modal video understand- ing benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understand- ing benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195– ...

    2024

  13. [21]

    Contrastive decoding: Open-ended text genera- tion as optimization

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Ja- son Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text genera- tion as optimization. InProceedings of the 61st Annual Meet- ing of the Association for Computational ...

    2023

  14. [22]

    Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023. 1

    2023 arXiv

  15. [23]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 3

    2014

  16. [24]

    Mitigating hallucination in large multi-modal models via robust instruction tuning.arXiv preprint arXiv:2306.14565, 2023

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Ya- coob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning.arXiv preprint arXiv:2306.14565, 2023. 2

    2023 arXiv

  17. [25]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 1, 2

    2023

  18. [26]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024. 2, 3, 8

    2024

  19. [27]

    A survey on hallucination in large vision-language models

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiu- tian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253, 2024. 1

    2024 arXiv

  20. [28]

    Paying more at- tention to image: A training-free method for alleviating hal- lucination in lvlms

    Shi Liu, Kecheng Zheng, and Wei Chen. Paying more at- tention to image: A training-free method for alleviating hal- lucination in lvlms. InEuropean Conference on Computer Vision, pages 125–140. Springer, 2024. 1, 3, 5, 7, 8

    2024

  21. [29]

    Alleviating hallucinations in large vision- language models through hallucination-induced optimiza- tion.Advances in Neural Information Processing Systems, 37:122811–122832, 2024

    Xinyu Lyu, Beitao Chen, Lianli Gao, Hengtao Shen, and Jingkuan Song. Alleviating hallucinations in large vision- language models through hallucination-induced optimiza- tion.Advances in Neural Information Processing Systems, 37:122811–122832, 2024. 2

    2024

  22. [30]

    Wordnet: a lexical database for english

    George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995. 6

    1995

  23. [31]

    Interpreting gpt: The logit lens

    nostalgebraist. Interpreting gpt: The logit lens. https : / / www . lesswrong . com / posts / AcKRB8wDpdaN6v6ru / interpreting - gpt - the-logit-lens, 2020. 2, 4

    2020

  24. [32]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  25. [33]

    Object hallucination in image cap- tioning.arXiv preprint arXiv:1809.02156, 2018

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image cap- tioning.arXiv preprint arXiv:1809.02156, 2018. 4, 6, 7

    2018 arXiv

  26. [34]

    Mitigating ob- ject hallucination in mllms via data-augmented phrase-level alignment.arXiv preprint arXiv:2405.18654, 2024

    Pritam Sarkar, Sayna Ebrahimi, Ali Etemad, Ahmad Beirami, Sercan ¨O Arık, and Tomas Pfister. Mitigating ob- ject hallucination in mllms via data-augmented phrase-level alignment.arXiv preprint arXiv:2405.18654, 2024. 1

    2024 arXiv

  27. [35]

    Over- confidence in llm-as-a-judge: Diagnosis and confidence- driven solution.arXiv preprint arXiv:2508.06225, 2025

    Zailong Tian, Zhuoheng Han, Yanzhe Chen, Haozhe Xu, Xi Yang, Richeng Xuan, Houfeng Wang, and Lizi Liao. Over- confidence in llm-as-a-judge: Diagnosis and confidence- driven solution.arXiv preprint arXiv:2508.06225, 2025. 7

    2025 arXiv

  28. [36]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024. 1, 3

    2024

  29. [37]

    Mllm can see? dynamic correction decoding for hallucination mitiga- tion.arXiv preprint arXiv:2410.11779, 2024

    Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, and Huajun Chen. Mllm can see? dynamic correction decoding for hallucination mitiga- tion.arXiv preprint arXiv:2410.11779, 2024. 3, 7

    2024 arXiv

  30. [38]

    Amber: An llm-free multi-dimensional bench- mark for mllms hallucination evaluation.arXiv preprint arXiv:2311.07397, 2023

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang, et al. Amber: An llm-free multi-dimensional bench- mark for mllms hallucination evaluation.arXiv preprint arXiv:2311.07397, 2023. 6, 7

    2023 arXiv

  31. [39]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  32. [40]

    When language overrules: Revealing text domi- nance in multimodal large language models.arXiv preprint arXiv:2508.10552, 2025

    Huyu Wu, Meng Tang, Xinhan Zheng, and Haiyun Jiang. When language overrules: Revealing text domi- nance in multimodal large language models.arXiv preprint arXiv:2508.10552, 2025. 1, 3

    2025 arXiv

  33. [41]

    Mitigating hallucinations in large vision- language models via dpo: On-policy data hold the key

    Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, and Dongsheng Li. Mitigating hallucinations in large vision- language models via dpo: On-policy data hold the key. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10610–10620, 2025. 2

    2025

  34. [42]

    mplug- owl2: Revolutionizing multi-modal large language model with modality collaboration

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug- owl2: Revolutionizing multi-modal large language model with modality collaboration. InProceedings of the ieee/cvf conference on computer vision and pattern recognition, p...

    2024

  35. [43]

    Clearsight: Vi- sual signal enhancement for object hallucination mitigation in multimodal large language models

    Hao Yin, Guangzong Si, and Zilei Wang. Clearsight: Vi- sual signal enhancement for object hallucination mitigation in multimodal large language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 14625–14634, 2025. 1, 3

    2025

  36. [44]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback. InProceedings of the IEEE/CVF Conference on C...

    2024

  37. [45]

    Mitigating object hallucination in large vision-language models via classifier-free guidance.arXiv e-prints, pages arXiv–2402, 2024

    Linxi Zhao, Yihe Deng, Weitong Zhang, and Quanquan Gu. Mitigating object hallucination in large vision-language models via classifier-free guidance.arXiv e-prints, pages arXiv–2402, 2024. 1, 3

    2024

  38. [46]

    Beyond hallucinations: Enhanc- ing lvlms through hallucination-aware direct preference op- timization.arXiv preprint arXiv:2311.16839, 2023

    Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Ji- aqi Wang, and Conghui He. Beyond hallucinations: Enhanc- ing lvlms through hallucination-aware direct preference op- timization.arXiv preprint arXiv:2311.16839, 2023. 2

    2023 arXiv

  39. [47]

    Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754,

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.