REVIEW 4 major objections 6 minor 3 cited by
Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that object hallucination in large vision-language models arises from modality bias—attention concentrated on either image or text—and can be reduced at inference time by amplifying attention to the neglected modality.
desk verdict Interesting split in hallucination attention patterns, but the causal claim is undercut by per-benchmark hyperparameter tuning and a missing control. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central objects are the textual attention ratio (TAR) and visual attention ratio (VAR), the cumulative attention a generated token pays to instruction text and image tokens. The method, TV AI, modifies the pre-softmax attention scores: it adds $\alpha|A|$ to text-token entries and $\beta|A|$ to image-token entries (Eqs. 7–8), applied only in layers after attention-sink patterns appear, and then mixes the modified distribution with the original via contrastive decoding ($p_{\mathrm{final}} = \gamma p' + (1-\gamma)p$, Eq. 9). These ratios do the explanatory work: the paper shows that hallucinated tokens have imbalanced TAR/VAR while correct tokens are balanced, and the intervention directly targets that imbalance.
What would settle it
A causal test would hold a model's decoding fixed and, on a fresh set of images, record for each token its TAR/VAR imbalance before generation; if attention imbalance is causal, tokens with high predicted imbalance should be more likely to be hallucinated, and random non-targeted perturbations of attention of the same magnitude should not reproduce TV AI's improvements. Concretely, one could compare hallucination rates when attention is boosted toward the neglected modality versus when it is boosted toward the already-dominant modality: if the latter also reduces hallucination, modality-bias rebalancing is not the operative mechanism.
Extended reading notes
Core claim
The paper's central discovery is that hallucinated tokens in LVLMs show two opposite attention signatures, not a single language-prior failure. For generative hallucinations, the model's attention is concentrated on visual tokens, while for discriminative hallucinations, attention concentrates on textual tokens; non-hallucinated tokens show balanced attention to both modalities. The paper claims this modality bias is a primary driver of object hallucination and demonstrates that reweighting attention—adding a scaled absolute value of the original attention to text or image tokens in layers after attention sinks appear, plus a contrastive decoding term—reduces hallucination across LLaVA-1.5, MiniGPT-4, Shikra, and Qwen-VL-Chat on CHAIR and POPE, and preserves general capability on MMBench.
Load-bearing premise
The claim rests on the assumption that the measured attention imbalance during hallucinated tokens is a cause of the hallucination, not an after-effect: if that imbalance is only a symptom of some deeper mechanism, rebalancing attention would not be a principled fix and the reported gains would be incidental.
Editorial extensions
If this is right
- Hallucination can be mitigated without retraining or extra data, purely by modifying attention at inference time.
- Because the intervention operates on attention weights generically, it can be layered on top of any decoding strategy and applied to any LVLM architecture.
- The two-pattern taxonomy implies that a single intervention that only boosts image attention—as earlier methods do—is incomplete; text attention also needs strengthening when the model ignores the question.
- Combining attention rebalancing with contrastive decoding yields larger reductions than either alone, indicating the two mechanisms target complementary failure modes.
Reading between the lines
- If modality bias is as widespread as the paper's attention measurements suggest, then hallucination may be partially predictable from attention statistics alone, which could lead to cheap detectors that flag imbalanced TAR/VAR tokens during decoding.
- The same rebalancing principle could transfer to other multimodal settings—audio-text or video-text models—where a token's attention is split across input streams, though layer selection and intervention strength would need to be re-derived.
- The paper's static intervention with fixed $\alpha$, $\beta$ is a simplification; a dynamic controller that reads the current token's attention distribution and adjusts intervention per layer could adapt better across domains and models, a direction the authors themselves flag as future work.
- If the reported MMBench results hold on larger and more recent models, attention-level diagnosis might become a routine debugging step before committing to expensive fine-tuning.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that object hallucination in large vision-language models (LVLMs) is driven by "modality bias": during hallucinated generation, models over-attend to either the textual or the visual modality rather than jointly attending to both. It distinguishes generative hallucination (claimed to be visual-overfocus) from discriminative hallucination (claimed to be textual-overfocus) using two attention-ratio metrics, TAR and VAR. The proposed training-free method, TVAI, adds positive pre-softmax biases to text and image attention weights (Eqs. 7–8) and combines this with a contrastive decoding step (Eq. 9). Experiments on CHAIR, POPE, and MMBench across LLaVA-1.5, MiniGPT-4, and Shikra report consistent hallucination reduction relative to vanilla and to prior methods such as OPERA, VCD, and PAI. The paper also releases code and includes a reproducibility checklist.
Significance. If the causal claim were established, the paper would make a useful practical contribution: a training-free, inference-time intervention that reduces object hallucination across several LVLMs would be valuable, and the attention-ratio analysis is a simple diagnostic tool that others could reuse. The authors deserve credit for testing on multiple models and benchmarks, for reporting ablations of the separate text-only and image-only intervention variants, and for making code available. However, the current evidence does not uniquely support the central causal claim. The intervention's success is consistent with a generic boost of instruction-token attention rather than with modality rebalancing, and the hyperparameters are tuned separately on each evaluation benchmark. These issues are load-bearing because the paper's main novelty is the explanation of hallucination as modality bias, not merely the empirical observation that a particular intervention works. With additional control experiments and a corrected formulation of Eq. (9), the work could be made sound; as it stands, the manuscript overclaims.
major comments (4)
- [Eq. (9)] Equation (9) defines p_final = gamma * p' + (1 - gamma) * p with gamma > 1, where p' and p are conditional probability distributions over the vocabulary. For gamma > 1, the coefficient of p is negative, so p_final is not a probability distribution: it can have negative entries and does not sum to 1. The paper does not specify any clipping, renormalization, or alternative interpretation (e.g., log-probability interpolation). Since gamma values up to 1.5 are used in Table 4, the method as stated is mathematically undefined. Please specify the exact operation actually performed in the implementation (e.g., softmax over the interpolated logits, or clipping and renormalizing) and correct Eq. (9) accordingly.
- [Supplementary Material, Tables 6–8; main-text Ablation Studies] The hyperparameters alpha, beta, and gamma are tuned separately for each benchmark (CHAIR, POPE, MMBench) and each model, with the final values selected after inspecting the same benchmark results (Table 4 in the main text shows the selection for LLaVA-1.5 on CHAIR and for MiniGPT-4). This is effectively test-set tuning: the reported gains are optimistic estimates of what would be achieved on unseen data. No validation split, cross-validation procedure, or held-out hyperparameter choice is described. Additionally, no error bars, repeated-run statistics, or significance tests appear anywhere in the paper, despite the Reproducibility Checklist answering "yes" to items 4.10 (number of runs), 4.11 (measures of variation, confidence), and 4.12 (statistical tests). This discrepancy is concerning and must be addressed; the "generalization" claim across benchmarks should be backed by either fixed hyperparameters or an honest selection protocol with uncertainty estimates.
- [Methodology, Eqs. (7)–(8); Table 1] The causal interpretation of modality bias is confounded by a generic instruction-attention boost. Equations (7) and (8) add positive quantities to both text-token and image-token attention scores before softmax. Because softmax is normalized over the full sequence, this necessarily reduces attention to previously generated tokens and attention sinks, increasing the total probability mass assigned to instruction tokens. The TAR and VAR metrics (Eqs. 5–6) sum only over text and image instruction tokens, so they cannot distinguish "balanced text/image attention" from "more attention to instructions overall". The ablations in Table 1 are consistent with the latter explanation: TAI-only (alpha intervention only) and VAI-only (beta intervention only) each improve over vanilla on CHAIR despite moving attention in opposite directions (35.8 and 36.8 vs 47.6 for CHAIRS). The common factor is boosting instruction-token attention. To support the claim that modality imbalance is the causal driver, the authors should include a control condition that boosts attention to a matched set of non-instruction tokens (e.g., previously generated tokens or a fixed set of indices) and show that this control does not produce similar hallucination reduction. Without such a control, the reported gains do not uniquely validate the modality-balance mechanism.
- [Supplementary Material, Table 7; Section "Generative and Discriminative Hallucinations"] There is an internal inconsistency between the paper's taxonomy and the chosen hyperparameters. The paper states that discriminative hallucination arises from textual overfocus and therefore should be remedied by increasing visual attention (a visual boost, i.e., beta > alpha). However, for the POPE adversarial configuration on LLaVA-1.5 (Table 7), the hyperparameters are alpha = 0.93, beta = 0.5, so the textual intervention is stronger than the visual intervention. This is the opposite of what the mechanism predicts. Similarly, the MMBench configuration uses alpha = beta = 0.5, which is neutral. Please explain this discrepancy, or adjust the hyperparameter selection so that the chosen settings are consistent with the proposed causal account.
minor comments (6)
- [Methodology, first paragraph] The phrase "as described in Section" is missing the section number (likely referring to the Introduction or Related Work). Please correct the cross-reference.
- [Figure 1] The Figure 1 caption contains the stray string "55323 199940", which appears to be a LaTeX artifact; it should be removed.
- [Eqs. (5)–(8)] The notation "tnTX", "vnVX", and similar expressions is not formally introduced and is hard to read. Please use clear index sets, e.g., sum over text token indices {t_1, ..., t_T} and visual token indices {v_1, ..., v_V}, and make explicit that the intervention is applied row-wise to the attention matrix before softmax.
- [Table 4] Table 4 is split into three blocks, but the caption says "left and middle" for LLaVA-1.5 and "right" for MiniGPT-4. It would be clearer to label each block directly (e.g., "LLaVA-1.5, alpha/beta sweep" and "LLaVA-1.5, gamma sweep") to avoid ambiguity.
- [Experiments] The abstract and introduction say experiments were run on four LVLMs, but the quantitative evaluation tables cover only LLaVA-1.5, MiniGPT-4, and Shikra; Qwen-VL-Chat appears only in the attention-pattern analysis. Please clarify the scope, or add Qwen-VL-Chat to the main evaluation tables.
- [Reproducibility Checklist] The checklist item 3.3 says "All novel datasets introduced in this paper are included in a data appendix," but no novel dataset or data appendix is present. Please either mark this item as NA or add the appendix.
Circularity Check
Benchmark-specific hyperparameter fitting makes the reported hallucination-mitigation gains partially forced; the diagnostic modality-bias observation itself is not circular.
-
fitted input called prediction
[Ablation Studies (Table 4) and Supplementary Implementation Details (Tables 6-8)]
"Results are reported for LLaV A-1.5 (left and middle) and MiniGPT-4 (right). The configuration of the main experimental results in this study is highlighted as bold font. [...] Given that datasets such as CHAIR (Rohrbach et al. 2018), POPE (Li et al. 2023), and MMBench (Liu et al. 2024d) have small sample sizes and highly imbalanced distributions, different hyperparameter configurations are required to adapt to these small datasets, with specific settings detailed in Table 6, 7, and 8."
The intervention strength parameters (α, β, γ) are fitted per benchmark on the very CHAIR, POPE, and MMBench splits whose scores are then reported as evidence of the method's 'generalizability and efficacy'. Table 4 shows CHAIR metrics across a grid of α/β/γ and explicitly marks the setting used for the main results; the supplementary confirms that separate configurations are selected for each small benchmark. Thus the headline improvements are in-sample evaluations, not independent predictions: the benchmark performance is the optimization target used to choose the hyperparameters, so the reported gains are partly forced by construction. The TAR/VAR diagnostic is an independent empirical measurement and is not circular, but the mitigation evidence is partially fitted.
full rationale
The paper's causal chain—measure attention to text and image tokens on hallucinated outputs, label the imbalance 'modality bias', intervene to rebalance attention, and report reduced hallucination—is not self-definitional in its diagnostic part: TAR and VAR (Eqs. 5-6) are direct sums of attention weights, and hallucination labels come from external CHAIR/POPE annotations, not from the method's equations. No load-bearing self-citation chain or imported uniqueness theorem appears; citations to PAI and attention-sink work are used for experimental configuration and background. However, one evaluation step reduces to its own input: α, β, and γ are selected by per-benchmark performance on CHAIR, POPE, and MMBench, and the same benchmarks are then used to demonstrate the method's superiority. This makes the headline mitigation results partially a product of benchmark-specific fitting rather than an independent test of the modality-bias hypothesis. The ablation confound—boosting both modalities increases instruction attention generally—further weakens the causal interpretation, but that is a validity concern rather than a formal circular reduction. Overall, a partial circularity score of 6 reflects the fitted-input-called-prediction pattern in the evaluation, while the empirical modality-bias diagnosis retains independent content.
Assumptions & free parameters
free parameters (4)
- alpha (text attention intervention strength) =
0.93 (LLaVA-1.5 CHAIR/POPE), 0.8 (MiniGPT-4 CHAIR, Shikra CHAIR), 0.1 (MiniGPT-4 POPE), 0.5 (LLaVA-1.5 MMBench)
- beta (visual attention intervention strength) =
0.5 (LLaVA-1.5 CHAIR/POPE/MMBench), 0.4 (MiniGPT-4 CHAIR), 0.6 (Shikra CHAIR), 0.1 (MiniGPT-4 POPE)
- gamma (contrast decoding strength) =
1.1 (LLaVA-1.5 CHAIR/POPE/MMBench), 1.1 (MiniGPT-4 CHAIR), 1.2 (Shikra CHAIR), 1.1 (Shikra POPE), 1.2 (LLaVA-1.5 POPE)
- Intervention layer range / excitation location =
Not specified; described as determined by divergence of attention patterns (Fig. 4)
assumptions (5)
- standard math Standard transformer self-attention with softmax normalization as in Eq. 1-2.
- domain assumption Pretrained visual encoders and language decoders have robust comprehension capabilities for their respective modalities.
- domain assumption Attention sink patterns emerge in deeper layers, indicating attentional redundancy.
- domain assumption CHAIR and POPE ground truth correctly identify hallucinated and non-hallucinated tokens.
- ad hoc to paper Hallucination can be cleanly divided into generative and discriminative classes with distinct attention patterns.
Cite this review
Pith. "Pith review of Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens." pith.science (2026). https://pith.science/paper/AMX3BAS5
@misc{pith2026250802419,
author = {Pith},
title = {Pith review of: Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens},
year = {2026},
howpublished = {\url{https://pith.science/paper/AMX3BAS5}},
note = {Machine review of arXiv:2508.02419}
}
read the original abstract
Large vision-language models (LVLMs) have demonstrated remarkable multimodal comprehension and reasoning capabilities, but they still suffer from severe object hallucination. Previous studies primarily attribute the flaw to linguistic prior caused by the scale mismatch between visual encoders and large language models (LLMs) in LVLMs. Specifically, as current LVLMs are built upon LLMs, they tend to over-rely on textual prompts and internal knowledge of LLMs, generating descriptions inconsistent with visual cues. However, through an in-depth investigation of the hallucinated mechanisms, we empirically reveal a previously overlooked phenomenon: LVLMs may ignore not only visual information but also textual modality during hallucination, a behavior termed as modality bias, which indicates that LVLMs struggle to simultaneously attend to both visual and textual modalities, leading to fragmented understanding of user-provided instructions. Based on this observation, we propose a simple yet effective training-free method to mitigate object hallucination. Concretely, we intervene and adjust the attention weights of textual and visual tokens, balancing cross-modal compatibility for better alignment with user intentions. Furthermore, we adopt a contrastive decoding strategy to reduce the LVLM's overreliance on its parametric knowledge, synergistically enhancing our attention manipulation. Extensive experiments confirm the widespread presence of modality bias in LVLMs. Notably, our method effectively mitigates hallucination across multiple open-source LVLMs and benchmarks, highlighting its generalizability and efficacy.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 3 Pith papers
-
Personalizing MLLMs via Reinforced Multimodal Reference Game
RRG trains MLLMs via a reinforced multimodal reference game with contrastive rewards on hard positives and negatives to produce accurate, discriminative concept descriptions, achieving SOTA on personalization benchmarks.
-
OmniDrive-R1: Reinforcement-driven Interleaved Multi-modal Chain-of-Thought for Trustworthy Vision-Language Autonomous Driving
OmniDrive-R1 boosts VLM reasoning score from 51.77% to 80.35% and answer accuracy from 37.81% to 73.62% on DriveLMM-o1 via reinforcement-driven interleaved multi-modal chain-of-thought with annotation-free grounding.
-
Evaluating Stochastic Collapse and Implicit Bias in Multimodal Large Language Models
MLLMs show stochastic collapse with top-1 probabilities up to 97% and low randomness indices when choosing among equivalent options.
Reference graph
Works this paper leans on
-
[1]
L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al
Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774
arXiv 2023
-
[2]
An, W.; Tian, F.; Leng, S.; Nie, J.; Lin, H.; Wang, Q.; Dai, G.; Chen, P.; and Lu, S. 2024. Agla: Mitigating object hallucinations in large vision-language models with assembly of global and local attention. arXiv preprint arXiv:2406.12718
arXiv 2024
-
[4]
Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023 b . Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1(2): 3
arXiv 2023
-
[5]
D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al
Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877--1901
2020
-
[6]
Chen, J.; Zhang, T.; Huang, S.; Niu, Y.; Zhang, L.; Wen, L.; and Hu, X. 2024 a . ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Models. arXiv preprint arXiv:2411.15268
arXiv 2024
-
[7]
Chen, K.; Zhang, Z.; Zeng, W.; Zhang, R.; Zhu, F.; and Zhao, R. 2023. Shikra: Unleashing multimodal llm's referential dialogue magic. arXiv preprint arXiv:2306.15195
arXiv 2023
-
[8]
Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B. 2024 b . An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, 19--35. Springer
2024
-
[9]
Chen, Z.; Zhao, Z.; Luo, H.; Yao, H.; Li, B.; and Zhou, J. 2024 c . Halc: Object hallucination reduction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.00425
arXiv 2024
Show all 46 references
-
[10]
E.; et al
Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3): 6
2023
-
[11]
Cho, J.; Yoon, S.; Kale, A.; Dernoncourt, F.; Bui, T.; and Bansal, M. 2022. Fine-grained image captioning with clip reward. arXiv preprint arXiv:2205.13115
2022 arXiv
-
[12]
Chuang, Y.-S.; Qiu, L.; Hsieh, C.-Y.; Krishna, R.; Kim, Y.; and Glass, J. 2024. Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps. arXiv preprint arXiv:2407.07071
2024 arXiv
-
[13]
Chuang, Y.-S.; Xie, Y.; Luo, H.; Kim, Y.; Glass, J.; and He, P. 2023. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883
2023 arXiv
-
[14]
Cui, C.; Ma, Y.; Cao, X.; Ye, W.; Zhou, Y.; Liang, K.; Chen, J.; Lu, J.; Yang, Z.; Liao, K.-D.; et al. 2024. A survey on multimodal large language models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 958--979
2024
-
[15]
Darcet, T.; Oquab, M.; Mairal, J.; and Bojanowski, P. 2023. Vision transformers need registers. arXiv preprint arXiv:2309.16588
2023 arXiv
-
[16]
Gunasekar, S.; Zhang, Y.; Aneja, J.; Mendes, C. C. T.; Del Giorno, A.; Gopi, S.; Javaheripi, M.; Kauffmann, P.; de Rosa, G.; Saarikivi, O.; et al. 2023. Textbooks are all you need. arXiv preprint arXiv:2306.11644
2023 arXiv
-
[17]
Gunjal, A.; Yin, J.; and Bas, E. 2024. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 18135--18143
2024
-
[18]
Huang, L.; Yu, W.; Ma, W.; Zhong, W.; Feng, Z.; Wang, H.; Chen, Q.; Peng, W.; Feng, X.; Qin, B.; et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2): 1--55
2025
-
[19]
Huang, Q.; Dong, X.; Zhang, P.; Wang, B.; He, C.; Wang, J.; Lin, D.; Zhang, W.; and Yu, N. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vis...
2024
-
[20]
Jiang, Z.; Chen, J.; Zhu, B.; Luo, T.; Shen, Y.; and Yang, X. 2024. Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallucinations via attention lens. arXiv preprint arXiv:2411.16724
2024 arXiv
-
[21]
Lee, K.; Ippolito, D.; Nystrom, A.; Zhang, C.; Eck, D.; Callison-Burch, C.; and Carlini, N. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499
2021 arXiv
-
[22]
Leng, S.; Zhang, H.; Chen, G.; Li, X.; Lu, S.; Miao, C.; and Bing, L. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13872--13882
2024
-
[23]
Li, C.; Wong, C.; Zhang, S.; Usuyama, N.; Liu, H.; Yang, J.; Naumann, T.; Poon, H.; and Gao, J. 2024 a . Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36
2024
-
[24]
Li, L.; Xie, Z.; Li, M.; Chen, S.; Wang, P.; Chen, L.; Yang, Y.; Wang, B.; Kong, L.; and Liu, Q. 2024 b . VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment. arXiv preprint arXiv:2410.09421
2024 arXiv
-
[25]
X.; and Wen, J.-R
Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, W. X.; and Wen, J.-R. 2023. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355
2023 arXiv
-
[26]
Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer vision--ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 1...
2014
-
[27]
Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26296--26306
2024
-
[28]
Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916
2023
-
[29]
Liu, H.; Xue, W.; Chen, Y.; Chen, D.; Zhao, X.; Wang, K.; Hou, L.; Li, R.; and Peng, W. 2024 b . A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253
2024 arXiv
-
[30]
Liu, S.; Ye, H.; Xing, L.; and Zou, J. 2024 c . Reducing hallucinations in vision-language models via latent space steering. arXiv preprint arXiv:2410.15778
2024 arXiv
-
[31]
Liu, S.; Zheng, K.; and Chen, W. 2024. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision, 125--140. Springer
2024
-
[32]
Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; et al. 2024 d . Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, 216--233. Springer
2024
-
[33]
Peng, B.; Galley, M.; He, P.; Cheng, H.; Xie, Y.; Hu, Y.; Huang, Q.; Liden, L.; Yu, Z.; Chen, W.; et al. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813
2023 arXiv
-
[34]
W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al
Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR
2021
-
[35]
A.; Burns, K.; Darrell, T.; and Saenko, K
Rohrbach, A.; Hendricks, L. A.; Burns, K.; Darrell, T.; and Saenko, K. 2018. Object hallucination in image captioning. arXiv preprint arXiv:1809.02156
2018 arXiv
-
[36]
Tian, X.; Gu, J.; Li, B.; Liu, Y.; Wang, Y.; Zhao, Z.; Zhan, K.; Jia, P.; Lang, X.; and Zhao, H. 2024. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289
2024 arXiv
-
[37]
Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971
2023 arXiv
-
[38]
N.; Kaiser, .; and Polosukhin, I
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30
2017
-
[39]
Wang, C.; Chen, X.; Zhang, N.; Tian, B.; Xu, H.; Deng, S.; and Chen, H. 2024. Mllm can see? dynamic correction decoding for hallucination mitigation. arXiv preprint arXiv:2410.11779
2024 arXiv
-
[40]
Wu, J.; Liu, Q.; Wang, D.; Zhang, J.; Wu, S.; Wang, L.; and Tan, T. 2024. Logical closed loop: Uncovering object hallucinations in large vision-language models. arXiv preprint arXiv:2402.11622
2024 arXiv
-
[41]
Xiao, G.; Tian, Y.; Chen, B.; Han, S.; and Lewis, M. 2023. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453
2023 arXiv
-
[42]
Xu, Y.; Wang, Y.; Zhou, F.; Ma, J.; Yang, S.; Lin, H.; Wang, X.; Wang, J.; Liang, L.; Han, A.; et al. 2024. A multimodal knowledge-enhanced whole-slide pathology foundation model. arXiv preprint arXiv:2407.15362
2024 arXiv
-
[43]
Zhao, H.; Si, S.; Chen, L.; Zhang, Y.; Sun, M.; Zhang, M.; and Chang, B. 2024. Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance. arXiv preprint arXiv:2411.14279
2024 arXiv
-
[44]
Zhou, Y.; Cui, C.; Yoon, J.; Zhang, L.; Deng, Z.; Finn, C.; Bansal, M.; and Yao, H. 2023. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754
2023 arXiv
-
[45]
Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592
2023 arXiv
-
[46]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[47]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.