Pith. sign in

REVIEW 3 major objections 6 minor 88 references

INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding game-theory interaction scores to key tokens cuts LVLM hallucinations.

desk verdict A useful empirical decoding patch for LVLM hallucination wrapped in a game-theoretic story whose central interaction term rests on unvalidated counterfactual baselines. read the letter →

arxiv 2507.05056 v2 pith:QXEM2AFD submitted 2025-07-07 cs.CV cs.AI

Xin Dong , Shichao Dong , Jin Wang , Jing Huang , Li Zhou , Zenghui Sun , Lihua Jing , Jingsong Lan
show 2 more authors
Xiaoyong Zhu Bo Zheng
This is my paper
classification cs.CVcs.AI
keywords hallucinationmitigationlargevision-languagemodelsHarsanyidividendmultimodalinteractiondecodingstrategytraining-freegametheorysamplingcorrection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that large vision-language models, like people, lean on the combined image-text meaning of a sample rather than on either modality alone, and that this multimodal interaction mainly shapes the few content words of an answer. The authors measure that interaction with the Harsanyi dividend from game theory and find three things: the interaction exists, it is concentrated on keyword tokens, and stronger interaction on a keyword predicts a more accurate answer. They then turn the measurement into a decoding rule, INTER, which adds the interaction logit to keyword tokens during sampling. Because the rule needs no training and no extra data, it is a drop-in correction for any autoregressive vision-language model and any existing decoding strategy; on six benchmarks it improves scores by up to 3.4% on average across five models.

What carries the argument

The load-bearing object is the Harsanyi dividend $I(A|\{v,p\})_{y_t}$, a game-theoretic measure of how much the coalition of image and text contributes to the logit of candidate token $y_t$ beyond what each modality contributes alone. It is computed by taking the logit on the full input minus the logits on the image-only and text-only counterfactual inputs plus the logit on the empty input, per Eq. 3. INTER uses two modules built from it: the Interactive Guided Locator flags a step as a keyword step when the variance of $I(A)$ over the vocabulary exceeds a threshold $k$, and the Interaction Probability Modifier adds $I(A)$ to the logits of those steps before softmax. The variance gate is what keeps the correction on content words and off grammatical connectives, preserving fluency.

What would settle it

Compute $I(A)$ using a different null state for the missing modality—for example, a black or blurred image instead of random noise, or a padded duplicate of the prompt instead of an empty string—and rerun INTER on POPE. If the hallucination reduction largely disappears or flips sign, the interaction term is an artifact of the particular counterfactual rather than a genuine multimodal signal.

Watch

Extended reading notes

Core claim

The central discovery is that multimodal interaction information, computed as the Harsanyi dividend of the image and prompt players for each candidate token, is a usable, training-free steering signal for generation. Concretely, the paper proposes to replace the sampling distribution at keyword positions with $P_t = \mathrm{SoftMax}(M_\theta(v,p,y_{<t}) + \beta \cdot I(A))$, where $I(A)$ is the image-text interaction logit for $A=\{v,p\}$ and $\beta$ is a 0/1 gate set by whether the variance of $I(A)$ across the vocabulary exceeds a threshold. The authors establish the three premises behind this rule—interactions exist in LVLM decoding, they are concentrated on keywords, and they correlate positively with correctness—and then demonstrate that the rule reduces object- and attribute-level hallucinations on POPE, MME, MM-Bench, MMStar, CHAIR, and LLaVA-Bench across five LVLMs and multiple decoding strategies, including the current state-of-the-art InternVL2.5-MPO.

Load-bearing premise

The method assumes that the counterfactual inputs—an empty text prompt paired with the image, and a random-noise image paired with the text—cleanly isolate what each modality alone contributes, so that subtracting them leaves a true measure of image-text interaction; if those artificial states do not isolate modality contributions, the locator and modifier are steering by an artifact.

Editorial extensions

If this is right

  • Adding interaction logits to keyword tokens reduces object and attribute hallucinations on POPE, MME, MM-Bench, MMStar, CHAIR, and LLaVA-Bench without any training.
  • The correction composes with existing decoding strategies, improving nucleus sampling, beam search, greedy search, VCD, and OPERA when applied on top of them.
  • The method transfers across model families (InstructBLIP, LLaVA-v1.5, Qwen-VL, mPLUG-Owl2) and across parameter scales from 1B to 26B.
  • The three verified insights imply that hallucination is partly a decision-time failure to reuse multimodal interaction knowledge the model already has, not only a knowledge gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the interaction logits are a genuine steering signal, they could be combined with other training-free methods such as contrastive decoding by summing corrections, potentially yielding larger gains than either alone; the paper does not test this combination.
  • The variance gate appears to be an implicit part-of-speech detector, flagging content words without explicit tagging; a testable extension is whether using explicit POS tags instead of the variance threshold gives the same or better hallucination reduction.
  • The MME correlation analysis suggests that interaction strength on the 'yes'/'no' keyword predicts answer correctness; this could be turned into a hallucination detector that flags low-interaction keyword decisions at inference time.
  • Because the counterfactual inputs are cheap to batch, the approach may extend to other multimodal settings such as audio-image or video-text models, where the same Harsanyi construction would measure different interaction coalitions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a training-free decoding intervention, INTER, for reducing hallucinations in large vision-language models (LVLMs). The authors formalize the LVLM decoding logits as a cooperative game over two players, the image v and the prompt p, and use the Harsanyi dividend to define a multimodal interaction logit I({v,p}) for each candidate token. From this quantity they derive three insights: LVLMs do use image-text interactions, these interactions concentrate on a few keyword tokens, and stronger interactions correlate with more accurate responses. Based on these insights, INTER combines an Interactive Guided Locator (IGL), which uses the variance of interaction values to detect keyword tokens, and an Interaction Probability Modifier (IPM), which adds the interaction logits to the original logits before softmax at keyword positions. Experiments on POPE, MME, MM-Bench, MMStar, CHAIR, and LLaVA-Bench across several LVLMs and decoding strategies report consistent improvements over nucleus sampling, beam search, VCD, and OPERA.

Significance. If the proposed mechanism were fully validated, INTER would be a notable contribution: it is training-free, model-agnostic, and appears to improve both VQA and captioning benchmarks across a range of LVLMs, including InternVL2.5-MPO. The paper also ships an unusually broad evaluation matrix, covering multiple models, decoding strategies, and benchmarks, which is a genuine strength. However, the central mechanistic claim depends entirely on the validity of the Harsanyi interaction term defined in Eq. (3). The current counterfactual baselines for 'image-only' and 'text-only' states are not justified, and the insights are to some extent built into the construction of the metric. The empirical gains are interesting and may survive as a heuristic, but the paper does not yet establish that they arise from multimodal interaction understanding rather than from a generic prompt-contrast or noise-contrast correction.

major comments (3)
  1. [§3.2, Eq. (3); §5.1] The interaction term I({v,p}) is computed from logits L(v, empty-text), L(noise, p), and L(noise, empty-text). These counterfactual inputs do not isolate modality absence: removing the prompt changes the task and the expected answer distribution, while Gaussian noise is outside the visual encoder's training distribution. Consequently I({v,p}) conflates task mismatch and out-of-distribution artifacts with genuine multimodal interaction. Because Eq. (3) is the basis for Insights 1–3, the IGL locator, and the IPM modifier, the paper's central explanatory claim that INTER works by 'reapplying multimodal interaction understanding' is not supported by the present evidence. The empirical gains could equally arise from a prompt-contrast or noise-contrast correction. The authors should validate the counterfactual states against alternative no-image and no-prompt baselines, or reframe the method as a contrastive decoding heuristic and substantiate that framing.
  2. [§3.3, Table 1] Insight 1 is near-tautological as stated. For any four distinct logit vectors, the alternating sum in Eq. (3) is generically nonzero, so reporting mean |I(A)| > 0 does not establish that LVLMs 'capture multimodal interactions'; it only shows that the four logit vectors differ. A null model is needed before the existence claim can be distinguished from arbitrary input sensitivity. For example, the authors could compare the measured interaction values with those obtained under random image-prompt pairings or with a dummy-player baseline, and show that real interactions are significantly larger.
  3. [§3.5, Fig. 3; §10, Figs. 7–8] The design loop uses the same benchmarks for insight, threshold selection, and evaluation. Insight 3 is derived from MME yes/no correlations, and the interaction-guidance threshold k is tuned on MME and CHAIR (Figs. 7–8), while MME and CHAIR are also among the six evaluation benchmarks. The reported gains are therefore at risk of selection on the test set. The authors should report results with k fixed a priori or chosen on a separate validation set, or provide a sensitivity analysis on a holdout benchmark not used in any design step.
minor comments (6)
  1. [§14] The heading contains a typo: 'substasks' should be 'subtasks'.
  2. [§4, Eq. (5)] The binary selector β is described in the IGL paragraph but not explicitly defined in Eq. (5); adding β = 1{D_yt(I(A)_yt) > k} directly below the equation would improve readability.
  3. [§4, 'Overall Mechanism'] The paper says IGL 'prevents interference from image-text interactions to contextual connectives, thereby preserving linguistic coherence,' but no coherence metric or linguistic-quality analysis is reported; GPT-4o scores on LLaVA-Bench measure accuracy and detail, not coherence.
  4. [§17] The claim that runtime overhead is 'negligible' because all subset evaluations are batched is not supported by measurements; please report wall-clock or latency comparisons for at least one model.
  5. [Table 4 and §5.2] The text states that INTER achieves 'the highest reduction of 34.6% and 18.9% on CS and CI respectively,' but these numbers are not directly traceable in Table 4; the specific baseline and setting should be identified.
  6. [§3.1] The statement that the Harsanyi dividend 'satisfies the efficiency, linearity, dummy, symmetry axioms' is imprecise: those axioms apply to the Shapley value, and the Harsanyi dividend is the unique decomposition from which the Shapley value is assembled, not itself an axiomatic value.

Circularity Check

2 steps flagged · score 4.0 of 10

MME and CHAIR are used both to build/tune INTER and to evaluate its headline gains, so part of the reported improvement is a fitted-input result; independent benchmarks keep the central claim only partially circular.

  1. fitted input called prediction [Section 3.5 (Insight 3) and Equation (4), evaluated in Table 9]
    "we propose conducting experiments on the MME [19] benchmark for convenient verification. In MME, the keywords in the responses are limited to “yes” or “no”, which are also used to determine the accuracy of the responses. ... The appearance of the yes/no keywords directly indicates the binary correctness of the response. ... Then, we use I(A) ∈ RN to modify the logit of each candidate token and generate sampling probabilities, formulated as follows: ePt = SoftMax(Mθ(v, p, y<t) + I(A))."

    The central premise of IPM is the MME-derived correlation that high interaction values I(A) on the emitted yes/no token correlate with correct answers. Equation (4) then adds the very same I(A) to the softmax logits. For a two-alternative MME question, this addition mechanically raises the probability of whichever yes/no token has the larger interaction value. So the large MME gains in Table 9 are the same correlation re-applied at decoding time rather than an independent confirmation that boosting I(A) improves accuracy. The magnitude is still nontrivial because softmax nonlinearity and sequence context intervene, but the direction of the effect is selected for by the MME-based insight.

  2. fitted input called prediction [Section 3.4 and Section 4 (IGL), with Section 10 parameter analysis; Tables 4 and 9]
    "we sample a subset in CHAIR [46] following OPERA [25] and conduct experiments to analyze the differences in Dyt(I(A)yt) between keywords and contextual connectives. ... Following Sec. 3.4, the module calculates the variance of interactions Dyt(I(A)yt) at t-th step during answer synthesis. ... if the variance exceeds the predefined threshold k, IGL designates β = 1{Dyt(I(A)yt)>k}. ... Through experiments on CHAIR [46] and MME [19] benchmarks, we analyze how the interaction guidance coefficient k affects the performance of INTER."

    The IGL key-token detector is a thresholded variance rule derived from CHAIR keyword/connective annotations. The supplementary parameter analysis then selects k using the same two benchmarks, CHAIR and MME, reporting model-dependent optima such as InstructBLIP at k=1.3 on MME. Consequently, the CHAIR and MME improvements in Tables 4 and 9 are partially fitted-input results: both the locator and its threshold were developed and tuned on the benchmarks used as evidence. The robustness range k ∈ [0.5, 1.5] mitigates the overfitting, but it does not remove the fact that these two headline benchmarks were used in the design loop, unlike POPE, MM-Bench, MMStar, and LLaVA-Bench.

full rationale

The algebraic definition of I(A) as a Harsanyi dividend and its use as an additive logit correction in Equation (5) are self-contained, and INTER is not a formal derivation of benchmark scores but an empirical decoding intervention. The main circularity is evaluative. Insight 3 and the Interaction Probability Modifier are built from MME yes/no correctness, and the Interactive Guided Locator plus its threshold k are derived and tuned from CHAIR annotations and CHAIR/MME scores. The paper then reports large gains on those same MME and CHAIR benchmarks as primary evidence, and on MME the logit addition directly re-applies the yes/no interaction-accuracy correlation that motivated the method. That is a fitted-input evaluation for those two benchmarks. However, INTER also improves POPE, MM-Bench, MMStar, and LLaVA-Bench, which were not used for insight derivation or k-tuning, so the central claim retains independent empirical content beyond the tuned benchmarks. The self-citation [56] is used only as methodological inspiration for applying the Harsanyi dividend to LVLMs and is not load-bearing in the sense of an imported uniqueness theorem. Score 4 reflects this partial, benchmark-driven circularity rather than full equivalence-by-construction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on treating LVLM logits as a cooperative game, on the validity of artificial unimodal counterfactual inputs, on the variance gate identifying keywords, and on an MME-derived correlation generalizing to other tasks. No new physical or architectural entities are introduced. The only fitted quantity is the interaction threshold k.

free parameters (1)
  • Interaction guidance threshold k = default k=1; model-dependent optima, e.g., InstructBLIP k=1.3 on MME
    The variance gate in the Interactive Guided Locator uses k to decide which tokens receive the interaction logit boost. The paper analyzes k on MME and CHAIR and reports model-dependent optimal values, so the reported gains partly depend on selecting k from evaluation data.
assumptions (4)
  • domain assumption LVLM logits M_theta(v,p,y<t) define a cooperative game whose Harsanyi dividends measure true multimodal interactions.
    Eq. 3 computes I(A) from full, image-only, text-only, and empty inputs; the paper does not justify that these algebraic differences correspond to cognitive or semantic interaction.
  • domain assumption Image-only and text-only forward passes are valid counterfactual baselines.
    Section 5.1 substitutes random noise for the image and empty text for the prompt; no validation shows that these states isolate pure modality contributions.
  • domain assumption Variance of I(A) across the vocabulary identifies keyword tokens.
    The Interactive Guided Locator in Section 4 relies on this variance threshold, validated only by AUC values above 0.5 in Figure 2, which is a weak separation.
  • domain assumption Multimodal interaction strength positively causes accuracy beyond the MME yes/no setting.
    Insight 3 is a binned correlation on MME yes/no responses, but INTER applies the same principle to all token types and benchmarks without direct evidence of causality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling." pith.science (2026). https://pith.science/paper/QXEM2AFD

@misc{pith2026250705056,
  author       = {Pith},
  title        = {Pith review of: INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QXEM2AFD}},
  note         = {Machine review of arXiv:2507.05056}
}
read the original abstract

Hallucinations in large vision-language models (LVLMs) pose significant challenges for real-world applications, as LVLMs may generate responses that appear plausible yet remain inconsistent with the associated visual content. This issue rarely occurs in human cognition. We argue that this discrepancy arises from humans' ability to effectively leverage multimodal interaction information in data samples. Specifically, humans typically first gather multimodal information, analyze the interactions across modalities for understanding, and then express their understanding through language. Motivated by this observation, we conduct extensive experiments on popular LVLMs and obtained insights that surprisingly reveal human-like, though less pronounced, cognitive behavior of LVLMs on multimodal samples. Building on these findings, we further propose \textbf{INTER}: \textbf{Inter}action Guidance Sampling, a novel training-free algorithm that mitigate hallucinations without requiring additional data. Specifically, INTER explicitly guides LVLMs to effectively reapply their understanding of multimodal interaction information when generating responses, thereby reducing potential hallucinations. On six benchmarks including VQA and image captioning tasks, INTER achieves an average improvement of up to 3.4\% on five LVLMs compared to the state-of-the-art decoding strategy. The code will be released when the paper is accepted.

Figures

Figures reproduced from arXiv: 2507.05056 by the authors.

Figure 1
Figure 1. Comparisons with existing decoding strategies on the state-of-the-art LVLM InternVL2.5-MPO [10]. Our approach INTER achieved optimal results across six benchmarks. Besides, the detailed output of INTER in a case of complex inputs is pre￾sented. The hallucinated words are highlighted in red. 31, 38, 63]. These models blend visual and textual infor￾mation, enhancing our ability to interpret and interact with the world… view at source ↗
Figure 2
Figure 2. Verifying the scope of interactions using the Receiver Operating Characteristic (ROC) curve. Results (AUC > 0.5) show that there exists a moderate class separation between key￾words tokens and contextual connectives tokens based on the vari￾ance of multimodal interactions. These findings suggest that mul￾timodal interactions primarily impact the the generations of key￾words in LVLMs’ responses. Based on the results … view at source ↗
Figure 3
Figure 3. Verifying the impact of interactions through the Pearson correlation coefficient. The set of figures for each LVLM illustrates the correlation between the accuracy and the multimodal interaction contributions, the correlation between the accuracy and the visual modality contributions, and the correlation between the accuracy and the textual modality contributions on MME [19] benchmark. Each point corresponds to stat… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Overview of INTER. The Interactive Guided Locator (IGL) uses variance of multimodal interactions to identify key￾words of interest. The Interaction Probability Modifier (IPM) uses multimodal interactions to guide the model to sample tokens re￾sponsive to visual informa…
Figure 5
Figure 5. Figure 5: The scores of 14 subtasks on MME benchmark [19]. The radar charts show that the areas filled in purple are significantly larger, suggesting that using INTER results in noticeable improvements. method InstructBLIP [13] LLaVA-v1.5 (7B) [38] Qwen-VL [3] mPLUG-owl2 [63] CO…
Figure 6
Figure 6. Figure 6: The robustness of the selection of hyperparameter (k). [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Parameter analysis of k in Interaction Guide Locator. Evaluation of CI and CS after using different k to guide Beam Search [4, 21, 53] on various lengths on CHAIR [46]. k MME Score LLaVA-v1.5 mPLUG-owl2 InstructBLIP 1700 1550 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Parameter analysis of k on MME [19]. Each value represents total score of using INTER on Beam Search [4, 21, 53]. tions at non-critical positions, as evidenced by the perfor￾mance drop compared to k = 1.0 [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Prompts of GPT-4o [28] for evaluations. Max Token method InstructBLIP [13] LLaVA-v1.5 (7B) [38] mPLUG-owl2 [63] CS CI CS CI CS CI ◦ Greedy [49] 26.2 13.8 22.0 6.7 23.0 8.3 64 • Greedy+INTER 25.8 9.2 22.0 6.7 20.6 7.9 ◦ Greedy [49] 49.2 21.9 48.8 13.4 58.2 18.5 512 • Gr…
Figure 10
Figure 10. Figure 10: Evaluating accuracy and detailedness on response of 60 Image-Text pairs in LLaVA-Bench [37] using GPT-4o [28] [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Case study on LLaVA-v1.5 through LLaVA-Bench. Hallucinations are marked in red [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Case study on mPLUG-owl2 through LLaVA-Bench. Hallucinations are marked in red [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Case study on InstructBLIP through LLaVA-Bench. Hallucinations are marked in red [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

88 extracted references · 49 canonical work pages

  1. [1]

    Agla: Mitigating object hallucinations in large vision- language models with assembly of global and local attention

    Wenbin An, Feng Tian, Sicong Leng, Jiahao Nie, Haonan Lin, QianYing Wang, Guang Dai, Ping Chen, and Shijian Lu. Agla: Mitigating object hallucinations in large vision- language models with assembly of global and local attention. arXiv preprint arXiv:2406.12718, 2024. 2, 3

    arXiv 2024

  2. [2]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 2

    arXiv 2023

  3. [3]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 1, 2, 4, 6, 7, 13, 14, 16, 17

    arXiv 2023

  4. [4]

    Audio chord recognition with recurrent neural networks

    Nicolas Boulanger-Lewandowski, Yoshua Bengio, and Pas- cal Vincent. Audio chord recognition with recurrent neural networks. In ISMIR, pages 335–340. Curitiba, 2013. 7, 8, 12, 13, 14, 17, 21, 22

    2013

  5. [5]

    Explaining a series of models by propagating shapley values

    Hugh Chen, Scott M Lundberg, and Su-In Lee. Explaining a series of models by propagating shapley values. Nature communications, 13(1):4512, 2022. 3

    2022

  6. [6]

    Harsanyinet: Computing accurate shapley values in a single forward propagation

    Lu Chen, Siyu Lou, Keyan Zhang, Jin Huang, and Quan- shi Zhang. Harsanyinet: Computing accurate shapley values in a single forward propagation. arXiv preprint arXiv:2304.01811, 2023. 3

    arXiv 2023

  7. [7]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330,

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330,

    arXiv

  8. [8]

    Mitigating hallucination in visual language models with visual supervi- sion

    Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jinqiao Wang, and Ming Tang. Mitigating hallucination in visual language models with visual supervi- sion. arXiv preprint arXiv:2311.16479, 2023. 1, 2

    arXiv 2023

Show all 88 references
  1. [9]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024. 2, 6, 8

    2024 arXiv

  2. [10]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer...

    2024

  3. [11]

    Halc: Object hallucination re- duction via adaptive focal-contrast decoding

    Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Halc: Object hallucination re- duction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.00425, 2024. 1, 2, 3

    2024 arXiv

  4. [12]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023) , 2(3):6,

    2023

  5. [13]

    Instructblip: Towards general- purpose vision-language models with instruction tuning,

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose vision-language models with instruction tuning,

  6. [14]

    Discovering and explaining the representation bottleneck of dnns

    Huiqi Deng, Qihan Ren, Hao Zhang, and Quanshi Zhang. Discovering and explaining the representation bottleneck of dnns. arXiv preprint arXiv:2111.06236, 2021. 3

    2021 arXiv

  7. [15]

    Explaining deepfake detection by analysing im- age matching

    Shichao Dong, Jin Wang, Jiajun Liang, Haoqiang Fan, and Renhe Ji. Explaining deepfake detection by analysing im- age matching. In European Conference on Computer Vision, pages 18–35. Springer, 2022. 3

    2022

  8. [16]

    Mitigating object hallucinations in large vision- language models through visual contrastive decoding

    Leng et al. Mitigating object hallucinations in large vision- language models through visual contrastive decoding. arXiv preprint arXiv:2311.16922, 2023. 1, 2, 3, 7, 8, 12, 13, 14, 15, 17, 21, 22

    2023 arXiv

  9. [17]

    Multi-modal hal- lucination control by visual information grounding

    Alessandro Favero, Luca Zancato, Matthew Trager, Sid- dharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hal- lucination control by visual information grounding. In Pro- ceedings of the IEEE/CVF Conference on Computer V...

    2024

  10. [18]

    Shapley val- ues for feature selection: The good, the bad, and the axioms

    Daniel Fryer, Inga Str ¨umke, and Hien Nguyen. Shapley val- ues for feature selection: The good, the bad, and the axioms. Ieee Access, 9:144352–144360, 2021. 3

    2021

  11. [19]

    Mme: A comprehensive evaluation benchmark for multimodal large language models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2023. 2, 4, 5, 6, 7, 8, 12, 13, 14, 17

    2023 arXiv

  12. [20]

    An axiomatic approach to the concept of interaction among players in cooperative games

    Michel Grabisch and Marc Roubens. An axiomatic approach to the concept of interaction among players in cooperative games. International Journal of game theory , 28(4):547– 565, 1999. 3

    1999

  13. [21]

    Sequence transduction with recurrent neural networks

    Alex Graves. Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711, 2012. 7, 8, 12, 13, 14, 17, 21, 22

    2012 arXiv

  14. [22]

    A simplified bar- gaining model for the n-person cooperative game

    John C Harsanyi and John C Harsanyi. A simplified bar- gaining model for the n-person cooperative game. Papers in game theory, pages 44–70, 1982. 2, 3, 6

    1982

  15. [23]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In Inter- national Conference on Learning Representations, 2019. 7, 8, 13, 14, 17, 21, 22

    2019

  16. [24]

    spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing

    Matthew Honnibal. spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. (No Title), 2017. 4

    2017

  17. [25]

    Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Con- ghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Confer...

    2024

  18. [26]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6700–6709, 2019. 12, 13

    2019

  19. [27]

    Self-introspective de- coding: Alleviating hallucinations for large vision-language models

    Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. Self-introspective de- coding: Alleviating hallucinations for large vision-language models. arXiv preprint arXiv:2408.02032, 2024. 3, 15

    2024 arXiv

  20. [28]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 7, 15, 16

    2024 arXiv

  21. [29]

    Vcoder: Ver- satile vision encoders for multimodal large language models

    Jitesh Jain, Jianwei Yang, and Humphrey Shi. Vcoder: Ver- satile vision encoders for multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27992–28002, 2024. 1

    2024

  22. [30]

    Hallucination augmented contrastive learn- ing for multimodal large language model

    Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learn- ing for multimodal large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  23. [31]

    mplug: Effective and efficient vision-language learning by cross-modal skip-connections

    Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, et al. mplug: Effective and efficient vision-language learning by cross-modal skip-connections. arXiv preprint arXiv:2205.12005, 2022. 1

    2022 arXiv

  24. [32]

    Contrastive decoding: Open-ended text gen- eration as optimization

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text gen- eration as optimization. arXiv preprint arXiv:2210.15097 ,

    arXiv

  25. [34]

    Evaluating object hallucina- tion in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucina- tion in large vision-language models. arXiv preprint arXiv:2305.10355, 2023. 7, 13, 16

    2023 arXiv

  26. [35]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  27. [36]

    Mitigating hallucination in large multi-modal models via robust instruction tuning

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Ya- coob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. arXiv preprint arXiv:2306.14565, 2023. 1

    2023 arXiv

  28. [37]

    Llava-bench in the wild dataset

    Haotian Liu. Llava-bench in the wild dataset. https: / / huggingface . co / datasets / liuhaotian / llava-bench-in-the-wild . Accessed: 2025-02-20. 2, 7, 13, 16

    2025

  29. [38]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023. 1, 2, 4, 6, 7, 8, 13, 14, 15, 16, 17, 21, 22

    2023

  30. [39]

    Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023. 2, 6, 7, 8, 12, 13, 14, 17, 21

    2023 arXiv

  31. [40]

    Convis: Contrastive decoding with hallucination visualiza- tion for mitigating hallucinations in multimodal large lan- guage models

    Yeji Park, Deokyeong Lee, Junsuk Choe, and Buru Chang. Convis: Contrastive decoding with hallucination visualiza- tion for mitigating hallucinations in multimodal large lan- guage models. arXiv preprint arXiv:2408.13906, 2024. 2, 3

    2024 arXiv

  32. [41]

    Aloha: A new measure for hallucination in captioning mod- els

    Suzanne Petryk, David M Chan, Anish Kachinthaya, Haodi Zou, John Canny, Joseph E Gonzalez, and Trevor Darrell. Aloha: A new measure for hallucination in captioning mod- els. arXiv preprint arXiv:2404.02904, 2024. 2

    2024 arXiv

  33. [42]

    Look, compare, decide: Alleviating hallucination in large vision- language models via multi-view multi-path reasoning

    Xiaoye Qu, Jiashuo Sun, Wei Wei, and Yu Cheng. Look, compare, decide: Alleviating hallucination in large vision- language models via multi-view multi-path reasoning. arXiv preprint arXiv:2408.17150, 2024. 2, 3

    2024 arXiv

  34. [43]

    A unified game-theoretic interpretation of adversarial robustness

    Jie Ren, Die Zhang, Yisen Wang, Lu Chen, Zhanpeng Zhou, Yiting Chen, Xu Cheng, Xin Wang, Meng Zhou, Jie Shi, et al. A unified game-theoretic interpretation of adversarial robustness. arXiv preprint arXiv:2111.03536, 4, 2021. 3

    2021 arXiv

  35. [44]

    Can we faithfully represent masked states to compute shap- ley values on a dnn?arXiv preprint arXiv:2105.10719, 2021

    Jie Ren, Zhanpeng Zhou, Qirui Chen, and Quanshi Zhang. Can we faithfully represent masked states to compute shap- ley values on a dnn?arXiv preprint arXiv:2105.10719, 2021. 2

    2021 arXiv

  36. [45]

    Defining and quantifying the emergence of sparse concepts in dnns

    Jie Ren, Mingjie Li, Qirui Chen, Huiqi Deng, and Quanshi Zhang. Defining and quantifying the emergence of sparse concepts in dnns. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 20280–20289, 2023. 2

    2023

  37. [46]

    Object hallucination in image cap- tioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image cap- tioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045,

    2018

  38. [47]

    A-okvqa: A benchmark for visual question answering using world knowl- edge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowl- edge. In European Conference on Computer Vision , pages 146–162. Springer, 2022. 7, 12, 13, 16

    2022

  39. [48]

    A value for n-person games

    Lloyd S Shapley. A value for n-person games. Contribution to the Theory of Games, 2, 1953. 2, 3

    1953

  40. [49]

    The good, the bad, and the greedy: Evaluation of llms should not ignore non-determinism

    Yifan Song, Guoyin Wang, Sujian Li, and Bill Yuchen Lin. The good, the bad, and the greedy: Evaluation of llms should not ignore non-determinism. arXiv preprint arXiv:2407.10457, 2024. 7, 13, 14, 16, 17, 21, 22

    2024 arXiv

  41. [50]

    Aligning large multi- modal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multi- modal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023. 1, 2

    2023 arXiv

  42. [51]

    The many shapley values for model explanation

    Mukund Sundararajan and Amir Najmi. The many shapley values for model explanation. InInternational conference on machine learning, pages 9269–9278. PMLR, 2020. 3

    2020

  43. [52]

    The shapley taylor interaction index

    Mukund Sundararajan, Kedar Dhamdhere, and Ashish Agar- wal. The shapley taylor interaction index. In International conference on machine learning, pages 9259–9268. PMLR,

  44. [53]

    Sequence to sequence learning with neural networks

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27, 2014. 7, 8, 12, 13, 14, 17, 21, 22

    2014

  45. [54]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 2

    2023 arXiv

  46. [55]

    An llm-free multi-dimensional benchmark for mllms hallu- cination evaluation

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An llm-free multi-dimensional benchmark for mllms hallu- cination evaluation. arXiv preprint arXiv:2311.07397, 2023. 1, 2

    2023 arXiv

  47. [56]

    Diagnosing the compositional knowledge of vision language models from a game-theoretic view

    Jin Wang, Shichao Dong, Yapeng Zhu, Kelu Yao, Weidong Zhao, Chao Li, and Ping Luo. Diagnosing the compositional knowledge of vision language models from a game-theoretic view. arXiv preprint arXiv:2405.17201, 2024. 2, 3

    2024 arXiv

  48. [57]

    A unified approach to interpret- ing and boosting adversarial transferability

    Xin Wang, Jie Ren, Shuyun Lin, Xiangming Zhu, Yisen Wang, and Quanshi Zhang. A unified approach to interpret- ing and boosting adversarial transferability. In International Conference on Learning Representations, 2020. 3, 15

    2020

  49. [58]

    Interpreting attributions and interactions of adversar- ial attacks

    Xin Wang, Shuyun Lin, Hao Zhang, Yufei Zhu, and Quanshi Zhang. Interpreting attributions and interactions of adversar- ial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1095–1104, 2021. 3

    2021

  50. [59]

    Ritual: Random image transformations as a universal anti-hallucination lever in lvlms

    Sangmin Woo, Jaehyuk Jang, Donguk Kim, Yubin Choi, and Changick Kim. Ritual: Random image transformations as a universal anti-hallucination lever in lvlms. arXiv preprint arXiv:2405.17821, 2024. 15

    2024 arXiv

  51. [60]

    Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302, 2024. 15, 16

    2024 arXiv

  52. [61]

    Towards understanding the generalization of deepfake detectors from a game-theoretical view

    Kelu Yao, Jin Wang, Boyu Diao, and Chao Li. Towards understanding the generalization of deepfake detectors from a game-theoretical view. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2031– 2041, 2023. 3

    2023

  53. [62]

    mplug-owl: Modularization empowers large language models with mul- timodality

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Jun- feng Tian, Qian Qi, Ji Zhang, and Fei Huang. mplug-owl: Modularization empowers large language models with mul- tim...

    2023 arXiv

  54. [63]

    mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , ...

    2024

  55. [64]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85. Springer, 2016. 16

    2016

  56. [65]

    Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data

    Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wen- tao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yueting Zhuang. Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  57. [66]

    Less is more: Mitigat- ing multimodal hallucination from an eos decision perspec- tive

    Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigat- ing multimodal hallucination from an eos decision perspec- tive. arXiv preprint arXiv:2402.14545, 2024

    2024 arXiv

  58. [67]

    Halle-switch: Controlling ob- ject hallucination in large vision language models

    Bohan Zhai, Shijia Yang, Chenfeng Xu, Sheng Shen, Kurt Keutzer, and Manling Li. Halle-switch: Controlling ob- ject hallucination in large vision language models. arXiv e-prints, pages arXiv–2310, 2023. 1

    2023

  59. [68]

    Building interpretable interaction trees for deep nlp models

    Die Zhang, Hao Zhang, Huilin Zhou, Xiaoyi Bao, Da Huo, Ruizhao Chen, Xu Cheng, Mengyue Wu, and Quanshi Zhang. Building interpretable interaction trees for deep nlp models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 14328–14337, 2021. 3

    2021

  60. [69]

    Game-theoretic interactions of different orders

    Hao Zhang, Xu Cheng, Yiting Chen, and Quanshi Zhang. Game-theoretic interactions of different orders. arXiv preprint arXiv:2010.14978, 2020. 3

    2010 arXiv

  61. [70]

    Interpreting multivariate shapley interac- tions in dnns

    Hao Zhang, Yichen Xie, Longjie Zheng, Die Zhang, and Quanshi Zhang. Interpreting multivariate shapley interac- tions in dnns. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10877–10886, 2021. 3, 15

    2021

  62. [71]

    Explaining gen- eralization power of a dnn using interactive concepts

    Huilin Zhou, Hao Zhang, Huiqi Deng, Dongrui Liu, Wen Shen, Shih-Han Chan, and Quanshi Zhang. Explaining gen- eralization power of a dnn using interactive concepts. In AAAI Conference on Artificial Intelligence, 2023. 3

    2023

  63. [72]

    Look twice before you answer: Memory-space visual retracing for hallucination mitigation in multimodal large language mod- els

    Xin Zou, Yizhou Wang, Yibo Yan, Sirui Huang, Kening Zheng, Junkai Chen, Chang Tang, and Xuming Hu. Look twice before you answer: Memory-space visual retracing for hallucination mitigation in multimodal large language mod- els. arXiv preprint arXiv:2410.03577, 2024. 2, 3

    arXiv

  64. [76]

    The Polling-based Object Probing Evaluation (POPE) [33] utilizes images sampled from several datasets, including MSCOCO [35], A-OKVQA [47], and GQA [26]

    Details of the Benchmarks POPE. The Polling-based Object Probing Evaluation (POPE) [33] utilizes images sampled from several datasets, including MSCOCO [35], A-OKVQA [47], and GQA [26]. Every question in POPE is ”Is there a <object> in the image?”. For each dataset, it incorpo...

  65. [77]

    As shown in Tab

    Result on InternVL2.5-MPO In order to further demonstrate the effectiveness of IN- TER, we conducted a comparison on the current state-of- the-art LVLM InternVL2.5-MPO (8B) [10]. As shown in Tab. 6, the performance of INTER is superior to the base- line methods across various ...

  66. [78]

    Ablation Study on Interaction Guide Loca- tor. In addition to the effectiveness analysis of the Interaction Guide Locator based on Beam Search [4, 21, 53], we also conducted ablation experiments on various decoding strate- gies for IGL. As shown in Tab. 7, we evaluated the per...

  67. [79]

    Through experiments on CHAIR [46] and MME [19] benchmarks, we analyze how the interaction guidance co- efficient k affects the performance of INTER

    Parameter Analysis of Interaction Guide Locator. Through experiments on CHAIR [46] and MME [19] benchmarks, we analyze how the interaction guidance co- efficient k affects the performance of INTER. As shown in Fig. 7, varyingk values lead to significantly different behaviors i...

  68. [80]

    The results, as shown in the Tab

    Result on POPE In this subsection, we evaluate the performance of the pro- posed INTER on the GQA [26] dataset within the POPE benchmark. The results, as shown in the Tab. 8, indicate that significant performance improvements across four models. Furthermore, these enhancements...

  69. [81]

    As shown in Tab

    Result on MME In addition to demonstrating the performance improve- ments brought by INTER across various decoding strategies in 14 subtasks, we also conducted comparisons in terms of the total score and perception total score of MME [19]. As shown in Tab. 9, after correction ...

  70. [82]

    10, 19 and 20

    Result on MM-Bench To illustrate the improvement of INTER on MM-Bench in more detail, we present the performance of each subtask in Tabs. 10, 19 and 20. As we can see, using INTER results in an improvement across various metrics. In addition, to val- idate the performance of I...

  71. [83]

    12, 23 and 24

    Result on MMStar Likewise, to assess the effectiveness of INTER on MMStar, we also present the performance of each subtask on LLaV A- v1.5 (7B) [38] in Tabs. 12, 23 and 24. The results indicate that our approach achieves good performance across most model method CHAIR (512) MM...

  72. [84]

    8 to 13, 15, 16 and 19 to 26, we demonstrated the effectiveness of INTER in correcting the Greedy Search across various benchmarks

    Result on Greedy Search In Tabs. 8 to 13, 15, 16 and 19 to 26, we demonstrated the effectiveness of INTER in correcting the Greedy Search across various benchmarks. It is evident that there is a sig- nificant improvement across different benchmarks, indicat- ing that our metho...

  73. [85]

    Result on LLaV A-Bench To more intuitively demonstrate the performance of INTER, detailed case studies were conducted using LLaV A-Bench. In Figs. 11 to 13, examples of the captioning and complex reasoning task for each model are presented. The hallucina- tion parts are highli...

  74. [86]

    Computation Efficiency Similar to VCD [16], which require additional forward passes, INTER also necessitates extra inference to compute the logits under different subsets of A. While INTER in- creases the total number of forward passes, the actual run- time overhead remains ne...

  75. [87]

    We conducted experiments with M3ID [17], Ritual [59] and SID [27] in Tab

    Comparison with Other Methods. We conducted experiments with M3ID [17], Ritual [59] and SID [27] in Tab. 14. The results demonstrate that our IN- TER achieves comparable performance among compared methods

  76. [88]

    We conducted experiments with DeepSeek-VL2 [60] on the visual grounding task

    Performance on Other Types of Tasks or Different LVLMs. We conducted experiments with DeepSeek-VL2 [60] on the visual grounding task. As shown in Tab. 17, results show that the INTER boosts the model performance on this task

  77. [89]

    The value range of I(A)yt could be influenced by several factors, e.g., benchmarks, LVLMs, etc

    The Range of the Harsanyi dividend. The value range of I(A)yt could be influenced by several factors, e.g., benchmarks, LVLMs, etc. These complexi- ties make it challenging to establish a theoretical bound for its value range. Nevertheless, we conducted experiments to empirica...

  78. [2018]

    2, 4, 7, 8, 12, 13, 16, 17

  79. [2023]

    1, 2, 4, 6, 7, 8, 13, 14, 15, 16, 17

  80. [2024]

    2, 6, 7, 8, 12, 13, 14, 21, 22

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.