Pith. sign in

REVIEW 3 major objections 7 minor 74 references

Test-Time Hallucination Control in Large Vision-Language Models

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a zero-shot image-text validator at decoding time suppresses object hallucinations in vision-language models without retraining, weight editing, or extra decoding passes.

desk verdict A useful, well-tested decoding-time hallucination reducer, but the novelty claim collapses without a head-to-head against Seeing is Believing, which the paper mischaracterizes and omits. read the letter →

arxiv 2608.11474 v1 pith:BMFMWG4W submitted 2026-08-11 cs.CV

Mehran Tamjidi , Hamidreza Dastmalchi , Ali Cheraghian , Mohammadreza Alimoradijazi , Aijun An , Hossein Rahmani This is my paper · ORCID
classification cs.CV
keywords ObjectHallucinationLargeVision-LanguageModelsMitigationTraining-FreeDecodingCLIPEntropy-BasedFusionTokenValidationMulti-ModalAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Object hallucinations—models confidently naming objects that are not in the image—make large vision-language models unreliable in practice. This paper proposes a training-free fix called TTH: at each decoding step it takes the top candidate object tokens, checks each one against the image with a zero-shot image-text classifier using prompts like 'a photo of dog,' and mixes those verification scores into the model's own next-token scores. The mixing weight is set by the entropy of the model's prediction, so uncertain object choices are corrected more strongly while confident ones are left alone. The paper reports that TTH lowers hallucination rates on CHAIR, improves accuracy and F1 on OPOPE, and raises accuracy ratings on LLaVA-Bench across the LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2 models, with less computational overhead than contrastive decoding methods.

What carries the argument

The load-bearing mechanism is the token-validator module: a zero-shot image-text classifier (CLIP) that scores an object token $w_i$ by the cosine similarity between the image embedding and the text embedding of 'a photo of $w_i$.' WordNet supplies the object filter that decides which top-$k$ candidate tokens are objects, Eq. (7) rescales the CLIP scores into the range of the model's logits, and Eq. (9) fuses the two with weight $H_t$, the normalized entropy of the model's distribution over the object candidates. The entropy term makes the intervention adaptive: at high entropy the validator's ground-truth signal takes over, while at low entropy the model's own prediction stands.

What would settle it

Construct an image that contains object A but not object B, with both among the model's near-tied top-k object candidates and with the image-text classifier scoring 'a photo of B' higher than 'a photo of A' (as can happen for commonly co-occurring objects). If TTH then raises B above A in the generated output, the method has replaced a correct token with a hallucination; counting how often this flip occurs across a set of such images, for example in the negative-object setting of OPOPE, would settle whether the central claim holds.

Watch

Extended reading notes

Core claim

The central claim is that object hallucination can be reduced at test time by treating each decoding step as a small verification problem: identify the object words the model is weighing, ask an external zero-shot image-text classifier whether each word matches the image, and replace the candidate logits with an entropy-weighted mixture of the model's own logits and the validator's scores. The correction is surgical because only object tokens in the candidate set are touched, and it preserves fluency because the language model's own prediction continues to dominate whenever its confidence is high. The experiments are designed to show that the improvement holds across three architecture families and three benchmarks, with the main comparisons against Nullu, OPERA, and HALC.

Load-bearing premise

The method depends on the external image-text classifier's score for 'a photo of {word}' being a trustworthy measure of whether the object is present in the image, even after the scores are rescaled to fit the model's much narrower logit range; if that score is wrong or biased for a given image or object, the entropy weighting can steer the decoder toward the wrong word.

Editorial extensions

If this is right

  • Any off-the-shelf vision-language model can receive a hallucination reduction by adding a frozen image-text classifier and a lexical lookup, with no fine-tuning and no access to internal weights.
  • Because TTH needs only a single generation pass, it can be deployed in latency-sensitive settings where contrastive or multi-generation decoders are too slow.
  • Only uncertain object tokens are reweighted, so the model's language prior is preserved; the reported BLEU scores stay near or above the unmodified greedy baseline.
  • The entropy-based weighting is itself the defense mechanism: fixed fusion weights are shown to be sensitive to their chosen coefficient, while the adaptive rule improves CHAIR scores on both sentence and instance level.
  • The reported gains across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2 indicate the mechanism transfers across different vision-language architectures and connector designs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to swap the validator for a stronger or ensemble image-text model: the method's ceiling is set by the validator's zero-shot reliability, so gains should track validator accuracy.
  • The same entropy-gated fusion could be applied to other token classes, such as attributes or actions, by replacing the WordNet object filter with a part-of-speech filter; an experiment analogous to CHAIR with color or verb hallucinations would test this.
  • Because CLIP similarity is trained on broad internet image-text pairs, it may favor frequent objects; stratifying CHAIR results by object frequency would reveal whether the validator's bias becomes the method's bias.
  • TTH does not modify weights, so it could be composed with training-based debiasing; an additive-gain experiment would show whether the two correction mechanisms overlap or stack.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes Test-Time Hallucination Mitigation (TTH), a training-free decoding-time method for reducing object hallucinations in large vision-language models. At each decoding step, TTH takes the top-k tokens from the LVLM's logits, filters them to object candidates using WordNet, verifies each candidate by computing CLIP similarity between the image and the prompt 'a photo of {w}', rescales these similarities into the LVLM logit range, and fuses the two logit sets with an entropy-based weighting that upweights CLIP when the LVLM is uncertain. The method is evaluated on CHAIR, OPOPE, and LLaVA-Bench across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, reporting consistent improvements over Nullu, OPERA, HALC, VCD, and greedy/beam baselines, with a reported modest throughput cost. The paper also provides ablations over the number of candidates and the fusion weighting scheme.

Significance. If the results hold, TTH is a simple, practical plug-in that reduces object hallucinations without retraining or internal-state modification, and the authors release code and report three-seed averages with fixed random subsets, which aids reproducibility. The consistency of the gains across three LVLMs and three benchmarks, including the OPOPE accuracy improvements over strong baselines, is a useful contribution, and the efficiency analysis in Figure 4 quantifies the practical cost. The principal caveat is that the closest prior work, SiB (CLIP-guided decoding), is not benchmarked, so the claimed novelty and advantage over existing single-pass CLIP-guided methods remain unverified.

major comments (3)
  1. [§2, Related Work; §4.1, Baselines; Tables 1–2] The manuscript does not include a comparison with SiB (ref. [13]), the closest prior work in CLIP-guided decoding, and the related-work description of [13] as relying on 'multiple generations from the original prompt' appears inconsistent with SiB's stated single-pass CLIP-guided decoding design. Since the paper's central novelty claim is that TTH is the first training-free single-pass method that avoids both multiple decoding rounds and internal-state editing, the absence of SiB from Tables 1 and 2 leaves that claim unsubstantiated. The authors should add SiB as a baseline, report its results on CHAIR and OPOPE for the same LVLMs, and either demonstrate a clear margin or explicitly characterize the technical differences (e.g., candidate selection, rescaling, fusion) that justify the claimed advance.
  2. [§3.2, Eq. (7) and Eq. (9)] The min-max rescaling of CLIP similarities into the range of the LVLM candidate logits (Eq. 7) can become ill-conditioned when the candidate logit range is very narrow, amplifying small differences in CLIP scores; conversely, the entropy-weighted fusion (Eq. 9) can allow noisy CLIP scores to override a confident LVLM prediction. The paper provides no analysis of this sensitivity, such as the distribution of candidate logit ranges, the calibration of CLIP similarity scores across images, or failure cases where TTH changes a correct object into an incorrect one. A concrete test would be to compare the proposed rescaling against alternatives (e.g., softmax temperature matching or score standardization) and to report the frequency and severity of such flips.
  3. [§4.3, Figure 5; §4.2, Tables 1–2] The ablation over the number of candidates shows a sharp optimum at k=10, and all main results use this value, but the paper does not state how k was selected or whether it was chosen on the same MSCOCO subset used for evaluation. If k was tuned on the evaluation set, the reported improvements may reflect some degree of overfitting. The authors should clarify the selection procedure, report whether k was chosen on a validation split, and provide results for neighboring values of k (e.g., k=5, 15, 20) in the main tables to demonstrate robustness.
minor comments (7)
  1. [Title; Abstract; §6] The method name is inconsistent across the paper: the title uses 'Test-Time Hallucination Control', the abstract says 'Test-Time Hallucination Mitigation (TTH)', and Section 6 calls it 'Test-Time Hallucination (TTH)'; the authors should standardize the name.
  2. [§3.1, Eq. (1)] The image and the query are both denoted by the symbol I in Equation (1) and the surrounding text, which is confusing; please use distinct symbols, for example I_img for the image and q for the prompt.
  3. [§1, Introduction, page 2] The sentence 'The first category of hallucination mitigation approaches relies on extensive fine-tuning [6,13,30,44,51,74]' misplaces several decoding-based methods (HALC [6], SiB [13], and VCD [30]) in the fine-tuning category; these are inference-time or decoding methods and should be cited in the appropriate category.
  4. [§4.2, GPT-4V Aided Evaluation on LLaVA-Bench; Table 3] The LLaVA-Bench evaluation does not report standard deviations, the number of independent GPT-4V evaluations, or the exact prompt used for scoring; please provide these details to allow reproducibility and to assess the stability of the reported improvements.
  5. [Figure 4] The annotations '3% 6xFaster' and '5% 1.5xFaster' in Figure 4 are ambiguous; please clarify what the percentages refer to and how the speedup factors are computed.
  6. [References] References [67] and [68] appear to be duplicates of the same preprint; please merge or remove the redundant entry.
  7. [§5, Limitations] The limitations section does not mention that only object candidates that appear in the LVLM's top-k set can be validated; if the correct object is outside the top-k, TTH cannot introduce it. This is an inherent design limitation and should be stated explicitly.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: TTH is an externally benchmarked decoding-time fusion; the only minor self-citation is provenance, not a load-bearing circular reduction.

full rationale

The derivation chain is not circular. TTH constructs a candidate pool from top-k LVLM logits (Eq. 6), filters WordNet objects, computes zero-shot CLIP similarity scores for 'a photo of {w_i}', rescales them (Eq. 7), and fuses them with LVLM logits via entropy weighting (Eq. 9). None of these quantities is defined in terms of the CHAIR, OPOPE, or LLaVA-Bench targets; no fitted parameter is relabeled as a prediction. The entropy-based fusion is attributed to the authors' own prior work [49], and the candidate count k=10 is chosen from an ablation on the evaluation benchmark, but the in-paper ablation independently demonstrates the benefit of the adaptive weighting, and the central empirical claim is tested against external ground-truth benchmarks rather than derived from the citation. The absence of a head-to-head comparison with SiB [13] is a scientific completeness and novelty concern, not a circularity concern. The stated limitation to object-level hallucination similarly narrows the scope without making the evaluation circular. Hence a low score of 2 reflects the minor self-citation and benchmark-tuned hyperparameter, with no circular step exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

All components are built from existing pretrained models (LVLM and CLIP) plus a lexical resource. No new physical entities are introduced. The central claim depends on assumptions about CLIP's reliability, WordNet's coverage, and the entropy-hallucination link.

free parameters (1)
  • k (top-k candidate pool size) = 10
    Set to 10 based on the CHAIR ablation in Figure 5; performance degrades for k=3 and k=50, so the result depends on this tuned value.
assumptions (4)
  • domain assumption WordNet tagging correctly identifies object tokens in the LVLM's tokenizer output.
    Used to form the object candidate set O_t; if object tokens are missed or non-object tokens are tagged as objects, the fusion either does not trigger or corrupts non-object predictions (Section 3.2).
  • domain assumption CLIP zero-shot similarity scores on the phrase 'a photo of {w_i}' measure visual presence of the object.
    Core of the token validator; if CLIP is unreliable for an image or object class, the fused logits can be misled (Eq. 7-9).
  • domain assumption High entropy over object candidates correlates with higher hallucination risk.
    The entropy-based weighting in Eq. 9 assumes this correlation, citing prior work [25]; if the correlation is weak, the adaptive weighting provides no benefit over fixed weighting.
  • domain assumption Min-max rescaling of CLIP scores to the LVLM logit range preserves ranking and relative confidence.
    Eq. 7 maps CLIP scores via the min/max of the LVLM object logits; when that range is tiny, CLIP score differences can be amplified into large logit differences, so the assumption is load-bearing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Test-Time Hallucination Control in Large Vision-Language Models." pith.science (2026). https://pith.science/paper/BMFMWG4W

@misc{pith2026260811474,
  author       = {Pith},
  title        = {Pith review of: Test-Time Hallucination Control in Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BMFMWG4W}},
  note         = {Machine review of arXiv:2608.11474}
}
read the original abstract

Object Hallucination in large vision-language models (LVLMs), where models generate non-factual content about input images, remains a critical barrier to their reliability in real-world applications. Existing mitigation strategies can be categorized into training-based and training-free methods. Training-based methods often achieve strong performance but are costly, requiring extensive computational resources, large-scale data, and time-consuming fine-tuning. Training-free approaches are particularly appealing due to their efficiency. However, existing training-free methods either require multiple decoding rounds, which adds computational overhead, or modify internal states in a model-specific way that risks degrading pretrained knowledge. We propose Test-Time Hallucination Mitigation (TTH) method, a novel training-free method that addresses both limitations. TTH introduces a token-validator module, implemented as a zero-shot Multi-Modal Classifier (MMC), to generate auxiliary logits grounded in the input image. These logits are fused with the original LVLM outputs at the token level for object tokens selected from a candidate pool. An entropy-based weighting scheme is then applied to enable robust and accurate predictions. Extensive experiments across multiple LVLM families and diverse benchmarks demonstrate that TTH consistently improves accuracy and robustness, underscoring its generalizability and practical effectiveness. Code is released at https://github.com/Mehran-TAM/TTH

Figures

Figures reproduced from arXiv: 2608.11474 by the authors.

Figure 1
Figure 1. (a) A major category of training-free hallucination mitigation strategies relies on multiple generations during decoding, which is computationally inefficient. (b) An￾other category modifies internal layers (editing) or hidden states (steering) before or during inference, which is often complex and may disrupt the model’s knowledge and reasoning. (c) In contrast, our proposed method, TTH, overcomes both limitations … view at source ↗
Figure 2
Figure 2. Overview of the proposed Test-Time Hallucination mitigation (TTH) frame￾work. The LVLM first generates token logits conditioned on the image and prompt. The Token Validator Module extracts top-k candidates, filters object tokens using Word￾Net, and verifies them with CLIP by matching object-specific prompts (e.g., “a photo of jacket”) against the image. Verified scores are rescaled and fused with the LVLM logits usi… view at source ↗
Figure 1
Figure 1. Logit Generation and Candidate Selection. At each decoding step t, the LVLM produces a logit vector zt = [zt,w1 , ..., zt,w|V| ] ∈ R |V| over the vocabulary V using its LM head. These logits correspond to the probability distribution defined in Equation 3. From zt, we extract the top-k candidate tokens with the highest probabilities: R_t = \{w_1, w_2, \dots , w_k\} = \text {TopK}(\textbf {z}_t, k). (6) Object Candid… view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Illustration on LLaVA-Bench: TTH curbs hallucinations and improves ground￾ing quality. Model Method Accuracy↑ Detailedness↑ LLaVA-1.5 Original 6.46 6.25 TTH 6.71 6.46 MiniGPT-4 Original 5.70 6.04 TTH 5.96 6.30 mPLUG-Owl2 Original 5.62 5.54 TTH 6.12 6.08 [PITH_FULL_IMA…
Figure 4
Figure 4. Figure 4: Throughput (\protect \text {items}/s ) vs. CHAIR_S comparison over decoding-based baseline methods. Greedy search is used for TTH in the throughput evaluation. Higher through￾put indicates better efficiency, while lower CHAIR_S reflects reduced hallucination. 3 5 8 10 …
Figure 5
Figure 5. Figure 5: Ablation study on the number of object candidates using LLaVA-1.5. Left: CHAIR_S; Right: CHAIR_I . Inference Efficiency. To evaluate the efficiency of TTH, we measure infer￾ence throughput (\protect \text {items}/s ) alongside the CHAIR_S metric on LLaVA-1.5-7B for all…
Figure 6
Figure 6. Figure 6: Ablation study on the effectiveness of Adaptive Entropy-Based Fusion in LLaVA-1.5. Left: CHAIR_S; Right: CHAIR_I . CHAIR_I and CHAIR_S metrics. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 36 canonical work pages

  1. [13]

    arXiv preprint arXiv:2402.15300 (2024)

    Deng, A., Chen, Z., Hooi, B.: Seeing is believing: Mitigating hallucination in large vision-language models via clip-guided decoding. arXiv preprint arXiv:2402.15300 (2024)

    arXiv 2024

  2. [1]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    An, W., Tian, F., Leng, S., Nie, J., Lin, H., Wang, Q., Chen, P., Zhang, X., Lu, S.: Mitigating object hallucinations in large vision-language models with assembly of global and local attention. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 29915–29926 (2025)

    2025

  3. [2]

    arXiv preprint arXiv:2309.16609 (2023)

    Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al.: Qwen technical report. arXiv preprint arXiv:2309.16609 (2023)

    arXiv 2023

  4. [3]

    arXiv preprint arXiv:2308.12966 (2023)

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J.: Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966 (2023)

    arXiv 2023

  5. [4]

    arXiv preprint arXiv:2306.15195 (2023)

    Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., Zhao, R.: Shikra: Unleash- ing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195 (2023)

    arXiv 2023

  6. [5]

    arXiv preprint arXiv:2602.01047 (2026)

    Chen, X., Chu, X., Qiu, Y., Zhang, H., Xiong, J., Tang, S., Liu, S., Yang, S., Yang, C., So, H.K.H., et al.: Residual decoding: Mitigating hallucinations in large vision-language models via history-aware residual guidance. arXiv preprint arXiv:2602.01047 (2026)

    work page arXiv 2026

  7. [6]

    Chen, Z., Zhao, Z., Luo, H., Yao, H., Li, B., Zhou, J.: Halc: Object hallucination reduction via adaptive focal-contrast decoding (2024)

    work page 2024

  8. [7]

    Chiang, W.L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J.E., Stoica, I., Xing, E.P.: Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality (March 2023),https://lmsys.org/ blog/2023-03-30-vicuna/

    2023

Show all 74 references
  1. [8]

    In: ICLR (2023)

    Chuang, Y.S., Xie, Y., Luo, H., Kim, Y., Glass, J.R., He, P.: Dola: Decoding by contrasting layers improves factuality in large language models. In: ICLR (2023)

    2023

  2. [9]

    In: Proceedings of the IEEE/CVF winter conference on applications of computer vision

    Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.D., et al.: A survey on multimodal large language models for autonomous driving. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 958–979 (2024)

    2024

  3. [10]

    arXiv preprint arXiv:2306.04387 (2023)

    Dai, W., Li, J., Li, D., Tiong, A.M.H., Zhao, J., Wang, W., Li, B., Fung, P., Hoi, S.: Instructblip: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2306.04387 (2023)

    2023 arXiv

  4. [11]

    arXiv preprint arXiv:2603.16284 (2026)

    Dang, T., Bi, C., Shen, S., Liu, J., Huang, Q., Wang, S.: Locate-then-sparsify: At- tribution guided sparse strategy for visual hallucination mitigation. arXiv preprint arXiv:2603.16284 (2026)

    2026 arXiv

  5. [12]

    arXiv preprint arXiv:2603.10470 (2026)

    Dastmalchi, H., An, A., Cheraghian, A., Barzamini, H.: Fighting hallucinations with counterfactuals: Diffusion-guided perturbations for lvlm hallucination sup- pression. arXiv preprint arXiv:2603.10470 (2026)

    2026

  6. [14]

    arXiv preprint arXiv:2603.10360 (2026)

    Fa, Z., Duan, Y., Zhang, J., Qi, L., Shi, Y.: One token, two fates: A unified frame- work via vision token manipulation against mllms hallucination. arXiv preprint arXiv:2603.10360 (2026)

    2026

  7. [15]

    In: CVPR

    Favero, A., Zancato, L., Trager, M., Choudhary, S., Perera, P., Achille, A., Swami- nathan, A., Soatto, S.: Multi-modal hallucination control by visual information grounding. In: CVPR. pp. 14303–14312 (2024)

    2024

  8. [16]

    arXiv preprint arXiv:1702.01806 (2017) 16 M

    Freitag, M., Al-Onaizan, Y.: Beam search strategies for neural machine translation. arXiv preprint arXiv:1702.01806 (2017) 16 M. Tamjidi et al

    2017 arXiv

  9. [17]

    arXiv preprint arXiv:2604.00455 (2026)

    Ha, J., Baek, J., So, J.: First logit boosting: Visual grounding method to mitigate object hallucination in large vision-language models. arXiv preprint arXiv:2604.00455 (2026)

    2026

  10. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition

    He, Y., Yang, G., Ge, R., Chen, Y., Coatrieux, J.L., Wang, B., Li, S.: Geomet- ric visual similarity learning in 3d medical image self-supervised pre-training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 9538–9547 (2023)

    2023

  11. [19]

    arXiv preprint arXiv:2511.11502 (2025)

    Hoang-Xuan, N., Vu, M., Thai, M.T., Bhattarai, M.: Pas: Prelim attention score for detecting object hallucinations in large vision–language models. arXiv preprint arXiv:2511.11502 (2025)

    2025

  12. [20]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Hou, X., Li, W., Li, Y., Shu, H., Wang, Y., Chen, X., Wang, S.: Ves-rft: Reward- ing visual evidence sensitivity to mitigate hallucinations in large vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4168–4177 (2026)

    2026

  13. [21]

    In: CVPR

    Huang, Q., Dong, X., Zhang, P., Wang, B., He, C., Wang, J., Lin, D., Zhang, W., Yu, N.: Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In: CVPR. pp. 13418–13427 (2024)

    2024

  14. [22]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion

    Ji, J., Liu, Q., Yang, W., He, Z.: Causallens: Sensitivity-guided multi-head causal intervention for hallucination mitigation in large vision-language models. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. pp. 4199–4209 (2026)

    2026

  15. [23]

    Jiang, B., Chen, S., Liao, B., Zhang, X., Yin, W., Zhang, Q., Huang, C., Liu, W., Wang, X.: Senna: Bridging large vision-language models and end-to-end au- tonomous driving (2024)

    2024

  16. [24]

    In: CVPR

    Jiang, C., Xu, H., Dong, M., Chen, J., Ye, W., Yan, M., Ye, Q., Zhang, J., Huang, F., Zhang, S.: Hallucination augmented contrastive learning for multimodal large language model. In: CVPR. pp. 27036–27046 (2024)

    2024

  17. [25]

    arXiv preprint arXiv:2410.02762 (2024)

    Jiang, N., Kachinthaya, A., Petryk, S., Gandelsman, Y.: Interpreting and edit- ing vision-language representations to mitigate hallucinations. arXiv preprint arXiv:2410.02762 (2024)

    2024 arXiv

  18. [26]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Jiang, Z., Chen, J., Zhu, B., Luo, T., Shen, Y., Yang, X.: Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallu- cinations via attention lens. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp....

    2025

  19. [27]

    IEEE Trans- actions on Medical Imaging (2024)

    Kong, Y., Zhang, X., Wang, W., Zhou, Y., Li, Y., Yuan, Y.: Multi-scale spatial- temporal attention networks for functional connectome classification. IEEE Trans- actions on Medical Imaging (2024)

    2024

  20. [28]

    Lee, J., Cha, S., Lee, Y., Yang, C.: Visual question answering instruction: Unlock- ing multimodal large language model to domain-specific visual multitasks (2024), https://arxiv.org/abs/2402.08360

    2024 arXiv

  21. [29]

    arXiv preprint arXiv:2606.29847 (2026)

    Lei, Y., Lyu, W., Du, Y., Zhen, X., Snoek, C.G., Shao, L.: See only when needed: Context-aware attention intervention for mitigating hallucinations in lvlms. arXiv preprint arXiv:2606.29847 (2026)

    2026 arXiv

  22. [30]

    In: CVPR

    Leng, S., Zhang, H., Chen, G., Li, X., Lu, S., Miao, C., Bing, L.: Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In: CVPR. pp. 13872–13882 (2024)

    2024

  23. [31]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, J., Zhang, J., Jie, Z., Ma, L., Li, M., Luo, X., Li, G.: Cross-modal attention calibration for lvlm hallucination mitigation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 40186–40196 (2026) TTH 17

    2026

  24. [32]

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. pp. 19730–19742 (2023)

    2023

  25. [33]

    In: Computer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Computer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. pp. 740–755...

    2014

  26. [34]

    arXiv preprint arXiv:2602.22727 (2026)

    Lin, Y., Fang, Q., Li, Y., Sun, J., Gao, J., Sang, J.: Hulluedit: Single-pass evidence- consistent subspace editing for mitigating hallucinations in large vision-language models. arXiv preprint arXiv:2602.22727 (2026)

    2026

  27. [35]

    arXiv preprint arXiv:2306.14565 (2023)

    Liu, F., Lin, K., Li, L., Wang, J., Yacoob, Y., Wang, L.: Mitigating hallucina- tion in large multi-modal models via robust instruction tuning. arXiv preprint arXiv:2306.14565 (2023)

    2023 arXiv

  28. [36]

    arXiv preprint arXiv:2402.00253 (2024)

    Liu, H., Xue, W., Chen, Y., Chen, D., Zhao, X., Wang, K., Hou, L., Li, R., Peng, W.: A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253 (2024)

    2024 arXiv

  29. [37]

    arXiv preprint arXiv:2310.03744 (2023)

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744 (2023)

    2023 arXiv

  30. [38]

    NeurIPS36(2024)

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. NeurIPS36(2024)

    2024

  31. [39]

    In: ECCV (2024)

    Liu, S., Zheng, K., Chen, W.: Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In: ECCV (2024)

    2024

  32. [40]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Lu, F., Wu, W., Zheng, K., Ma, S., Gong, B., Liu, J., Zhai, W., Cao, Y., Shen, Y., Zha, Z.J.: Benchmarking large vision-language models via directed scene graph for comprehensive image captioning. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 1...

    2025

  33. [41]

    Advances in Neural Information Processing Systems37, 122811–122832 (2024)

    Lyu, X., Chen, B., Gao, L., Shen, H., Song, J.: Alleviating hallucinations in large vision-language models through hallucination-induced optimization. Advances in Neural Information Processing Systems37, 122811–122832 (2024)

    2024

  34. [42]

    Communications of the ACM 38(11), 39–41 (1995)

    Miller, G.A.: Wordnet: a lexical database for english. Communications of the ACM 38(11), 39–41 (1995)

    1995

  35. [43]

    In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition

    Min, J., Kim, C., Choi, H., Kim, Y., Kim, K., Jo, Y., Lee, J.: Fine-grained multi image object hallucination benchmark. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition. pp. 18295–18305 (2026)

    2026

  36. [44]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Park, Y., Lee, D., Choe, J., Chang, B.: Convis: Contrastive decoding with hallu- cination visualization for mitigating hallucinations in multimodal large language models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 6434–6442 (2025)

    2025

  37. [45]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)

    2021

  38. [46]

    arXiv preprint arXiv:1809.02156 (2018)

    Rohrbach, A., Hendricks, L.A., Burns, K., Darrell, T., Saenko, K.: Object halluci- nation in image captioning. arXiv preprint arXiv:1809.02156 (2018)

    2018 arXiv

  39. [47]

    In: Proceedings of the IEEE/CVF winter conference on applications of computer vision

    Rotstein, N., Bensaid, D., Brody, S., Ganz, R., Kimmel, R.: Fusecap: Leveraging large language models for enriched fused image captions. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 5689–5700 (2024)

    2024

  40. [48]

    In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition

    Shao, Z., Yu, Z., Wang, M., Yu, J.: Prompting large language models with answer heuristics for knowledge-based visual question answering. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition. pp. 14974– 14983 (2023) 18 M. Tamjidi et al

    2023

  41. [49]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Tamjidi,M.,Dastmalchi,H.,Alimoradijazi,M.,Cheraghian,A.,An,A.,Saberi,M.: Adapt-as-you-walk through the clouds: training-free online test-time adaptation of 3d vision-language foundation models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 40, pp. 929...

    2026

  42. [50]

    arXiv preprint arXiv:2307.09288 (2023)

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bash- lykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  43. [51]

    arXiv preprint arXiv:2506.13130 (2025)

    Wada, Y., Matsuda, K., Sugiura, K., Neubig, G.: Zina: Multimodal fine-grained hallucination detection and editing. arXiv preprint arXiv:2506.13130 (2025)

    2025 arXiv

  44. [52]

    arXiv preprint arXiv:2502.00717 (2025)

    Wang, C., Yang, J., Zhou, Y.: Mint: Mitigating hallucinations in large vision- language models via token reduction. arXiv preprint arXiv:2502.00717 (2025)

    2025 arXiv

  45. [53]

    Wang,H.,Lai,C.,Sun,Y.,Ge,W.:Weaklysupervisedgaussiancontrastiveground- ing with large multimodal models for video question answering (2024)

    2024

  46. [54]

    arXiv preprint arXiv:2601.01957 (2026)

    Wang, T., Ma, Y., Liao, K., Zhang, Z., Li, S., Guo, J., Liu, X.: After: Mitigating the object hallucination of lvlm via adaptive factual-guided activation editing. arXiv preprint arXiv:2601.01957 (2026)

    2026

  47. [55]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Z., Yang, S., Peng, B., Tang, Z., Li, Y., Dong, B., Dong, J.: Same attention, different truths: Put logit-lens over visual attention to detect and mitigate lvlm object hallucination. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p...

    2026

  48. [56]

    arXiv preprint arXiv:2512.04643 (2025)

    Wu, C.H., Chang, K.P., Sheng, Y.Y., Chung, H.K., Wang, K.C., Wang, Y.C.F.: Season: Mitigating temporal hallucination in video large language models via self- diagnostic contrastive decoding. arXiv preprint arXiv:2512.04643 (2025)

    2025

  49. [57]

    arXiv preprint arXiv:2402.11622 (2024)

    Wu,J.,Liu,Q.,Wang,D.,Zhang,J.,Wu,S.,Wang,L.,Tan,T.:Logicalclosedloop: Uncovering object hallucinations in large vision-language models. arXiv preprint arXiv:2402.11622 (2024)

    2024 arXiv

  50. [58]

    Advances in Neural Information Processing Systems37, 45206– 45234 (2024)

    Wu, M., Cai, X., Ji, J., Li, J., Huang, O., Luo, G., Fei, H., Jiang, G., Sun, X., Ji, R.: Controlmllm: Training-free visual prompt learning for multimodal large language models. Advances in Neural Information Processing Systems37, 45206– 45234 (2024)

    2024

  51. [59]

    In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference

    Yang, L., Zheng, Z., Chen, B., Zhao, Z., Lin, C., Shen, C.: Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection. In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference. pp. 14635– 14645 (2025)

    2025

  52. [60]

    arXiv preprint arXiv:2304.14178 (2023)

    Ye, Q., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P., Shi, Y., et al.: mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178 (2023)

    2023 arXiv

  53. [61]

    In: CVPR

    Ye, Q., Xu, H., Ye, J., Yan, M., Hu, A., Liu, H., Qian, Q., Zhang, J., Huang, F.: mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In: CVPR. pp. 13040–13051 (2024)

    2024

  54. [62]

    arXiv preprint arXiv:2602.21704 (2026)

    Yin, J., Chen, Q., Chen, K., Zhou, J., Wu, X., He, L.: Dynamic multimodal acti- vation steering for hallucination mitigation in large vision-language models. arXiv preprint arXiv:2602.21704 (2026)

    2026

  55. [63]

    arXiv preprint arXiv:2310.16045 (2023)

    Yin, S., Fu, C., Zhao, S., Xu, T., Wang, H., Sui, D., Shen, Y., Li, K., Sun, X., Chen, E.: Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045 (2023)

    2023 arXiv

  56. [64]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition

    Yu, Q., Li, J., Wei, L., Pang, L., Ye, W., Qin, B., Tang, S., Tian, Q., Zhuang, Y.: Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 12944–12953 (2024) TTH 19

    2024

  57. [65]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yuan, Y., Li, W., Liu, J., Tang, D., Luo, X., Qin, C., Zhang, L., Zhu, J.: Os- prey: Pixel understanding with visual instruction tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 28202– 28211 (2024)

    2024

  58. [66]

    arXiv preprint arXiv:2502.06130 (2025)

    Zhang, C., Wan, Z., Kan, Z., Ma, M.Q., Stepputtis, S., Ramanan, D., Salakhutdi- nov, R., Morency, L.P., Sycara, K., Xie, Y.: Self-correcting decoding with genera- tive feedback for mitigating hallucinations in large vision-language models. arXiv preprint arXiv:2502.06130 (2025)

    2025 arXiv

  59. [68]

    Zhang, C., Sun, C., Jiang, X., Li, W., Tian, X.: Prefill-time intervention for miti- gating hallucination in large vision-language models (2026),https://arxiv.org/ abs/2604.25642

    2026 arXiv

  60. [69]

    Zhao, B., Wu, B., He, M., Huang, T.: Svit: Scaling up visual instruction tuning (2023),https://arxiv.org/abs/2307.04087

    2023 arXiv

  61. [70]

    arXiv preprint arXiv:2603.04908 (2026)

    Zhong, L., He, Z., Zheng, J., Li, J., Wang, Z.J., Kang, X.: Adaiat: Adaptively increasing attention to generated text to alleviate hallucinations in lvlm. arXiv preprint arXiv:2603.04908 (2026)

    2026

  62. [71]

    arXiv preprint arXiv:2310.00754 (2023)

    Zhou, Y., Cui, C., Yoon, J., Zhang, L., Deng, Z., Finn, C., Bansal, M., Yao, H.: Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754 (2023)

    2023 arXiv

  63. [72]

    Zhu, B., Niu, Y., Lee, S., Hur, M., Zhang, H.: Debiased fine-tuning for vision- language models by prompt regularization (2025),https://arxiv.org/abs/2301. 12429

    2025

  64. [73]

    arXiv preprint arXiv:2304.10592 (2023)

    Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023)

    2023 arXiv

  65. [74]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Zhu, L., Ji, D., Chen, T., Xu, P., Ye, J., Liu, J.: Ibd: Alleviating hallucinations in large vision-language models via image-biased decoding. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 1624–1633 (2025)

    2025

  66. [75]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Zhuang, X., Zhu, Z., Xie, Y., Liang, L., Zou, Y.: Vasparse: Towards efficient visual hallucination mitigation via visual-aware token sparsification. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 4189–4199 (2025)

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.