Pith. sign in

REVIEW 10 cited by

Unified Hallucination Detection for Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.03190 v4 pith:3VKYMRST submitted 2024-02-05 cs.CL cs.AIcs.IRcs.LGcs.MM

Xiang Chen , Chenxi Wang , Yida Xue , Ningyu Zhang , Xiaoyan Yang , Qiang Li , Yue Shen , Lei Liang
show 2 more authors
Jinjie Gu Huajun Chen
This is my paper · ORCID
classification cs.CLcs.AIcs.IRcs.LGcs.MM
keywords hallucinationdetectionmultimodalevaluationhallucinationsapplicationcategorieslanguage
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Despite significant strides in multimodal tasks, Multimodal Large Language Models (MLLMs) are plagued by the critical issue of hallucination. The reliable detection of such hallucinations in MLLMs has, therefore, become a vital aspect of model evaluation and the safeguarding of practical application deployment. Prior research in this domain has been constrained by a narrow focus on singular tasks, an inadequate range of hallucination categories addressed, and a lack of detailed granularity. In response to these challenges, our work expands the investigative horizons of hallucination detection. We present a novel meta-evaluation benchmark, MHaluBench, meticulously crafted to facilitate the evaluation of advancements in hallucination detection methods. Additionally, we unveil a novel unified multimodal hallucination detection framework, UNIHD, which leverages a suite of auxiliary tools to validate the occurrence of hallucinations robustly. We demonstrate the effectiveness of UNIHD through meticulous evaluation and comprehensive analysis. We also provide strategic insights on the application of specific tools for addressing various categories of hallucinations.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features

    cs.CV 2024-11 conditional novelty 7.0 of 10

    SAVs extract a sparse set of attention head outputs from a frozen large multimodal model and use them as nearest-centroid features, achieving state-of-the-art few-shot vision-language classification without finetuning.

  2. UHP Detection: LVLMs have their Unique Hallucination Pattern in the Consistency Space

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Hallucination detection in vision-language models is improved by classifying a structured pattern of consistency across image/text perturbations and statement/negation probes, rather than relying on one uncertainty score.

  3. ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A self-attention model over hidden states of a vision-language model identifies hallucinated findings in AI-generated radiology reports with AUROC 0.8751, outperforming prior detectors.

  4. Advancing Fine-Grained Visual Understanding with Multi-Scale Alignment in Multi-Modal Models

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A three-stage training method that aligns object texts, coordinates, and cropped images, plus a GPT-4V-based data synthesis pipeline, lets 1.5B and 3B models match or exceed larger MLLMs on grounding and VQA benchmarks.

  5. Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Uncertainty-o estimates uncertainty in large multimodal models by perturbing prompts and computing entropy over semantically clustered answers, improving hallucination detection across five modalities.

  6. An archaeological Catalog Collection Method Based on Large Vision-Language Models

    cs.CV 2024-12 reject novelty 5.0 of 10

    A VLM-based pipeline with localize, comprehend, and match modules collects artifact catalogs at AP up to 35.4%, though evaluation circularity weakens the claim.

  7. Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Current image-text alignment metrics, including CLIPScore and DSGScore, produce unstable model rankings under random seeds and are highly sensitive to tiny image perturbations.

  8. MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs

    cs.CV 2024-11 conditional novelty 4.0 of 10

    A broad survey that organizes MLLM evaluation benchmarks into capability categories, explains benchmark construction and scoring methods, and identifies gaps in current evaluation practice.

  9. Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models

    cs.CL 2025-06 reject novelty 3.0 of 10

    A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.

  10. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Pith tools