Pith. sign in

REVIEW 5 cited by

Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.05338 v2 pith:AG6XBWED submitted 2023-10-09 cs.CV cs.CL

Holy Lovenia , Wenliang Dai , Samuel Cahyawijaya , Ziwei Ji , Pascale Fung This is my paper
classification cs.CVcs.CL
keywords modelsobjecthallucinationvisualnegativenegpnopeobjects
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Object hallucination poses a significant challenge in vision-language (VL) models, often leading to the generation of nonsensical or unfaithful responses with non-existent objects. However, the absence of a general measurement for evaluating object hallucination in VL models has hindered our understanding and ability to mitigate this issue. In this work, we present NOPE (Negative Object Presence Evaluation), a novel benchmark designed to assess object hallucination in VL models through visual question answering (VQA). We propose a cost-effective and scalable approach utilizing large language models to generate 29.5k synthetic negative pronoun (NegP) data of high quality for NOPE. We extensively investigate the performance of 10 state-of-the-art VL models in discerning the non-existence of objects in visual questions, where the ground truth answers are denoted as NegP (e.g., "none"). Additionally, we evaluate their standard performance on visual questions on 9 other VQA datasets. Through our experiments, we demonstrate that no VL model is immune to the vulnerability of object hallucination, as all models achieve accuracy below 10\% on NegP. Furthermore, we uncover that lexically diverse visual questions, question types with large scopes, and scene-relevant objects capitalize the risk of object hallucination in VL models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  2. The 3D Mirage: Probing and Taming 3D Hallucinations

    cs.CV 2025-12 reject novelty 6.0 of 10

    Depth models hallucinate 3D bumps on flat illusion images when context is cropped; the paper adds a benchmark, two scores, and a LoRA fine-tune that reduces the artifact on the same dataset.

  3. Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models

    cs.CV 2025-07 reject novelty 4.0 of 10

    Expert-CFG combines entropy-based uncertainty selection with classifier-free guidance over expert-highlighted text to refine MedVLM outputs, reporting gains on VQA-RAD, SLAKE, and PathVQA.

  4. HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMs

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A progressive two-stage knowledge distillation framework (HKD4VLM) reports first-place F1 scores of 98.2% and 98.4% on multimodal hallucination and factuality detection, but its ablation lacks a directly fine-tuned baseline.

  5. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Pith tools