Pith. sign in

REVIEW 20 cited by

Woodpecker: Hallucination Correction for Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.16045 v2 pith:ZJ5DO3WS submitted 2023-10-24 cs.CV cs.AIcs.CLcs.LG

Shukang Yin , Chaoyou Fu , Sirui Zhao , Tong Xu , Hao Wang , Dianbo Sui , Yunhang Shen , Ke Li
show 2 more authors
Xing Sun Enhong Chen
This is my paper
classification cs.CVcs.AIcs.CLcs.LG
keywords woodpeckerhallucinationmodelscorrectiondifferentfivegeneratedhallucinations
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content. In order to mitigate hallucinations, existing studies mainly resort to an instruction-tuning manner that requires retraining the models with specific data. In this paper, we pave a different way, introducing a training-free method named Woodpecker. Like a woodpecker heals trees, it picks out and corrects hallucinations from the generated text. Concretely, Woodpecker consists of five stages: key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction. Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages. We evaluate Woodpecker both quantitatively and qualitatively and show the huge potential of this new paradigm. On the POPE benchmark, our method obtains a 30.66%/24.33% improvement in accuracy over the baseline MiniGPT-4/mPLUG-Owl. The source code is released at https://github.com/BradyFU/Woodpecker.

Discussion (0). Sign in to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VidHal: Benchmarking Temporal Hallucinations in Vision LLMs

    cs.CV 2024-11 unverdicted novelty 7.0 of 10

    VidHal is a new benchmark that evaluates VLLM temporal hallucinations through a caption ordering task on videos with varying hallucination levels.

  2. HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

    cs.CV 2023-10 unverdicted novelty 7.0 of 10

    HallusionBench shows GPT-4V reaches only 31.42% accuracy on paired questions testing language hallucination and visual illusion in LVLMs, with other models below 16%.

  3. Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers

    cs.CV 2026-07 accept novelty 6.0 of 10

    A single-prefill, white-box risk detector that estimates signed visual evidence from internal attention/value readouts and fuses it with confidence improves error ranking by 0.134 mean AP across 12 model–benchmark pairs.

  4. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  5. Deep Pre-Alignment for VLMs

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Deep Pre-Alignment uses a small VLM perceiver instead of ViT to pre-align visual features with LLM text space, yielding 1.9-3.0 point gains on multimodal benchmarks and 32.9% less language forgetting.

  6. Dual-Pathway Circuits of Object Hallucination in Vision-Language Models

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Vision-language models contain identifiable grounding and hallucination pathways; suppressing the latter reduces object hallucinations by up to 76% while preserving accuracy.

  7. Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination

    cs.MM 2026-05 unverdicted novelty 6.0 of 10

    LVLMs show vocabulary hijacking by inert tokens that decode to hijacking anchors; HABI locates them, NHAR finds resilient heads, and HAVAE boosts those heads to cut hallucinations.

  8. Controlling Multimodal LLMs via Reward-guided Decoding

    cs.CV 2025-08 conditional novelty 6.0 of 10

    MRGD guides MLLM decoding with a learned hallucination reward and a detector-based recall reward, allowing users to trade off object precision, recall, and test-time compute while reducing object hallucinations on CHA...

  9. ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localization

    cs.CV 2024-10 unverdicted novelty 6.0 of 10

    ForgeryGPT integrates a forgery localization expert and mask encoder into an LLM for pixel-level forgery detection, localization, and explainable output via three-stage training on custom mask-text and instruction datasets.

  10. Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

    cs.CV 2023-11 unverdicted novelty 6.0 of 10

    Video-LLaVA creates a unified visual representation for images and videos via pre-projection alignment, enabling mutual enhancement from joint training and strong results on image and video benchmarks.

  11. AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation

    cs.CL 2023-11 unverdicted novelty 6.0 of 10

    AMBER is an LLM-free multi-dimensional benchmark for evaluating hallucinations in MLLMs across generative and discriminative tasks.

  12. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

    cs.CV 2023-06 unverdicted novelty 6.0 of 10

    MME is a manually annotated benchmark evaluating MLLMs on perception and cognition across 14 subtasks to avoid data leakage and support fair model comparisons.

  13. A Nash Equilibrium Framework For Training-Free Multimodal Step Verification

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    A Nash equilibrium framework for training-free multimodal step verification that uses cross-modal agreement and disagreement signals for filtering and ranking reasoning steps.

  14. Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    MPD reduces hallucinations in LVLMs by 23.4% while retaining 97.4% of general capability through semantic disentanglement and selective parameter updates.

  15. Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibration

    cs.CV 2025-05 unverdicted novelty 5.0 of 10

    CAAC mitigates hallucinations in LVLMs via Visual-Token Calibration and Adaptive Attention Re-Scaling guided by model confidence, showing gains on CHAIR, AMBER, and POPE especially in long-form generation.

  16. Hallucination of Multimodal Large Language Models: A Survey

    cs.CV 2024-04 accept novelty 5.0 of 10

    The survey organizes causes of hallucinations in MLLMs, reviews evaluation benchmarks and metrics, and outlines mitigation approaches plus open questions.

  17. Aligning Modalities in Vision Large Language Models via Preference Fine-tuning

    cs.LG 2024-02 unverdicted novelty 5.0 of 10

    POVID generates AI-created preference data to fine-tune vision-language models with DPO, reducing hallucinations and improving benchmark scores.

  18. InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model

    cs.CV 2024-01 unverdicted novelty 5.0 of 10

    InternLM-XComposer2 introduces Partial LoRA on InternLM2-7B to enable high-quality free-form text-image composition while matching or exceeding GPT-4V on select vision-language benchmarks.

  19. A Survey on Hallucination in Large Vision-Language Models

    cs.CV 2024-02 unverdicted novelty 3.0 of 10

    This survey reviews the definition, symptoms, evaluation benchmarks, root causes, and mitigation methods for hallucinations in large vision-language models.

  20. A Survey on Multimodal Large Language Models

    cs.CV 2023-06 accept novelty 3.0 of 10

    This survey organizes the architectures, training strategies, data, evaluation methods, extensions, and challenges of Multimodal Large Language Models.

Pith tools