Pith. sign in

REVIEW 5 cited by

NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.20081 v2 pith:CCIYBUNN submitted 2024-05-30 cs.CV cs.AI

Kai Wu , Boyuan Jiang , Zhengkai Jiang , Qingdong He , Donghao Luo , Shengzhi Wang , Qingwen Liu , Chengjie Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords noiseboostmllmsmodelslanguagelargedatahallucinationsnoise
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal large language models (MLLMs) contribute a powerful mechanism to understanding visual information building on large language models. However, MLLMs are notorious for suffering from hallucinations, especially when generating lengthy, detailed descriptions for images. Our analysis reveals that hallucinations stem from the inherent summarization mechanism of large language models, leading to excessive dependence on linguistic tokens while neglecting vision information. In this paper, we propose NoiseBoost, a broadly applicable and simple method for alleviating hallucinations for MLLMs through the integration of noise feature perturbations. Noise perturbation acts as a regularizer, facilitating a balanced distribution of attention weights among visual and linguistic tokens. Despite its simplicity, NoiseBoost consistently enhances the performance of MLLMs across common training strategies, including supervised fine-tuning and reinforcement learning. Further, NoiseBoost pioneerly enables semi-supervised learning for MLLMs, unleashing the power of unlabeled data. Comprehensive experiments demonstrate that NoiseBoost improves dense caption accuracy by 8.1% with human evaluation and achieves comparable results with 50% of the data by mining unlabeled data. Code and models are available at https://kaiwu5.github.io/noiseboost.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring

    cs.NI 2025-08 unverdicted novelty 6.0 of 10

    DUAL-Health is an uncertainty-aware multimodal fusion framework that quantifies sensor noise, customizes fusion weights accordingly, and aligns modality distributions to improve outdoor health monitoring.

  2. Learning to Insert [PAUSE] Tokens for Better Reasoning

    cs.CL 2025-06 reject novelty 6.0 of 10

    A likelihood-based [PAUSE] token insertion method for fine-tuning shows small gains on GSM8K and MBPP, but the AQUA-RAT result is unreliable because the test set contains training samples.

  3. OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination

    cs.AI 2025-08 conditional novelty 5.0 of 10

    OmniDPO extends direct preference optimization with audio-video alignment and modality-degradation preference pairs to reduce omni-modal hallucination.

  4. Causal-LLaVA: Causal Disentanglement for Mitigating Hallucination in Multimodal Large Language Models

    cs.AI 2025-05 reject novelty 5.0 of 10

    A causal intervention architecture with confounder dictionaries is applied to LLaVA, producing modest hallucination reductions on POPE and CHAIR but with methodological caveats.

  5. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Pith tools