Pith. sign in

REVIEW 2 major objections 2 minor 60 cited by

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

AMBER provides an LLM-free benchmark to evaluate hallucinations in multi-modal models across existence, attribute and relation dimensions for generative and discriminative tasks.

2026-05-16 06:41 UTC pith:4QMBPWEV

load-bearing objection AMBER sets up an LLM-free benchmark for multi-dimensional hallucination evaluation in MLLMs, but lacks shown validation for its pipeline. the 2 major comments →

arxiv 2311.07397 v2 pith:4QMBPWEV submitted 2023-11-13 cs.CL cs.CV

AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation

Junyang Wang , Yuhang Wang , Guohai Xu , Jing Zhang , Yukai Gu , Haitao Jia , Jiaqi Wang , Haiyang Xu
show 3 more authors
Ming Yan Ji Zhang Jitao Sang
This is my paper
classification cs.CL cs.CV
keywords AMBER benchmarkMLLM hallucinationexistence hallucinationattribute hallucinationrelation hallucinationLLM-free evaluationmulti-modal benchmarkgenerative discriminative tasks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces AMBER as a benchmark that assesses hallucinations in Multi-modal Large Language Models without using other LLMs or human evaluators. It supports evaluation of both generative outputs and discriminative judgments, with specific coverage of existence hallucinations where objects are invented, attribute hallucinations where properties are wrongly assigned, and relation hallucinations where connections between elements are misstated. A low-cost automated pipeline accompanies the benchmark to score model responses efficiently. The authors apply it to models including GPT-4V to produce comparative results and offer mitigation guidelines. This approach addresses the high costs and limited scope of prior hallucination checks, which matter because unchecked hallucinations can produce misleading or harmful multi-modal responses in deployed systems.

Core claim

AMBER is an LLM-free multi-dimensional benchmark that evaluates MLLMs on generative and discriminative tasks for existence, attribute, and relation hallucinations, supported by a low-cost evaluation pipeline that allows comprehensive assessment of mainstream models.

What carries the argument

The AMBER benchmark, which supplies curated image-text pairs and automated scoring rules to detect and categorize hallucinations without external LLM assistance.

Load-bearing premise

The low-cost evaluation pipeline accurately detects and categorizes hallucinations without introducing new biases or missing cases that would need LLM or human judgment.

What would settle it

Human annotations on a held-out set of MLLM outputs showing that AMBER pipeline scores differ substantially from human labels on hallucination presence or type.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Mainstream MLLMs receive consistent scores on hallucination rates that distinguish generative from discriminative performance.
  • Existence, attribute, and relation hallucinations can be measured separately to reveal which error type dominates in a given model.
  • Mitigation guidelines derived from the benchmark results can be tested directly on the same evaluation sets.
  • Wider adoption of the pipeline reduces reliance on expensive human or advanced-LLM judging for routine MLLM checks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark generalizes beyond the tested models, it could serve as a standard reference set for tracking hallucination reduction over successive MLLM releases.
  • Separate scoring of the three hallucination types may expose trade-offs, such as models that improve on relations but worsen on attributes.
  • The automated pipeline opens the possibility of incorporating AMBER-style checks into training loops to penalize hallucination during fine-tuning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces AMBER, an LLM-free multi-dimensional benchmark for evaluating hallucinations in Multi-modal Large Language Models (MLLMs). It supports both generative and discriminative tasks across existence, attribute, and relation hallucination types, includes a low-cost evaluation pipeline, reports comprehensive evaluations on models such as GPT-4V, and offers mitigation guidelines. The data and code are released publicly.

Significance. If the LLM-free pipeline is shown to match human or LLM-based judgments with high fidelity, AMBER would provide a scalable, low-cost alternative to existing high-cost hallucination benchmarks, enabling broader model assessment and iterative improvement in the MLLM community. The multi-dimensional coverage and public release are clear strengths.

major comments (2)
  1. [§4] §4 (Evaluation Pipeline): The central claim that the pipeline accurately detects and categorizes hallucinations in a fully LLM-free manner lacks any quantitative validation. No precision/recall figures, inter-annotator agreement with human labels, or ablation on edge cases (partial attribute matches, relational ambiguities) are reported, making it impossible to assess whether the pipeline introduces systematic misses or new biases.
  2. [§5] §5 (Experiments): The reported hallucination rates for GPT-4V and other models are presented without direct comparison to human-annotated ground truth or to existing LLM-based benchmarks on the same test cases. This omission leaves the practical utility of the benchmark unverified and weakens the claim of comprehensive evaluation.
minor comments (2)
  1. [Abstract, §3.1] The abstract and §3.1 would benefit from a concise table summarizing the three hallucination dimensions and the generative vs. discriminative task distinctions.
  2. [§4] Notation for the pipeline components (e.g., matching rules for attributes) is introduced without a formal definition or pseudocode, reducing reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on AMBER. We address the two major comments below and will incorporate revisions to strengthen the validation of the evaluation pipeline and experimental results.

read point-by-point responses

  1. Referee: [§4] §4 (Evaluation Pipeline): The central claim that the pipeline accurately detects and categorizes hallucinations in a fully LLM-free manner lacks any quantitative validation. No precision/recall figures, inter-annotator agreement with human labels, or ablation on edge cases (partial attribute matches, relational ambiguities) are reported, making it impossible to assess whether the pipeline introduces systematic misses or new biases.

    Authors: We agree that quantitative validation against human judgments is essential to substantiate the pipeline's reliability. In the revised manuscript, we will add a dedicated subsection in §4 reporting precision, recall, and F1 scores computed on a human-annotated subset of 500 samples. We will also report inter-annotator agreement (Cohen's kappa) and include targeted ablations addressing partial attribute matches and relational ambiguities, with explicit discussion of any observed biases or failure modes. revision: yes

  2. Referee: [§5] §5 (Experiments): The reported hallucination rates for GPT-4V and other models are presented without direct comparison to human-annotated ground truth or to existing LLM-based benchmarks on the same test cases. This omission leaves the practical utility of the benchmark unverified and weakens the claim of comprehensive evaluation.

    Authors: We acknowledge the value of direct comparisons for verifying practical utility. The revised §5 will include a new table comparing AMBER-derived hallucination rates against human-annotated ground truth on a shared subset of test cases, as well as side-by-side results with at least two existing LLM-based benchmarks (e.g., POPE and LURE) on overlapping samples where feasible. This will be accompanied by analysis of agreement rates and discrepancies. revision: yes

Circularity Check

0 steps flagged

No significant circularity in AMBER benchmark proposal

full rationale

The paper introduces AMBER as a new LLM-free multi-dimensional benchmark for evaluating hallucinations in MLLMs across generative and discriminative tasks (existence, attribute, relation). The central claim is the construction and application of this benchmark with a low-cost evaluation pipeline. No self-definitional loops, fitted parameters renamed as predictions, or load-bearing self-citations appear in the provided abstract or description. The pipeline is positioned as an independent, rule-based mechanism rather than deriving from its own outputs or prior author results by construction. This is a standard benchmark proposal with self-contained content against external model evaluations.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The proposal rests on domain assumptions about hallucination categories and the viability of automated evaluation without external models.

axioms (1)
  • domain assumption Hallucinations in MLLMs can be reliably categorized into existence, attribute, and relation types.
    This categorization underpins the multi-dimensional design of the benchmark.

pith-pipeline@v0.9.0 · 5503 in / 1034 out tokens · 31208 ms · 2026-05-16T06:41:41.603498+00:00 · methodology

0 comments
read the original abstract

Despite making significant progress in multi-modal tasks, current Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations, which may lead to harmful consequences. Therefore, evaluating MLLMs' hallucinations is becoming increasingly important in model improvement and practical application deployment. Previous works are limited in high evaluation costs (e.g., relying on humans or advanced LLMs) and insufficient evaluation dimensions (e.g., types of tasks and hallucinations). In this paper, we propose an LLM-free multi-dimensional benchmark AMBER, which can be used to evaluate both generative task and discriminative task including existence, attribute and relation hallucination. Based on AMBER, we design a low-cost and efficient evaluation pipeline. Additionally, we conduct a comprehensive evaluation and detailed analysis of mainstream MLLMs including GPT-4V(ision), and also give guideline suggestions for mitigating hallucinations. The data and code of AMBER are available at https://github.com/junyangwang0410/AMBER.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DataComp-VLM: Improved Open Datasets for Vision-Language Models

    cs.CV 2026-06 conditional novelty 8.0

    DataComp-VLM benchmark shows instruction-heavy data mixing outperforms filtering for VLM training, with DCVLM-Baseline achieving 63.6% on 33 tasks for 8B models (+5.4pp over FineVision).

  2. AVID: A Benchmark for Omni-Modal Audio-Visual Inconsistency Understanding via Agent-Driven Construction

    cs.MM 2026-04 unverdicted novelty 8.0

    AVID is the first large-scale benchmark for audio-visual inconsistency detection, grounding, classification, and reasoning in long videos, constructed via agent-driven methods and showing that state-of-the-art models ...

  3. MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs

    cs.CV 2025-11 unverdicted novelty 8.0

    MVI-Bench supplies the first taxonomy and dataset focused on misleading visual inputs to measure LVLM robustness, with tests on 18 models revealing clear weaknesses.

  4. MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models

    cs.CV 2026-07 unverdicted novelty 7.0

    MoHallBench is a new benchmark evaluating motion hallucination in VideoLLMs from co-occurrence priors, sequential inference, and similarity confusion, revealing decoupling from action recognition performance.

  5. Mind the Heads: Topological Representation Alignment for Multimodal LLMs

    cs.CV 2026-06 conditional novelty 7.0

    HeRA aligns least-aligned attention heads in MLLMs using an MKNN-based contrastive objective to preserve cross-modal topological structure, yielding gains on vision-centric tasks and reduced hallucinations across 18 b...

  6. P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization

    cs.CV 2026-06 unverdicted novelty 7.0

    P²-DPO generates on-policy preference pairs targeting focus-and-enhance perception and visual robustness, combined with a calibration loss, to reduce hallucinations in LVLMs more effectively than human-feedback baselines.

  7. MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialogue

    cs.CV 2026-05 unverdicted novelty 7.0

    MM-Snowball benchmark diagnoses hallucination snowballing in multi-turn MLLM dialogues; CAVR mitigates it via dual visual rectification at representation and logit levels.

  8. YARD: Y-Architecture Register Decoding for Efficient Hallucination Mitigation in Large Vision-Language Models

    cs.CV 2026-05 unverdicted novelty 7.0

    YARD is a training-free method using Y-shaped decoder architecture and register tokens to improve contrastive decoding for hallucination reduction in LVLMs with lower latency.

  9. The Abstraction Gap in Vision-Language Causal Reasoning

    cs.CL 2026-05 unverdicted novelty 7.0

    Introduces Abstraction Gap metric and CAGE benchmark showing seven of eight VLMs have large gaps between text plausibility and chain-based causal reasoning, with one model succeeding.

  10. Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation

    cs.CL 2026-05 unverdicted novelty 7.0

    CAS mitigates object hallucinations in MLLMs by extracting two context preference vectors from designed conflict samples and applying signed residual injection at mid-early MLP layers without retraining or added latency.

  11. How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study

    cs.AI 2026-05 unverdicted novelty 7.0

    EEG study of 27 participants reveals distinct neural patterns for AI-generated hallucinations, with misjudged ones failing to trigger standard fact verification pathways.

  12. Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution

    cs.CV 2026-05 conditional novelty 7.0

    SIRA mitigates hallucinations in LVLMs by internally contrasting full visual access against a masked late-layer branch that retains shared context but lacks fine-grained visual evidence.

  13. OxyEcomBench: Benchmarking Multimodal Foundation Models across E-Commerce Ecosystems

    cs.DB 2026-05 conditional novelty 7.0

    OxyEcomBench is a unified multimodal benchmark covering 6 capability areas and 29 tasks with authentic e-commerce data to measure how well foundation models handle real platform, merchant, and customer challenges.

  14. DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models

    cs.CV 2026-04 unverdicted novelty 7.0

    DO-Bench is a controlled benchmark that attributes VLM object hallucination errors to textual prior pressure, perceptual limits, or their interaction via two diagnostic dimensions and metrics.

  15. ZINA: Multimodal Fine-grained Hallucination Detection and Editing

    cs.CV 2025-06 unverdicted novelty 7.0

    ZINA detects fine-grained hallucinations in MLLM outputs, classifies errors into six types, and proposes edits, outperforming GPT-4o and Llama-3.2 on the new VisionHall dataset of annotated and synthetic samples.

  16. VidHal: Benchmarking Temporal Hallucinations in Vision LLMs

    cs.CV 2024-11 unverdicted novelty 7.0

    VidHal is a new benchmark that evaluates VLLM temporal hallucinations through a caption ordering task on videos with varying hallucination levels.

  17. ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

    cs.CV 2026-06 unverdicted novelty 6.0

    ADAPT reduces MLLM hallucinations 40-60% by aligning cross-attention dynamics via visual anchors, supervised inference, and preference tuning while preserving general capabilities.

  18. Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

    cs.CV 2026-06 unverdicted novelty 6.0

    OPPO is an evidence-aware preference optimization objective that contrasts faithful responses under varying visual evidence strengths to reduce hallucinations in MLLMs.

  19. DataComp-VLM: Improved Open Datasets for Vision-Language Models

    cs.CV 2026-06 unverdicted novelty 6.0

    DataComp-VLM benchmark shows instruction-heavy data mixtures outperform caption-heavy ones for VLM training, with DCVLM-Baseline reaching 63.6% on 33 tasks using 200B tokens, +5.4pp over FineVision.

  20. Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty

    cs.CV 2026-06 unverdicted novelty 6.0

    A counterfactual visual grounding uncertainty method detects hallucinations in LVLMs on medical images, improving over baselines with interpretable evidence and cross-model transfer.

  21. GAVEL: Grounded Caption Error Verification and Localization

    cs.CL 2026-06 unverdicted novelty 6.0

    GAVEL introduces a joint task, dataset, and benchmark for verifying, explaining, and localizing caption-image misalignments, with a supervised baseline that improves grounding and explanation metrics over strong close...

  22. Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMs

    cs.CV 2026-06 unverdicted novelty 6.0

    ViPSy constructs policy-aligned and visually grounded preference pairs for VLMs via visual cues from image variants, yielding SOTA hallucination reductions of 35.7% on AMBER and 24.5% on Object HalBench.

  23. Spectral Query-Key Product Weight Steering for Training-Free VLM Hallucination Mitigation

    cs.CV 2026-06 unverdicted novelty 6.0

    QK Product Steering suppresses dominant singular modes in the per-head QK product of selected middle layers via a closed-form query-only update, yielding 4.0% average relative CHAIR_s reduction on three GQA VLMs.

  24. OmniHalluc-L: Counterfactual Benchmarking and Modality-Perturbation Reliability Calibration for Long-Form Omni Hallucination

    cs.MM 2026-06 unverdicted novelty 6.0

    OmniHalluc-L benchmark shows open-weight omni models at 32-41% strict-pair accuracy on long-form hallucination, raised to 36-51% by Modality-Perturbation Reliability Calibration that fuses audio-negative probe shifts ...

  25. Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation

    cs.CV 2026-06 unverdicted novelty 6.0

    TLVS mitigates hallucinations in LVLMs via token-level extraction and visual-sensitivity-adaptive steering applied only at critical decoding steps.

  26. Learning from Fine-Grained Visual Discrepancies: Mitigating Multimodal Hallucinations via In-Context Visual Contrastive Optimization

    cs.CV 2026-05 unverdicted novelty 6.0

    IC-VCO places contrastive images in one context for a consistent DPO-style objective, adds Visual Contrast Distillation, and uses semantic perturbation for hard negatives, reporting best results on five benchmarks.

  27. Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization

    cs.AI 2026-05 unverdicted novelty 6.0

    RC-DPO adds a CoT-conditioned preference term to DPO and pairs it with MCTS-based positive CoT generation plus attention-guided pruning for negatives, yielding lower hallucination rates on multimodal benchmarks.

  28. DRScaffold: Boosting Dense-Scene Reasoning in Lightweight Vision Language Models

    cs.CV 2026-05 unverdicted novelty 6.0

    New benchmark DRBench and four-stage supervision framework DRScaffold improve dense-scene reasoning in lightweight VLMs, with a 3B model surpassing a frozen 32B model on the benchmark while maintaining general performance.

  29. Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation

    cs.CV 2026-05 unverdicted novelty 6.0

    AOD isolates hallucination signals in LVLM representations with an adversarial minimax objective and uses dual-forward contrastive decoding to reduce hallucinations while preserving utility.

  30. Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?

    cs.CV 2026-05 accept novelty 6.0

    VLMs achieve 53-97% on volumetric rearrangement planning but only 6-45% on occlusion and under 7% on reflections in a new 3,034-sample benchmark, with white-box analysis localizing the failure to visual-token merger i...

  31. Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?

    cs.CV 2026-05 unverdicted novelty 6.0

    VLMs achieve 53-97% on rearrangement planning but only 6-45% on occlusion and under 7% on reflections, with failures localized to visual token compression after the vision encoder.

  32. How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study

    cs.AI 2026-05 unverdicted novelty 6.0

    EEG study reveals distinct ERP patterns for AI hallucinations, with misjudged ones failing to trigger standard neurocognitive verification pathways.

  33. Deep Pre-Alignment for VLMs

    cs.CV 2026-05 unverdicted novelty 6.0

    Deep Pre-Alignment uses a small VLM perceiver instead of ViT to pre-align visual features with LLM text space, yielding 1.9-3.0 point gains on multimodal benchmarks and 32.9% less language forgetting.

  34. Dual-Pathway Circuits of Object Hallucination in Vision-Language Models

    cs.CV 2026-05 unverdicted novelty 6.0

    Vision-language models contain identifiable grounding and hallucination pathways; suppressing the latter reduces object hallucinations by up to 76% while preserving accuracy.

  35. Learning to See What You Need: Gaze Attention for Multimodal Large Language Models

    cs.CV 2026-05 unverdicted novelty 6.0

    Gaze Attention groups visual embeddings into selectable regions and dynamically restricts attention to task-relevant ones, matching dense baselines with up to 90% fewer visual KV entries via added context tokens.

  36. Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination

    cs.MM 2026-05 unverdicted novelty 6.0

    LVLMs show vocabulary hijacking by inert tokens that decode to hijacking anchors; HABI locates them, NHAR finds resilient heads, and HAVAE boosts those heads to cut hallucinations.

  37. Through the Lens of Character: Resolving Modality-Role Interference in Multimodal Role-Playing Agent

    cs.CV 2026-05 unverdicted novelty 6.0

    CAVI framework uses character-guided token pruning, orthogonal feature modulation, and modality-adaptive role steering to resolve modality-role interference in multimodal RPAs.

  38. R-CoV: Region-Aware Chain-of-Verification for Alleviating Object Hallucinations in LVLMs

    cs.CV 2026-04 conditional novelty 6.0

    R-CoV is a six-step region-aware chain-of-verification technique that elicits coordinate and description outputs from LVLMs themselves to detect and reduce object hallucinations without external models or retraining.

  39. Mitigating Multimodal Hallucination via Phase-wise Self-reward

    cs.CV 2026-04 unverdicted novelty 6.0

    PSRD mitigates visual hallucinations in LVLMs via phase-wise self-reward decoding, cutting rates by 50% on LLaVA-1.5-7B and outperforming prior methods on five benchmarks.

  40. SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

    cs.CV 2026-04 unverdicted novelty 6.0

    SIF creates semantically in-distribution fingerprints for LVLMs by distilling text watermarks into visual inputs and optimizing for robustness against detection and modification.

  41. Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention

    cs.CV 2025-11 unverdicted novelty 6.0

    VGA constructs precise visual grounding from token semantics to guide MLLM attention toward relevant regions, dynamically suppressing described areas in captioning, and achieves SOTA dehallucination with negligible overhead.

  42. ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models

    cs.CV 2025-09 unverdicted novelty 6.0

    ORCA is an inference-time agentic framework that boosts LVLM accuracy on hallucination benchmarks by 3.64-40.67% and adds adversarial robustness via cross-model validation with small vision tools.

  43. ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models

    cs.CV 2025-09 unverdicted novelty 6.0

    ORCA is an agentic reasoning framework that enhances factual accuracy and adversarial robustness of pretrained LVLMs via an Observe-Reason-Critique-Act loop with small vision models, reporting accuracy gains of up to ...

  44. TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs

    cs.CV 2025-07 unverdicted novelty 6.0

    TARS uses token-adaptive min-max preference optimization and FFT-based spectral regularization to cut hallucination rates in MLLMs from 26.4% to 13.2% with only 4.8k samples, outperforming standard DPO and larger data...

  45. Mitigating Object Hallucinations via Sentence-Level Early Intervention

    cs.CV 2025-07 conditional novelty 6.0

    SENTINEL reduces MLLM object hallucinations by over 90% via sentence-level early intervention with detector-bootstrapped preference data and C-DPO loss, outperforming prior SOTA on hallucination and capability benchmarks.

  46. Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

    cs.CV 2026-06 unverdicted novelty 5.0

    OPPO is an evidence-aware preference optimization that contrasts faithful responses under varying visual evidence strengths to reduce hallucinations in MLLMs.

  47. Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs

    cs.CV 2026-06 unverdicted novelty 5.0

    VIGIL is a counterfactual RL alignment method that reduces visual hallucinations in MLLMs by enforcing visual grounding via masked attention penalties, outperforming baselines with 25% of the data and showing emergent...

  48. Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality

    cs.CV 2026-06 unverdicted novelty 5.0

    MACCO applies cross-modal masked reconstruction of compositional concepts with inter- and intra-modal auxiliary objectives to improve visio-linguistic compositionality in VLMs.

  49. VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning

    cs.CV 2026-05 unverdicted novelty 5.0

    VCap pairs reference captions as witnesses with visual signals as adjudicators to deliver hypergeometric-precision rewards for RL in visual captioning, enabling an 8B model to outperform SOTA on benchmarks and improve...

  50. CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs

    cs.CV 2026-05 unverdicted novelty 5.0

    CHASD is an inference-time framework that gates contrastive decoding via an uncertainty threshold and constructs negative branches through attention-guided perturbations of salient visual tokens to mitigate hallucinat...

  51. MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs

    cs.CV 2026-05 unverdicted novelty 5.0

    MHSA mitigates hallucinations in LVLMs by training an MLP to steer cross-modal attention, extending detection work to mitigation via attention replacement at inference.

  52. Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models

    cs.LG 2026-05 unverdicted novelty 5.0

    UE-DPO quantifies epistemic uncertainty from grounding failures to direct more learning pressure on hard visual tokens in preferred samples while easing penalties on dispreferred ones.

  53. Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation

    cs.CV 2026-04 unverdicted novelty 5.0

    MPD reduces hallucinations in LVLMs by 23.4% while retaining 97.4% of general capability through semantic disentanglement and selective parameter updates.

  54. Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction

    cs.CV 2026-04 unverdicted novelty 5.0

    MESA reduces hallucinations in LVLMs via controlled selective latent intervention that preserves the original token distribution.

  55. Steering the Verifiability of Multimodal AI Hallucinations

    cs.AI 2026-04 unverdicted novelty 5.0

    Researchers create a human-labeled dataset of obvious and elusive multimodal hallucinations and use learned activation-space probes to control their verifiability in MLLMs.

  56. NoisyGRPO: Incentivizing Multimodal CoT Reasoning via Noise Injection and Bayesian Estimation

    cs.CV 2025-10 unverdicted novelty 5.0

    NoisyGRPO is an RL framework that perturbs visual inputs with Gaussian noise for exploration and computes trajectory advantages via Bayesian posterior fusion of noise prior and reward likelihood to improve multimodal ...

  57. Revisit What You See: Revealing Visual Semantics in Vision Tokens to Guide LVLM Decoding

    cs.CV 2025-06 unverdicted novelty 5.0

    ReVisiT refines LVLM output distributions during decoding by projecting selected vision tokens into text space via context-aware constrained divergence minimization.

  58. Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibration

    cs.CV 2025-05 unverdicted novelty 5.0

    CAAC mitigates hallucinations in LVLMs via Visual-Token Calibration and Adaptive Attention Re-Scaling guided by model confidence, showing gains on CHAIR, AMBER, and POPE especially in long-form generation.

  59. Hallucination of Multimodal Large Language Models: A Survey

    cs.CV 2024-04 accept novelty 5.0

    The survey organizes causes of hallucinations in MLLMs, reviews evaluation benchmarks and metrics, and outlines mitigation approaches plus open questions.

  60. Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning

    cs.CV 2026-06 unverdicted novelty 4.0

    ConsistRoll enforces cross-view consistency during RLVR training for MLLMs by joint rewards on grouped original and augmented views, yielding robustness gains on math, general, and hallucination benchmarks.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages · cited by 57 Pith papers · 8 internal anchors

  1. [1]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

    Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966. Jun Chen, Deyao Zhu Xiaoqian Shen Xiang Li, Zechun Liu Pengchuan Zhang, Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong, and Mohamed El- hoseiny

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

    Minigpt-v2: Large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al

    work page internal anchor Pith review arXiv

  3. [3]

    Holistic analysis of hallucination in gpt-4v (ision): Bias and interference chal- lenges,

    Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi

    work page arXiv

  4. [4]

    InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning

    In- structblip: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2308.06394 , year=

    De- tecting and preventing hallucinations in large vision language models. arXiv preprint arXiv:2308.06394. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen

    work page arXiv

  6. [6]

    Evaluating Object Hallucination in Large Vision-Language Models

    Eval- uating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

    Microsoft coco: Common objects in context. In European confer- ence on computer vision , pages 740–755. Springer. Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2023a. Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), l...

    work page internal anchor Pith review arXiv

  8. [8]

    GPT-4 Technical Report

    Gpt-4 technical report. ArXiv, abs/2303.08774. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    Object Hallucination in Image Captioning

    Object hallucination in image captioning. arXiv preprint arXiv:1809.02156. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971. 9 Hugo Touvron, Lou...

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2310.16534

    An early evaluation of gpt-4v (ision). arXiv preprint arXiv:2310.16534. Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia. 2023a. Cognitive mirage: A review of hallucinations in large language models. arXiv preprint arXiv:2309.06794. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, An- wen Hu, Pengcheng Shi, Yay...

    work page arXiv

  11. [11]

    Woodpecker: Hallucination correction for multimodal large language models,

    Woodpecker: Hallucina- tion correction for multimodal large language models. arXiv preprint arXiv:2310.16045. Bohan Zhai, Shijia Yang, Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Man- ling Li, Tan Yan, and Xiangjun Fan

    work page arXiv

  12. [12]

    Halle-switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption.arXiv preprint arXiv:2310.01779,

    Halle- switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption. arXiv preprint arXiv:2310.01779. Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. 2023a. Don’t make your llm an evaluation benchmark cheater. arXiv preprint arXiv:...

    work page arXiv

  13. [13]

    MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

    Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592. 10 A Appendix A.1 Details of AMBER A.1.1 Data Source Our images are sourced from the MS-COCO 2014 (Lin et al.,

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  14. [14]

    UnSplash adopts the Cre- ative Commons Zero (CC0) license, allowing free use anywhere without requiring attribution to the source or author

    It is important to note that the MS-COCO 2014 test set does not include annotations and is therefore not used as training data. UnSplash adopts the Cre- ative Commons Zero (CC0) license, allowing free use anywhere without requiring attribution to the source or author. A.1.2 Data Dtatistics We present the data statistics for AMBER in Ta- ble

    work page 2014

  15. [15]

    The category column represents the num- ber of different categories present in the data. It is worth mentioning that the resultant annotations cover 337 objects, which is more than 4 times the number of objects in the existing benchmarks (e.g., 80 specific objects in coco). Image 1004 Categories Object 337 Attribute 350 Prompt Generation 1004 Existence 49...

    work page 2023