Pith. sign in

REVIEW 5 cited by

Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.17820 v2 pith:FXNAGOVH submitted 2024-05-28 cs.CV cs.AI

Sangmin Woo , Donguk Kim , Jaehyuk Jang , Yubin Choi , Changick Kim This is my paper
classification cs.CVcs.AI
keywords visionattentionaviscblindlvlmstokensattentionalcalibration
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Vision Language Models (LVLMs) demonstrate strong capabilities in visual understanding and description, yet often suffer from hallucinations, attributing incorrect or misleading features to images. We observe that LVLMs disproportionately focus on a small subset of image tokens--termed blind tokens--which are typically irrelevant to the query (e.g., background or non-object regions). We hypothesize that such attention misalignment plays a key role in generating hallucinated responses. To mitigate this issue, we propose Attentional Vision Calibration (AvisC), a test-time approach that dynamically recalibrates the influence of blind tokens without modifying the underlying attention mechanism. AvisC first identifies blind tokens by analyzing layer-wise attention distributions over image tokens, then employs a contrastive decoding strategy to balance the influence of original and blind-token-biased logits. Experiments on standard benchmarks, including POPE, MME, and AMBER, demonstrate that AvisC effectively reduces hallucinations in LVLMs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  2. Focus on What Matters: Enhancing Medical Vision-Language Models with Automatic Attention Alignment Tuning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A3Tune aligns the visual attention of medical LVLMs to prompt-relevant regions via SAM and BioMedCLIP weak labels plus a Mixture-of-Experts over LoRA, improving VQA and report generation accuracy.

  3. SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering

    cs.CV 2026-07 conditional novelty 5.5 of 10

    Restructuring visual tokens via cross-modal prune–merge–refine consistently lowers hallucination rates on MME, POPE and AMBER across four 7B LVLMs without any training.

  4. ReCo: Reminder Composition Mitigates Hallucinations in Vision-Language Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    ReCo, a lightweight DPO-trained linear head that re-injects pooled image embeddings at every step, reduces hallucination on five benchmarks across three VLMs and combines with existing mitigation methods.

  5. Examining Vision Language Models through Multi-dimensional Experiments with Vision and Text Features

    cs.CV 2025-09 conditional novelty 4.0 of 10

    Prompt specificity measurably affects counting accuracy and attention allocation in Qwen2.5-VL and Kimi-VL, and can partially overcome learned visual priors.

Pith tools