Pith. sign in

REVIEW 5 cited by

ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.14766 v3 pith:GEJ4EWA4 submitted 2025-06-17 cs.CV cs.CL

Yujun Wang , Aniri , Jinhe Bi , Soeren Pirk , Yunpu Ma This is my paper
classification cs.CVcs.CL
keywords decodingascdattentioncontrastivesteeringacrossattention-steerablehallucination
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal large language models (MLLMs) frequently hallucinate by over-committing to spurious visual cues. Prior remedies-Visual and Instruction Contrastive Decoding (VCD, ICD)-mitigate this issue, yet the mechanism remains opaque. We first empirically show that their improvements systematically coincide with redistributions of cross-modal attention. Building on this insight, we propose Attention-Steerable Contrastive Decoding (ASCD), which directly steers the attention scores during decoding. ASCD combines (i) positive steering, which amplifies automatically mined text-centric heads-stable within a model and robust across domains-with (ii) negative steering, which dampens on-the-fly identified critical visual tokens. The method incurs negligible runtime and memory overhead and requires no additional training. Across five MLLM backbones and three decoding schemes, ASCD reduces hallucination on POPE, CHAIR, and MMHal-Bench by up to 38.2 percent while improving accuracy on standard VQA benchmarks, including MMMU, MM-VET, ScienceQA, TextVQA, and GQA. These results position attention steering as a simple, model-agnostic, and principled route to safer, more faithful multimodal generation.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dual Latent Memory for Visual Multi-agent System

    cs.AI 2026-01 conditional novelty 6.0 of 10

    L2-VMAS replaces text-based inter-agent communication in visual multi-agent systems with decoupled latent perception and thinking memories, improving accuracy by 2.7–5.4% and cutting token use by 21.3–44.8%.

  2. TECP: Token-Entropy Conformal Prediction for LLMs

    cs.CL 2025-08 reject novelty 4.0 of 10

    TECP applies split conformal prediction with token-entropy nonconformity scores to LLM question answering and reports reliable coverage, but its implementation requires the token probabilities it claims to avoid.

  3. FADE: Adversarial Concept Erasure in Flow Models

    cs.CV 2025-07 reject novelty 4.0 of 10

    FADE combines adversarial training with trajectory preservation to erase concepts from diffusion models, reporting state-of-the-art erasure on Stable Diffusion benchmarks, but the evidence is incomplete and the theore...

  4. Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guarantees

    cs.CL 2025-08 conditional novelty 3.0 of 10

    Repeatedly sampling an LLM and using the entropy of answer frequencies yields conformal prediction sets for multiple-choice questions with empirical miscoverage near the target, and AUROC comparable to logit-based scores.

  5. Conformal P-Value in Multiple-Choice Question Answering Tasks with Provable Risk Control

    cs.CL 2025-08 reject novelty 2.0 of 10

    A p-value reformulation of split conformal prediction for LLM multiple-choice QA achieves nominal miscoverage control on MMLU and MMLU-Pro.

Pith tools