Pith. sign in

REVIEW 6 cited by

Mutual Information Alleviates Hallucinations in Abstractive Summarization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2210.13210 v2 pith:GJUSSXW2 submitted 2022-10-24 cs.CL

Liam van der Poel , Ryan Cotterell , Clara Meister This is my paper · ORCID
classification cs.CL
keywords modelsprobabilitydecodinghallucinationssourceabstractivecontentduring
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Despite significant progress in the quality of language generated from abstractive summarization models, these models still exhibit the tendency to hallucinate, i.e., output content not supported by the source document. A number of works have tried to fix--or at least uncover the source of--the problem with limited success. In this paper, we identify a simple criterion under which models are significantly more likely to assign more probability to hallucinated content during generation: high model uncertainty. This finding offers a potential explanation for hallucinations: models default to favoring text with high marginal probability, i.e., high-frequency occurrences in the training set, when uncertain about a continuation. It also motivates possible routes for real-time intervention during decoding to prevent such hallucinations. We propose a decoding strategy that switches to optimizing for pointwise mutual information of the source and target token--rather than purely the probability of the target token--when the model exhibits uncertainty. Experiments on the XSum dataset show that our method decreases the probability of hallucinated tokens while maintaining the Rouge and BertS scores of top-performing decoding strategies.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  2. A Rate-Distortion Framework for Summarization

    cs.IT 2025-01 conditional novelty 6.0 of 10

    The summarizer rate-distortion function lower-bounds any summary's compression rate at a given distortion, and a Gaussian embedding approximation makes it computable on real datasets.

  3. ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A self-attention model over hidden states of a vision-language model identifies hallucinated findings in AI-generated radiology reports with AUROC 0.8751, outperforming prior detectors.

  4. Beyond ROUGE: N-Gram Subspace Features for LLM Hallucination Detection

    cs.CL 2025-09 reject novelty 4.0 of 10

    Singular values of label-grouped n-gram frequency tensors are used as MLP features for hallucination detection, with reported gains on HaluEval that rely on label-aware grouping.

  5. MASS: Overcoming Language Bias in Image-Text Matching

    cs.CV 2025-01 conditional novelty 4.0 of 10

    MASS re-scores image-text pairs with pointwise mutual information, estimated by comparing caption likelihood on the real image versus a black image, and reduces language bias on color, counting, gender, and compositio...

  6. Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents

    cs.AI 2024-11 conditional novelty 4.0 of 10

    A survey proposing a source-and-impact taxonomy (input, model, combined; security, privacy, ethics) for threats to LLM-based agents, with feature analysis and four case studies.

Pith tools