Pith. sign in

REVIEW 3 major objections 6 minor 75 references

Wiener Representation Filtering for VLM Hallucination Suppression

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Object hallucination in vision-language models can be suppressed by a closed-form Wiener filter on hidden representations, folded into the weights so inference runs at the same speed.

desk verdict A genuinely useful training-free hallucination filter with an over-stated MMSE-optimality story; the empirics are solid, the theory framing needs a reset, and one diagnostic table looks wrong. read the letter →

arxiv 2608.08167 v1 pith:YF4DXMEM submitted 2026-08-08 cs.CV cs.LG

Ameen Ali , Tamim Zoabi , Lidor Brami , Lior Wolf This is my paper · ORCID
classification cs.CVcs.LG
keywords objecthallucinationvision-languagemodelsWienerfilterrepresentation-spaceeditingMMSEestimationsuppressionlow-rankdistortiontraining-freemitigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that object hallucination in vision-language models is not random noise but a structured, low-rank distortion in the hidden representations of the language backbone, and that removing it is a linear minimum mean-square error problem. Treating each hidden state as a truthful component plus a hallucination-induced distortion, the optimal linear correction is a Wiener operator whose mode-wise gains are given in closed form from the covariances of paired truthful and hallucinated captions. Because the resulting spectral filter is applied once to the feed-forward output projections of selected deeper layers, the edited model generates at the same speed as the original, with no gradients, fine-tuning, or decoding-time overhead. On CHAIR, POPE, and MME, the filtered LLaVA-1.5, MiniGPT-4, mPLUG-Owl2, and Gemma3 hallucinate less while keeping caption fluency roughly unchanged, and the same filter carries over to TempCompass video reasoning and to grounded dialogue in a discrete diffusion language model.

What carries the argument

The load-bearing object is the Wiener spectral filter $F_\alpha = Q\,\mathrm{diag}(\tilde\gamma_1,\dots,\tilde\gamma_d)\,Q^\top$, a spectral operator built from the eigendecomposition of the hallucination covariance $\Sigma_H = Q\Lambda Q^\top$. For each mode $j$, the distortion-to-signal ratio $\nu_j = \lambda_j/\tau_j^2$, with $\tau_j^2 = q_j^\top \Sigma_T q_j$ the truthful variance along that mode, sets the gain $\tilde\gamma_j = (1 + \nu_j)^{-\alpha}$: hallucination-dominated modes are attenuated, truthful modes are left nearly intact, and $\alpha$ sharpens or softens the attenuation without reordering the modes. The filter is calibrated once offline from roughly 3,000 paired truthful/hallucinated LURE captions per model, using only forward passes and empirical second-order statistics, then folded into the FFN output projection $\tilde W_\ell^{\mathrm{out}} = F_\alpha W_\ell^{\mathrm{out}}$ of selected deeper layers, so inference cost is unchanged. A stability lemma, proved with operator-Lipschitz spectral maps, guarantees that small covariance estimation errors induce proportionally bounded changes in the operator.

What would settle it

Re-run the assumption diagnostics of Appendix L on Gemma3 measuring the normalized cross-covariance $\rho_{\mathrm{cross}} = \|\Sigma_{sn}\|_F/\sqrt{\|\Sigma_T\|_F\|\Sigma_H\|_F}$ from the actual paired activations instead of reporting it as zero, then replace $F_\alpha$ with the correlated estimator $A^\star_C = (\Sigma_T + C)(\Sigma_T + \Sigma_H + C + C^\top)^{-1}$ and compare CHAIR and POPE scores; if $A^\star_C$ clearly beats $F_\alpha$, the diagonal mode-wise filter is leaving measurable performance on the table, and if the scores are close, the uncorrelatedness assumption is not load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that hallucination suppression reduces to a closed-form linear filter in the representation space of the language backbone: model each hidden state as $h = s + n$, a superposition of a truthful component $s$ and a hallucination-associated distortion $n$, and assume the two are zero-mean and mutually uncorrelated. Then the covariance of the observed representation decomposes as $\mathrm{Cov}(h) = \Sigma_T + \Sigma_H$, and the linear MMSE estimate of the truthful component is $A^\star = \Sigma_T(\Sigma_T + \Sigma_H)^{-1}$. The paper implements a diagonalized version of this estimator in the eigenbasis of $\Sigma_H$: each spectral mode $j$ is attenuated by the gain $\tilde\gamma_j = (1 + \lambda_j/\tau_j^2)^{-\alpha}$, where $\lambda_j$ is the distortion variance, $\tau_j^2$ is the truthful variance along that mode, and $\alpha$ is a sharpness exponent. The resulting operator $F_\alpha = Q\,\mathrm{diag}(\tilde\gamma_1,\dots,\tilde\gamma_d)\,Q^\top$ is absorbed into the FFN output projection of selected deeper layers, $\tilde W_\ell^{\mathrm{out}} = F_\alpha W_\ell^{\mathrm{out}}$, so the deployed model runs unchanged and at the same speed. The empirical claim is that this weight edit lowers object hallucination on CHAIR, POPE, and MME across LLaVA-1.5, MiniGPT-4, mPLUG-Owl2, and Gemma3, extends to TempCompass video reasoning and FaithDial grounded dialogue, and stays stable because the gains are continuous functions of the estimated covariances.

Load-bearing premise

The filter stands on one statistical assumption: that the hallucination distortion and the truthful part of a hidden state are uncorrelated on average (zero cross-covariance), so the variance of any observed representation is simply the sum of the two variances, and if that cross-covariance is not negligible, the closed-form Wiener gains are no longer optimal and could in principle amplify the hallucination.

Editorial extensions

If this is right

  • Hallucination suppression becomes an offline weight edit: any VLM whose hidden states admit the paired calibration can be corrected before deployment, with zero added inference cost.
  • The filter touches only the language backbone's FFN output projections, so it applies equally to early-fusion, bridging, and mid-fusion VLM designs without architectural changes.
  • Because the edit is a weight transformation rather than a decoding rule, it composes with decoding-time strategies; for mPLUG-Owl2 the paper's best POPE numbers come from the filtered model combined with beam search.
  • The heavily tailed spectrum of $\Sigma_H$ means a handful of dominant modes captures most hallucination-related variance, so a compact spectral signature of a model's hallucination tendency is available from the top few dozen eigendirections.
  • The same representation-space edit improves temporal reasoning on TempCompass and grounded dialogue in a discrete diffusion language model, indicating that the distortion geometry is shared across modalities, tasks, and generation paradigms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Appendix L diagnostics double as a cheap transfer test for any new VLM: collect a few hundred paired captions and measure the additivity error $\delta_{\mathrm{add}}$ and top-16 mode concentration, and if they land near the reported values (about 0.1 and 0.95), the Wiener edit should transfer with little retuning, while large deviations would signal miscalibrated gains.
  • Table 9 reports the normalized cross-covariance $\rho_{\mathrm{cross}}$ as exactly 0 for both paired and random-pair settings, which cannot literally hold for finite high-dimensional samples; the entry is either an idealized or rounded report, or the activations were orthogonalized before the diagnostic ran, and measuring $\rho_{\mathrm{cross}}$ honestly and comparing the correlated estimator $A^\
  • The exact MMSE gain profile ($\alpha = 1$) is not the operating point: the paper's sensitivity analysis shows the best hallucination scores at sharpness values in the tens, so the closed-form optimality is a starting configuration that is then sharpened to compensate for covariance estimation error and distribution shift.
  • Nothing in the derivation is specific to vision: the paired-residual construction requires only truthful and hallucinated renderings of the same input, so the same calibration could be applied to language-only, speech, or retrieval-augmented backbones, and Appendix J's cross-domain results (filter learned on HaluEval, evaluated on CHAIR) already point toward dataset-transferable distortion geometr
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a training-free, post-hoc representation-space filter for suppressing object hallucinations in vision-language models. Hidden states are modeled as the sum of a truthful component and a hallucination-associated distortion, and the method derives a Wiener-type estimator from paired truthful/hallucinated calibration data. The resulting mode-wise spectral filter is absorbed into the feed-forward output projections of selected deep layers, so inference-time cost is unchanged. Experiments on LLaVA-1.5, MiniGPT-4, mPLUG-Owl2, Gemma3, and a diffusion-based dialogue model report consistent hallucination reductions on CHAIR, POPE, MME, TempCompass, and FaithDial, including cross-domain transfer results.

Significance. If the empirical findings hold, the method is practically valuable: it is training-free, requires no inference-time overhead, and generalizes across model families and modalities (image, video, and grounded dialogue). The consistent gains across benchmarks and the cross-domain transfer experiments are strengths, as is the clear presentation of the offline weight-edit procedure. However, the paper's central theoretical claim—that the deployed filter is the closed-form MMSE-optimal Wiener estimator—is not supported by the implementation or by the presented diagnostics. The actual method involves a tuned sharpness exponent and layer range, and the validation of the key uncorrelatedness assumption is implausible as reported. The contribution is better framed as a heuristic spectral shrinkage method with strong empirical support than as a provably optimal estimator. With appropriate revisions of the theoretical claims and diagnostics, the paper could be a useful contribution to hallucination mitigation.

major comments (3)
  1. [Section 3.3, Eqs. (9)-(12)] The deployed filter F_alpha in Eq. (12) is not the MMSE-optimal operator A* in Eq. (9) unless Sigma_T and Sigma_H are jointly diagonalizable, which the paper does not establish; the text itself concedes this at the end of Section 3.3. Since the sharpness exponent alpha and the edited layer range are selected per model on a validation set (Section 4, Implementation Details), the method actually evaluated is a tuned spectral shrinkage, not the closed-form Wiener solution highlighted in the abstract. The optimality language in the abstract and introduction should be removed or substantially qualified, and the authors should quantify, on the calibration set, the distance ||F_alpha - A*|| in the actual operating regime (for example, for the reported alpha values) so that the approximation error is explicit.
  2. [Appendix L, Table 9] The normalized cross-covariance rho_cross is reported as exactly 0 in both the paired condition and the random-pair control. For finite-sample high-dimensional activations, an exact zero in both settings is implausible and strongly suggests the reported quantity is not the sample cross-covariance defined in the appendix, or that an unintended orthogonalization was applied. Because the derivation of Eq. (9) rests on Assumption 1 (Eq. (7)) and Section 3.3 explicitly points to Appendix L as the empirical validation of that assumption, this diagnostic is load-bearing. The authors must recompute rho_cross according to its definition, report the nonzero values (with reasonable precision), and explain why the random-pair control gives the same number as the paired condition. Without a credible cross-covariance estimate, the MMSE interpretation is unsupported.
  3. [Section 3.2 (Eq. 4) vs. Algorithm 1 (line 11)] The covariances Sigma_T and Sigma_H are estimated from sequence-averaged representations x_i^+ and x_i^- as defined in Eq. (4), and the MMSE derivation concerns those averaged vectors. However, the correction is applied token-wise through the FFN output matrix, acting on every token-level hidden state at inference (Algorithm 1, line 11). The paper does not justify that per-token covariance statistics match the sequence-averaged ones; if they differ, the MMSE optimality (even under Assumption 1) does not transfer to the actual edited object. The authors should either calibrate on per-token features or provide evidence that the per-token and sequence-average distributions are comparable in the layers where the filter is applied.
minor comments (6)
  1. [Section 4.2, Table 4] The table rows for CHAIRS and CHAIRI contain concatenated numbers (for example, “14.013.0” and “5.04.9”), which makes the comparison between the “Our (layers 24–32)” column and the others ambiguous; please format the values as separate columns.
  2. [Section 4.1, Table 3 and text] The text states that FaithCritic reports a reduction from 92% to 74% and that a Claude Sonnet 4.6 judge shows a reduction from 93% to 50%, but Table 3's FaithCritic column shows 85.92 to 75.39. These numbers need to be reconciled; report the LLM-judge results in the table or clarify which metric corresponds to which set of numbers.
  3. [Section 3.3, Lemma 1] Lemma 1 restates the definition of an operator-Lipschitz function; the proof is correct but the lemma itself adds little beyond that definition. Consider stating the stability property directly for the specific spectral gains used here, or moving the lemma to the appendix and keeping only the non-expansiveness observation in the main text.
  4. [Section 4.1, Table 1] For mPLUG-Owl2, the BLEU score under Ours (0.142) is notably lower than several baselines (e.g., 0.151–0.154). The text says the method maintains “competitive” caption quality, but the drop should be addressed quantitatively, especially since caption fluency is one of the stated desiderata.
  5. [Appendix E] The sensitivity of alpha is only shown for LLaVA-1.5; since the reported alpha values differ across models (10, 20, 60), please show the analogous sensitivity curves for MiniGPT-4 and mPLUG-Owl2 (or state that the behavior is similar and give a representative plot).
  6. [Section 3.2, Eq. (6)] Sigma_H is defined as the sample second moment of the residuals d_i without explicit mean centering. Since the text later discusses centering the residual distribution, please clarify whether the mean is subtracted in practice and whether that affects the definition of Sigma_H in Eq. (6).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Wiener-filter derivation is self-contained, and the tuned sharpness/layer selection and the questionable Appendix L diagnostic are correctness concerns, not circular reductions.

full rationale

The derivation chain is not circular. Section 3.3 states an explicit additive model h = s + n, an explicit Assumption 1 (E[sn^T] = 0), and then derives the LMMSE operator A* = Sigma_T (Sigma_T + Sigma_H)^{-1} from standard LMMSE theory. The covariance matrices Sigma_T and Sigma_H are estimated from paired truthful/hallucinated representations in Eq. (6), not from the evaluation benchmarks, so the closed-form Wiener gains are not defined in terms of CHAIR/POPE/MME outcomes. The mode-wise filter F_alpha in Eq. (12) is admittedly an approximation of the full MMSE operator unless Sigma_T and Sigma_H are jointly diagonalizable, and the sharpness alpha and layer range are selected on a held-out CHAIR validation set (Section 4, Implementation Details). That is hyperparameter selection on a small disjoint subset, not a fitted input renamed as a prediction, and it does not make the subsequent CHAIR numbers statistically forced. The self-citation to Ali et al. (2024) is historical and non-load-bearing. The main non-circular red flag is Appendix L/Table 9, which reports normalized cross-covariance rho_cross = 0 for both paired and random-pair settings; for finite high-dimensional samples this is implausible and weakens the empirical support for Assumption 1, but it is a reporting/validity issue, not evidence that the derivation reduces to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The decomposition into truthful and hallucination components is a modeling device over existing representations, not a new physical or architectural entity. No new parameters, forces, or dimensions are introduced. The free parameters are α and the edited layer range, both tuned on validation data.

free parameters (2)
  • sharpness exponent α = LLaVA-1.5: 60.0, mPLUG-Owl2: 20.0, MiniGPT-4: 10.0, Gemma3: 10.0, TempCompass: 40.0
    Controls how strongly the Wiener gains are expressed; tuned on a 100-image validation set (Section 4, Implementation Details). The derived MMSE-optimal filter corresponds to α=1, so the deployed filter is not the derived one.
  • edited layer range L = LLaVA-1.5 layers 20-32, MiniGPT-4 layers 24-32, mPLUG-Owl2 layers 20-32, Gemma3 layers 12-26 or 12-34
    Selected per model based on validation-set CHAIR performance, as reported in Section 4 and Appendices G/I. Layer choice is a fitted hyperparameter, not derived from the covariance analysis.
assumptions (4)
  • domain assumption Assumption 1: truthful component s and hallucination distortion n are zero-mean and mutually uncorrelated (E[s n^T]=0)
    Stated in Section 3.3, Eq. (7). Empirically validated in Appendix L, but the reported cross-covariance of exactly 0 for both paired and random-pair settings is implausible and the approximation weakens the LMMSE optimality.
  • domain assumption Additive decomposition h = s + n in representation space
    Introduced in Section 3.3. The paper acknowledges it is a paired-residual construction, not a literal claim about transformer internals.
  • domain assumption The filter is approximated as diagonal in the eigenbasis of Σ_H, relying on joint diagonalizability of Σ_T and Σ_H
    Section 3.3 states F_α is exact only when Σ_T and Σ_H are jointly diagonalizable; no evidence of joint diagonalizability is provided.
  • standard math Operator-Lipschitz spectral map stability (Lemma 1) applies to the actual covariance-to-filter map
    Lemma 1 in Section 3.3 is a standard result for fixed spectral functions, but the implemented gains depend on Σ_T and eigenvectors of Σ_H, so the Lipschitz bound does not directly apply.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Wiener Representation Filtering for VLM Hallucination Suppression." pith.science (2026). https://pith.science/paper/YF4DXMEM

@misc{pith2026260808167,
  author       = {Pith},
  title        = {Pith review of: Wiener Representation Filtering for VLM Hallucination Suppression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YF4DXMEM}},
  note         = {Machine review of arXiv:2608.08167}
}
read the original abstract

Vision-language models (VLMs) excel at open-ended captioning and visual QA but often describe objects, attributes, or relations absent from the image, a phenomenon known as object hallucination. We propose a {training-free, post-hoc representation editing technique} that operates in the representation space of the language backbone. The method performs a lightweight, one-time offline calibration on a modest paired dataset to estimate the required covariance structures, using only forward passes and empirical second-order statistics with no gradient updates or fine-tuning, after which the correction is absorbed directly into the model's existing weights. By modeling hidden states as a superposition of truthful and hallucination-associated components, we derive a Wiener-type estimator whose optimal gains are given in closed form from the covariances of paired truthful and hallucinated representations. An eigendecomposition yields mode-wise attenuation that respects a stability criterion, i.e., the filter responds continuously to estimation noise. The correction is applied once to the feed-forward output projections of selected deeper layers, at inference time, the model runs unchanged and at the same speed. Experiments on LLaVA-1.5, MiniGPT-4, Gemma3, and mPLUG-Owl2 demonstrate consistent reductions in object hallucination on CHAIR, POPE, and MME while maintaining caption fluency and overall response quality. We further demonstrate the generality of our approach on the TempCompass video understanding benchmark and on discrete diffusion language models for grounded dialogue, showing that representation filtering reduces hallucinations even in temporal video reasoning and multi-step, sequence-wide denoising settings.

Figures

Figures reproduced from arXiv: 2608.08167 by the authors.

Figure 1
Figure 1. (a) UMAP visualization of MiniGPT4 and mPLUG-Owl2 hidden activations for [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A qualitative case study comparing model-generated descriptions for a complex [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Dolan–More performance profiles on 10 MME subsets for mPLUG-Owl2 (left) and ´ LLaVA-1.5 (right). Additional experiments evaluating the method on Gemma3-4B-it (Appendix G), per￾subset MME scores (Appendix F), the TempCompass video understanding benchmark (Appendix I), cross-domain transfer (Appendix J), and a faithfulness-versus-informativeness analysis (Appendix H) are provided in the supplementary material. FaithDi… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Hallucination spectrum (eigenvalues of ΣH = QΛQ⊤) for three vision–language models. The curves show a small set of high-variance “spikes” followed by a long decaying tail, indicating that hallucination behavior is concentrated in a low-dimensional set of dominant eigen…
Figure 5
Figure 5. Figure 5: Sensitivity of the Wiener sharpness parameter [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: CHAIR captioning examples showing how Nullu and greedy decoding introduce [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 39 canonical work pages

  1. [1]

    Chuang, Yung-Sung and Xie, Yujia and Luo, Hongyin and Kim, Yoon and Glass, James and He, Pengcheng , journal=

    work page

  2. [2]

    Zhao, Linxi and others , journal=

    work page

  3. [4]

    arXiv preprint , year=

    Combating Hallucinations in Multimodal Large Language Models , author=. arXiv preprint , year=

    work page

  4. [5]

    MICCAI , year=

    Vision Amplified: Visual Question Answering with Enhanced Medical Images , author=. MICCAI , year=

    work page

  5. [6]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =

  6. [7]

    Journal of Foo , volume = 14, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe and FirstName Gamow , title =. Journal of Foo , volume = 14, number = 1, pages =

  7. [8]

    FirstName Alpher and FirstName Gamow , title =

  8. [9]

    Computer Vision -- ECCV 2022 , year =

    2022

Show all 75 references
  1. [10]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  2. [11]

    arXiv preprint arXiv:2402.00253 , year=

    A survey on hallucination in large vision-language models , author=. arXiv preprint arXiv:2402.00253 , year=

    arXiv

  3. [12]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    arXiv

  4. [13]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

    Object hallucination in image captioning , author=. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

    2018

  5. [14]

    2023 , eprint=

    mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration , author=. 2023 , eprint=

    2023

  6. [15]

    2023 , eprint=

    mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality , author=. 2023 , eprint=

    2023

  7. [16]

    arXiv preprint arXiv:2310.09478 , year=

    Minigpt-v2: large language model as a unified interface for vision-language multi-task learning , author=. arXiv preprint arXiv:2310.09478 , year=

    arXiv

  8. [17]

    arXiv preprint arXiv:2304.10592 , year=

    Minigpt-4: Enhancing vision-language understanding with advanced large language models , author=. arXiv preprint arXiv:2304.10592 , year=

    arXiv

  9. [18]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Visual hallucinations of multi-modal large language models , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  10. [19]

    Object Hallucination in Image Captioning

    Rohrbach, Anna and Hendricks, Lisa Anne and Burns, Kaylee and Darrell, Trevor and Saenko, Kate. Object Hallucination in Image Captioning. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

    2018

  11. [20]

    arXiv preprint arXiv:2404.18930 , year=

    Hallucination of multimodal large language models: A survey , author=. arXiv preprint arXiv:2404.18930 , year=

    arXiv

  12. [21]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Improved baselines with visual instruction tuning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  13. [22]

    arXiv preprint arXiv:2306.13394 , year=

    Mme: A comprehensive evaluation benchmark for multimodal large language models , author=. arXiv preprint arXiv:2306.13394 , year=

    arXiv

  14. [23]

    Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models

    Zhong, Weihong and Feng, Xiaocheng and Zhao, Liang and Li, Qiming and Huang, Lei and Gu, Yuxuan and Ma, Weitao and Xu, Yuan and Qin, Bing. Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models. Proceedings of the 62nd Annual Meet...

    2024

  15. [24]

    and Reddy, Siva

    Dziri, Nouha and Kamalloo, Ehsan and Milton, Sivan and Zaiane, Osmar and Yu, Mo and Ponti, Edoardo M. and Reddy, Siva. F aith D ial: A Faithful Benchmark for Information-Seeking Dialogue. Transactions of the Association for Computational Linguistics. 2022

    2022

  16. [25]

    arXiv preprint arXiv:2508.15487 , year=

    Dream 7B: Diffusion Large Language Models , author=. arXiv preprint arXiv:2508.15487 , year=

    arXiv

  17. [26]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    Evaluating object hallucination in large vision-language models , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  18. [27]

    Visual Hallucinations of Multi-modal Large Language Models

    Huang, Wen and Liu, Hongbin and Guo, Minxin and Gong, Neil. Visual Hallucinations of Multi-modal Large Language Models. Findings of the Association for Computational Linguistics: ACL 2024

    2024

  19. [28]

    Beam Search Strategies for Neural Machine Translation

    Freitag, Markus and Al-Onaizan, Yaser. Beam Search Strategies for Neural Machine Translation. Proceedings of the First Workshop on Neural Machine Translation. 2017

    2017

  20. [29]

    Science China Information Sciences , volume=

    Woodpecker: Hallucination correction for multimodal large language models , author=. Science China Information Sciences , volume=. 2024 , publisher=

    2024

  21. [30]

    Halc: Object hallucination reduction via adaptive focal-contrast decoding , author=

  22. [31]

    Chen, Zhaorun and others , journal=

  23. [32]

    European conference on computer vision , pages=

    Microsoft coco: Common objects in context , author=. European conference on computer vision , pages=. 2014 , organization=

    2014

  24. [33]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Mitigating object hallucinations in large vision-language models through visual contrastive decoding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  25. [34]

    Proceedings of the First Workshop on Neural Machine Translation

    Beam search strategies for neural machine translation , author=. Proceedings of the First Workshop on Neural Machine Translation

  26. [35]

    1999 , publisher=

    Discrete-time signal processing , author=. 1999 , publisher=

    1999

  27. [36]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , year=

    Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , year=

  28. [37]

    European conference on computer vision , pages=

    A-okvqa: A benchmark for visual question answering using world knowledge , author=. European conference on computer vision , pages=. 2022 , organization=

    2022

  29. [38]

    Evaluating Object Hallucination in Large Vision-Language Models

    Li, Yifan and Du, Yifan and Zhou, Kun and Wang, Jinpeng and Zhao, Xin and Wen, Ji-Rong. Evaluating Object Hallucination in Large Vision-Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023

    2023

  30. [39]

    Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation , author=. 2024

    2024

  31. [40]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Looking beyond text: Reducing language bias in large vision-language models via multimodal dual-attention and soft-image guidance , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  32. [41]

    arXiv preprint arXiv:2404.05046 , year=

    Fgaif: Aligning large vision-language models with fine-grained ai feedback , author=. arXiv preprint arXiv:2404.05046 , year=

    arXiv

  33. [42]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Aligning large multimodal models with factually augmented rlhf , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  34. [43]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    arXiv

  35. [44]

    Advances in neural information processing systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in neural information processing systems , volume=

  36. [45]

    Proceedings of the 32nd ACM International Conference on Multimedia , publisher =

    Zhu, Jiawei and Liu, Yishu and Zhu, Huanjia and Lin, Hui and Jiang, Yuncheng and Zhang, Zheng and Chen, Bingzhi , title =. Proceedings of the 32nd ACM International Conference on Multimedia , publisher =

  37. [46]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

    Let there be a clock on the beach: Reducing object hallucination in image captioning , author=. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

  38. [47]

    arXiv e-prints , year=

    HallE-Switch: Rethinking and Controlling Object Existence Hallucinations in Large Vision-Language Models for Detailed Caption , author=. arXiv e-prints , year=

  39. [48]

    arXiv preprint, arXiv:2204.10757 , year=

    FaithDial: A Faithful Benchmark for Information-Seeking Dialogue , author=. arXiv preprint, arXiv:2204.10757 , year=

    arXiv

  40. [49]

    Dream 7b: Diffusion large language models, 2025 , author=

    2025

  41. [50]

    arXiv e-prints , year=

    The llama 3 herd of models , author=. arXiv e-prints , year=

  42. [51]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond , author=

  43. [52]

    Advances in neural information processing systems , volume=

    Instructblip: Towards general-purpose vision-language models with instruction tuning , author=. Advances in neural information processing systems , volume=

  44. [53]

    The Twelfth International Conference on Learning Representations , year=

    LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention , author=. The Twelfth International Conference on Learning Representations , year=

  45. [54]

    2023 , journal=

    Llama-adapter v2: Parameter-efficient visual instruction model , author=. 2023 , journal=

    2023

  46. [55]

    See https://vicuna

    Vicuna: An open-source chatbot impressing gpt-4 with 90\ author=. See https://vicuna. lmsys. org (accessed 14 April 2023) , volume=

    2023

  47. [56]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Discovering and mitigating visual biases through keyword explanation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  48. [57]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  49. [58]

    arXiv preprint arXiv:2310.00754 , year=

    Analyzing and mitigating object hallucination in large vision-language models , author=. arXiv preprint arXiv:2310.00754 , year=

    arXiv

  50. [59]

    OPT ,year=

    Evolution of the Spectral Dimension of Transformer Activations , author=. OPT ,year=

  51. [60]

    2025 , eprint=

    Small Singular Values Matter: A Random Matrix Analysis of Transformer Models , author=. 2025 , eprint=

    2025

  52. [61]

    arXiv preprint arXiv:2410.17770 , year=

    Small Singular Values Matter: A Random Matrix Analysis of Transformer Models , author=. arXiv preprint arXiv:2410.17770 , year=

  53. [62]

    2018 , eprint=

    Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning , author=. 2018 , eprint=

    2018

  54. [63]

    Journal of Machine Learning Research , volume=

    Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning , author=. Journal of Machine Learning Research , volume=

  55. [64]

    2022 , eprint=

    Understanding Dimensional Collapse in Contrastive Self-supervised Learning , author=. 2022 , eprint=

    2022

  56. [65]

    ICLR , year=

    Understanding Dimensional Collapse in Contrastive Self-supervised Learning , author=. ICLR , year=

  57. [66]

    2026 , eprint=

    SIGMA: Scalable Spectral Insights for LLM Model Collapse , author=. 2026 , eprint=

    2026

  58. [67]

    arXiv preprint arXiv:2601.03385 , year=

    SIGMA: Scalable Spectral Insights for LLM Collapse , author=. arXiv preprint arXiv:2601.03385 , year=

  59. [68]

    2025 , eprint=

    Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers , author=. 2025 , eprint=

    2025

  60. [69]

    arXiv preprint arXiv:2410.07799 , year=

    Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers , author=. arXiv preprint arXiv:2410.07799 , year=

    arXiv

  61. [70]

    2024 , eprint=

    Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance , author=. 2024 , eprint=

    2024

  62. [71]

    Mitigating object hallucination in large

    Zhao, Linxi and others , journal=. Mitigating object hallucination in large

  63. [72]

    Suppressing

    Ali, Ameen and Zoabi, Tamim and Brami, Lidor and Wolf, Lior , journal=. Suppressing

  64. [73]

    Exposing Hallucinations To Suppress Them:

    Deng, Jinhao and others , journal=. Exposing Hallucinations To Suppress Them:

  65. [74]

    Liu, Yuanxin and others , journal=

  66. [75]

    Lin, Stephanie and Hilton, Jacob and Evans, Owain , journal=

  67. [76]

    Li, Junyi and others , journal=

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.