Pith. sign in

REVIEW 5 major objections 7 minor 50 references

BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes a normalizing-flow bijection from a model response to a learned distribution of non-hallucinated responses and uses the resulting likelihood as a fine-tuning loss to reduce object hallucination in LLaVA v1.5 7B.

desk verdict The empirical gains on CHAIR/POPE are specific and plausible, but the paper's central bijection over token sequences is unproven and the representation is lossy, so the mechanism as written does not hold up. read the letter →

arxiv 2505.24649 v1 pith:5VHFAZEH submitted 2025-05-30 cs.CV cs.AI

Huu-Thien Tran , Thanh-Dat Truong , Khoa Luu This is my paper · ORCID
classification cs.CVcs.AI
keywords hallucinationmitigationlargevision-languagemodelsnormalizingflowsbijectivemappinginstructionfine-tuningCHAIRbenchmarkPOPEobject
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large vision-language models often name objects that are not in the image. This paper claims that such hallucinated responses can be detected and suppressed by a learned bijective map: a response is non-hallucinated exactly when a normalizing-flow transformation sends it into the distribution of ground-truth responses. The map's negative log-likelihood becomes an extra loss added to ordinary instruction fine-tuning, and the paper reports that this cuts hallucinated captions from 42.0 percent to 34.4 percent on CHAIRS and from 12.2 percent to 9.6 percent on CHAIRI. The same fine-tuned model reaches 85.06 average F1 on the POPE object-probing benchmark. If the bijective metric is as informative as claimed, it gives practitioners a single scalar that estimates how far any generated sentence is from a truthful answer.

What carries the argument

The load-bearing object is the normalizing-flow bijection $F(\hat{y}; \theta_F)$, implemented with RealNVP blocks and trained on a reference dataset of ground-truth responses encoded as $[0,1]^{1\times H\times W}$ tensors via one-hot encoding followed by bilinear downsampling to $128\times 128$. The change-of-variables formula (Eqn. 7) turns $F$ into a density estimator $q(\hat{y})$, and the bijective metric $L_B = \mathbb{E}_{\hat{y}\sim q}[ -\log q(\hat{y})]$ (Eqn. 9) is the negative log-likelihood used as a complementary fine-tuning objective. The frozen flow connects text space to the reference distribution: it assigns low $L_B$ to responses that map near $\pi_{\text{ref}}$ and high $L_B$ to responses far from it.

What would settle it

Take a fixed set of paired truthful and hallucinated captions for the same images, pass them through the trained flow, and rank them by $L_B$; the central claim predicts that truthful captions consistently score lower, so a ranking that separates the two sets no better than chance would falsify the metric. A complementary check is to apply $F^{-1}$ to samples from $\pi_{\text{ref}}$ and verify that the recovered points decode back to the original token sequences, since the bilinear downsampling would make any information loss visible.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is Eqn. 6: a model-produced response $\hat{y}$ is non-hallucinated if and only if there exists a bijective map $F$ with $F(\hat{y})$ distributed as $\pi_{\text{ref}}(y)$, the reference distribution of ground-truth responses. The paper makes this operational with a RealNVP flow trained by maximum likelihood on one-hot encoded, downsampled responses, giving a computable log-likelihood through the change-of-variables formula. That negative log-likelihood, $L_B$, is read as the degree of hallucination and added to the cross-entropy loss of LLaVA v1.5 7B during instruction fine-tuning. Empirically, the paper reports CHAIRS 34.4 percent, CHAIRI 9.6 percent, and a POPE average F1 of 85.06 percent, with the adversarial split F1 at 83.07 percent.

Load-bearing premise

The method assumes that collapsing a discrete token sequence into a fixed $128\times 128$ grid by one-hot encoding and bilinear interpolation leaves enough information for an invertible flow to map the result onto the distribution of non-hallucinated responses, with distance from that distribution measuring how hallucinated the response is.

Editorial extensions

If this is right

  • Adding the bijective loss $L_B$ to standard cross-entropy during instruction fine-tuning lowers CHAIRS to 34.4 percent and CHAIRI to 9.6 percent, cutting hallucinated captions and hallucinated objects by 7.6 and 2.6 points relative to prior decoding-based mitigators.
  • On POPE, the fine-tuned model reaches an average F1 of 85.06 percent, with the largest gains on the popular split (85.73 percent accuracy) and the adversarial split (83.07 percent F1).
  • The frozen bijection model can act as a reusable instrument: once trained on the reference distribution, it accompanies decoding without further adaptation and can be combined with other decoding strategies such as visual-token weighting.
  • Because longer responses give the flow more context for locating the response relative to $\pi_{\text{ref}}$, the paper expects the benefit to concentrate in long-form caption generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors' own limitation paragraph notes that every experiment uses LLaVA v1.5 7B, so whether the gains transfer to other vision-language models is untested; treating the method as architecture-independent is an extrapolation.
  • The $128\times 128$ grid is obtained by bilinearly interpolating over the vocabulary dimension of one-hot vectors, which has no natural spatial order; replacing that encoding with a permutation-invariant one is a testable way to see whether the hypothesized bijection is really about text semantics.
  • If $L_B$ is a reliable hallucination score, it could be deployed as a post-hoc ranker over candidate captions at inference time, without any fine-tuning, to select the most truthful output.
  • Since $\pi_{\text{ref}}$ is built only from ground-truth responses, truthful paraphrases that are rare in the instruction-tuning data may be scored as hallucinated; enlarging the reference set would change measured severity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes BIMA, a normalizing-flow-based method for predicting and mitigating hallucination in large vision-language models (LVLMs). The approach defines a reference distribution pi_ref of non-hallucinated responses, learns a bijective mapping F from model-generated responses to this distribution via a RealNVP flow, and uses the flow's negative log-likelihood as a 'bijective metric' L_B that is added to the instruction fine-tuning loss of LLaVA v1.5 7B. The authors report substantial reductions on CHAIR (CHAIRS 34.4%, CHAIRI 9.6%) and competitive or improved scores on POPE (e.g., adversarial F1 83.07%) compared to decoding-time mitigation methods.

Significance. If the proposed bijective mapping were well-defined and the loss L_B were computable and correctly derived, the paper would offer a novel flow-based perspective on hallucination quantification and mitigation in LVLMs. The empirical results are suggestive, especially the CHAIR improvements. However, the central theoretical construction is not sound as presented: the bijection is not defined on actual text responses, the likelihood objective in Eq. (8)-(9) is internally inconsistent, and the fine-tuning gradient with respect to the LVLM parameters is not specified. The lack of a controlled fine-tuning baseline also prevents attribution of the empirical gains to the proposed mechanism. These issues are load-bearing rather than cosmetic.

major comments (5)
  1. [Sec. 3.2-3.3, Eq. (6)-(7)] The bijective mapping F is not defined on the space of tokenized text responses. The input to the flow is obtained by one-hot encoding sequences with C=2048 and V=32002, then applying bilinear interpolation to a 128x128 grid (Sec. 3.3). This operation is not information-preserving: one-hot dimensions are unordered categorical indicators, so interpolation mixes unrelated vocabulary dimensions and collapses sequence-order information; many distinct token sequences map to the same or nearby grid points. No inverse map from the grid back to text is specified, and RealNVP invertibility on the continuous 128x128 space does not imply invertibility on the discrete pre-image. Consequently, the change-of-variables formula in Eq. (7) does not define a probability density over text responses, and L_B is not a valid measure of hallucination degree.
  2. [Eq. (8)-(9)] The maximum-likelihood objective for learning F is stated as minimizing E_{ŷ~q(ŷ)}[-log q(ŷ)], but q(ŷ) is the density being learned, making the objective circular and undefined. The correct objective would be, for example, maximum likelihood on the reference responses, E_{y~π_ref}[-log q_F(y)], where q_F is the flow-induced density. Additionally, Eq. (9) writes log π_ref(y) without the transformation y=F(ŷ), which is inconsistent with the change-of-variables formula; the expectation in Eq. (9) is over q(ŷ), not over the training data, so L_B as written is not an empirical loss that can be minimized during fine-tuning.
  3. [Sec. 3.3, Eq. (10)] The paper does not specify how the gradient of L_B with respect to the LVLM parameters is computed. The flow input is derived from a discrete token sequence via non-differentiable tokenization, one-hot encoding, and bilinear interpolation; the response ŷ is itself a discrete sample from the LVLM. Without a differentiable surrogate (e.g., Gumbel-softmax) or a gradient estimator, the loss in Eq. (10) cannot be optimized as written. It is also unclear whether ŷ in Eq. (10) is the ground-truth response y or a model-generated sample; the text in Sec. 3.3 says 'the LVLM must generate a response that is then compared with the reference response distribution,' suggesting generation, but the expectation is written over the dataset D.
  4. [Sec. 3.2, Sec. 4.2] The reference distribution π_ref is fit to the same instruction-tuning dataset (llava v1.5 mix665k) used to fine-tune the LVLM. Since hallucination degree is defined as negative log-likelihood under this fitted flow, the method equates hallucination with deviation from the training distribution. This is circular: the training data is assumed to be non-hallucinated (Remark 1), and L_B simply penalizes divergence from the training distribution, which cross-entropy already does. The claimed mechanism of a 'reference distribution of non-hallucinated responses' is not independently validated; the flow can only memorize the training data's statistics, including any hallucinations it contains.
  5. [Sec. 4.4, Tables 1-2] The empirical comparison is not controlled. BIMA is a fine-tuning method trained on the same 665k instruction-tuning dataset as the baseline LLaVA v1.5, while the compared methods (OPERA, ICD, VCD, SID, ProjectAway) are decoding-time interventions applied to a fixed model. No control experiment is reported, such as fine-tuning LLaVA v1.5 for the same number of steps with the same data and optimization settings but without L_B. Without such a control, the observed CHAIR and POPE improvements cannot be attributed to the proposed bijective loss rather than to additional fine-tuning, different training dynamics, or other artifacts. Error bars or multiple runs are also not provided.
minor comments (7)
  1. [Abstract] The abstract states BIMA reduces CHAIRS and CHAIRI by 7.6% and 2.6%, respectively, but does not specify the comparison baseline; the paper should state that these are relative to SID (or whichever method) and whether these are absolute percentage-point reductions.
  2. [Sec. 3.2, Remark 1] The symbol F is used both for the LVLM in Remark 1 and for the bijective mapping later in the same section; this is confusing and should use distinct notation.
  3. [Sec. 3.3] The description of the input representation mentions C=2048 and V=32002, but it is not stated how variable-length sequences are padded or truncated, nor how the one-hot tensor is reshaped for bilinear interpolation (e.g., whether the axes are sequence and vocab). This information is necessary for reproducibility.
  4. [Sec. 3.3] The downsampling from 1x2048x32002 to 1x128x128 loses a factor of roughly 2048*32002/(128*128) ~ 4000 in dimensionality; the paper should discuss whether any information about the original token sequence can survive such aggressive interpolation on one-hot vectors.
  5. [Eq. (2)] The objective in Eq. (2) is stated as negative log-likelihood, but later the paper refers to cross-entropy loss; the relationship should be clarified, since for autoregressive models the two coincide for token-level losses.
  6. [Sec. 3.2, Eq. (9)] The notation in Eq. (9) places the determinant inside a log with absolute value, which is correct for the change-of-variables term, but the variable y should be replaced by F(ŷ) for consistency with Eq. (7).
  7. [Sec. 4.3] The dataset link in Sec. 4.2 says 'LLaV A-Instruct-150K' and the text says 'llava v1 5 mix665k'; the paper should use a consistent name and provide the exact Hugging Face identifier.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-definitional metric; empirical claims rest on external benchmarks.

  1. self definitional [Sec. 3.2, Eqns (9)-(10); Sec. 3.3]
    "As illustrated in Fig. 3, this bijective metric LB quantifies the degree of hallucination induced by the LVLMs. ... θ∗ = arg min θ (E(x,v,y)∈D [− log pθ(y|x, v)] + λ · LB) (Eqn 10)."

    The paper defines the degree of hallucination as the flow negative log-likelihood LB (Eqn 9) and then adds λ·LB directly to the fine-tuning objective (Eqn 10). Consequently, any reduction in the internal hallucination measure is enforced by construction rather than discovered. However, the reported CHAIRS/CHAIRI/POPE improvements are external benchmarks, so this tautological internal metric does not by itself falsify the empirical claims; it means the 'degree of hallucination' is not an independently validated prediction.

full rationale

The paper's central empirical claims are measured on external benchmarks (CHAIR and POPE), so they do not reduce to the fitted flow or to the paper's own definitions. The main potential circularity is the internal 'bijective metric' LB: the paper states that LB quantifies the degree of hallucination, and then explicitly minimizes LB as part of the training loss (Eqn 10), making any decrease in that internal measure tautological. This step is self-definitional but not load-bearing for the externally validated performance numbers, which are the paper's substantive contribution. The separate concern that the one-hot to 128×128 bilinear downsampling may not define a genuine bijection on text responses is a correctness or verification gap, not a circularity, and therefore does not raise the circularity score. Given the external benchmarks, the overall circularity score is set to 2.

Assumptions & free parameters 5 free parameters · 5 assumptions · 3 invented entities

The ledger shows the central mechanism rests on the assumption that ground-truth training responses define non-hallucination, on an asserted invertible mapping from text to a fixed interpolated grid, and on using the fitted flow likelihood as the hallucination score. The only fitted numbers are the flow parameters and the hand-picked loss coefficient lambda; no independent hallucination labels are used.

free parameters (5)
  • lambda loss coefficient = 1e-6
    Chosen by hand to balance cross-entropy and bijective loss; the paper gives no ablation or sensitivity analysis.
  • target grid size H,W = 128, 128
    Downscaled size for the interpolated one-hot response vectors; chosen ad hoc and determines the dimensionality of the flow.
  • context length C and vocab size V = C=2048, V=32002
    Fixed limits for the one-hot encoding of responses; no treatment of variable-length sequences is provided.
  • RealNVP number of scales and blocks = scales=4, blocks=8
    Architecture choices reported without ablation.
  • reference flow parameters theta_F = trained on D_pi
    The flow is trained on the same ground-truth responses used for fine-tuning, so the reference distribution is a fitted density, not an external benchmark.
assumptions (5)
  • domain assumption Responses derived from the ground-truth dataset do not induce hallucinations.
    This is the foundation of pi_ref; it equates ground-truth-like with non-hallucinated without independent evidence. Location: Sec. 3.2, Remark 1.
  • domain assumption A reference response distribution pi_ref exists and can be approximated by optimizing negative log-likelihood on ground-truth responses.
    Assumes ground-truth responses share a common distribution and that optimizing NLL on the training set approximates it. Location: Sec. 3.2, Remarks 2 and 3.
  • ad hoc to paper There exists an invertible F mapping model-generated responses to pi_ref such that F(y-hat) follows pi_ref(y).
    This bijection is asserted; for variable-length discrete text mapped to a fixed 128 by 128 interpolated grid, invertibility is not established. Location: Sec. 3.2, Eqn 6.
  • ad hoc to paper Bilinear interpolation of one-hot encoded token sequences preserves the information needed to measure hallucination.
    Used to construct D_pi; one-hot vocab dimensions have no spatial structure, so interpolation is semantically meaningless and likely destroys token identity. Location: Sec. 3.3.
  • ad hoc to paper The negative log-likelihood under the fitted flow quantifies the degree of hallucination.
    Used to define the bijective metric L_B; no validation that NLL correlates with human-annotated hallucination. Location: Sec. 3.2, Eqn 9.
invented entities (3)
  • Reference response distribution pi_ref
    purpose: Defines the distribution of non-hallucinated responses against which model outputs are scored.
    It is constructed from the instruction-tuning dataset and there is no external validation that samples from it are consistently non-hallucinated. Location: Sec. 3.2.
  • Bijective hallucination mapping F
    purpose: Maps each generated response to a point in pi_ref and assigns a degree of hallucination via the flow log-likelihood.
    The existence and semantic meaning of F are asserted in Eqn 6, but no independent handle, such as a predicted hallucination score on held-out human-labeled data, is provided.
  • Bijective metric L_B
    purpose: Adds a regularizer to instruction fine-tuning that penalizes responses that are unlikely under pi_ref.
    This metric is defined as the negative log-likelihood under the fitted flow; its claim to measure hallucination is circular because hallucination is not independently labeled. Location: Sec. 3.2, Eqn 9.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models." pith.science (2026). https://pith.science/paper/5VHFAZEH

@misc{pith2026250524649,
  author       = {Pith},
  title        = {Pith review of: BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5VHFAZEH}},
  note         = {Machine review of arXiv:2505.24649}
}
read the original abstract

Large vision-language models have become widely adopted to advance in various domains. However, developing a trustworthy system with minimal interpretable characteristics of large-scale models presents a significant challenge. One of the most prevalent terms associated with the fallacy functions caused by these systems is hallucination, where the language model generates a response that does not correspond to the visual content. To mitigate this problem, several approaches have been developed, and one prominent direction is to ameliorate the decoding process. In this paper, we propose a new Bijective Maximum Likelihood Learning (BIMA) approach to hallucination mitigation using normalizing flow theories. The proposed BIMA method can efficiently mitigate the hallucination problem in prevailing vision-language models, resulting in significant improvements. Notably, BIMA achieves the average F1 score of 85.06% on POPE benchmark and remarkably reduce CHAIRS and CHAIRI by 7.6% and 2.6%, respectively. To the best of our knowledge, this is one of the first studies that contemplates the bijection means to reduce hallucination induced by large vision-language models.

Figures

Figures reproduced from arXiv: 2505.24649 by the authors.

Figure 1
Figure 1. Our BIMA approach helps mitigate hallucinations in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview framework of our proposed BIMA, a bijection-inspired decoding approach. The LVLM has two main components: [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of the Reference Response Distribution [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative results of our experimentation to assess the performance of BIMA approach in image caption generation task. We [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 35 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv, 2023. 1

    work page 2023

  2. [2]

    Hierarchi- cal neural story generation

    Fan Angela, Lewis Mike, and Dauphin Yann. Hierarchi- cal neural story generation. Proceedings of the 56th An- nual Meeting of the Association for Computational Linguis- tics (Volume 1: Long Papers), 2018. 2, 4

    work page 2018

  3. [3]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. Technical report, Anthropic, 2024. 1

    work page 2024

  4. [4]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report....

    work page 2025

  5. [5]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural in- formation processing systems, 33:1877–1901, 2020. 1

    1901

  6. [6]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhang- hao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yong- hao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. 1, 2, 7

    work page 2023

  7. [7]

    InstructBLIP: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Thirty- seventh Conference on Neural Information Processing Sys- tems, 2023. 1, 2

    work page 2023

  8. [8]

    Density estimation using real NVP

    Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real NVP. In International Con- ference on Learning Representations, 2017. 3, 7

    work page 2017

Show all 50 references
  1. [9]

    Vec2face: Unveil hu- man faces from their blackbox features in face recognition

    Chi Nhan Duong, Thanh-Dat Truong, Khoa Luu, Kha Gia Quach, Hung Bui, and Kaushik Roy. Vec2face: Unveil hu- man faces from their blackbox features in face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6132–6141, 2020. 3

    2020

  2. [10]

    Beam search strate- gies for neural machine translation

    Markus Freitag and Yaser Al-Onaizan. Beam search strate- gies for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation. Association for Computational Linguistics, 2017. 2, 4

    2017

  3. [11]

    Made: Masked autoencoder for distribution es- timation

    Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution es- timation. In International conference on machine learning , pages 881–889. PMLR, 2015. 3

    2015

  4. [12]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Ab- hinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 1, 2

    2024 arXiv

  5. [13]

    Sequence transduction with recurrent neural networks

    Alex Graves. Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711, 2012. 2, 4

    2012 arXiv

  6. [14]

    Detecting and preventing hallucinations in large vision language models

    Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 18135–18143, 2024. 2

    2024

  7. [15]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 1

    2025 arXiv

  8. [16]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In ICLP,

  9. [17]

    Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Con- ghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Confer...

    2024

  10. [18]

    Self-introspective de- coding: Alleviating hallucinations for large vision-language models

    Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. Self-introspective de- coding: Alleviating hallucinations for large vision-language models. In The Thirteenth International Conference on Learning Representations, 2025. 2, 3, 7, 8

    2025

  11. [19]

    Interpreting and editing vision-language representations to mitigate hallucinations

    Nicholas Jiang, Anish Kachinthaya, Suzanne Petryk, and Yossi Gandelsman. Interpreting and editing vision-language representations to mitigate hallucinations. In The Thir- teenth International Conference on Learning Representa- tions, 2025. 3, 7

    2025

  12. [20]

    Glow: Generative flow with invertible 1x1 convolutions

    Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. InAdvances in Neural Information Processing Systems, 2018. 3

    2018

  13. [21]

    Improved variational in- ference with inverse autoregressive flow.Advances in neural information processing systems, 2016

    Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational in- ference with inverse autoregressive flow.Advances in neural information processing systems, 2016. 3

    2016

  14. [22]

    Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogn...

    2024

  15. [23]

    Multimodal foundation models: From specialists to general-purpose as- sistants

    Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao, et al. Multimodal foundation models: From specialists to general-purpose as- sistants. Foundations and Trends® in Computer Graphics and Vision, 16(1-2):1–214, 2024. 1

    2024

  16. [24]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 2

    2022

  17. [25]

    Contrastive decoding: Open-ended text genera- tion as optimization

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text genera- tion as optimization. InProceedings of the 61st Annual Meet- ing of the Association for Computational Ling...

    2023

  18. [26]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Confer- ence on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023. 2, 6, 8

    2023

  19. [27]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  20. [28]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 1, 2, 6

    2023

  21. [29]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 1, 2, 5, 6, 7, 8

    2024

  22. [30]

    A survey on hallucination in large vision-language models

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiu- tian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253, 2024. 2

    2024 arXiv

  23. [31]

    Paying more at- tention to image: A training-free method for alleviating hal- lucination in lvlms

    Shi Liu, Kecheng Zheng, and Wei Chen. Paying more at- tention to image: A training-free method for alleviating hal- lucination in lvlms. In European Conference on Computer Vision, pages 125–140. Springer, 2024. 2

    2024

  24. [32]

    interpreting gpt: the logit lens

    nostalgebraist. interpreting gpt: the logit lens. https : / / www . lesswrong . com / posts / AcKRB8wDpdaN6v6ru / interpreting - gpt - the-logit-lens, 2020-08-30. 3

    2020

  25. [33]

    Masked autoregressive flow for density estimation.Advances in neural information processing systems, 30, 2017

    George Papamakarios, Theo Pavlakou, and Iain Murray. Masked autoregressive flow for density estimation.Advances in neural information processing systems, 30, 2017. 3

    2017

  26. [34]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  27. [35]

    Variational inference with normalizing flows

    Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on ma- chine learning, pages 1530–1538. PMLR, 2015. 3, 4

    2015

  28. [36]

    Object hallucination in image cap- tioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image cap- tioning. In Proceedings of the 2018 Conference on Empir- ical Methods in Natural Language Processing . Association for Computational Linguistics, 2018. 2, 6, 7

    2018

  29. [37]

    A comprehensive sur- vey of hallucination in large language, image, video and au- dio foundation models

    Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. A comprehensive sur- vey of hallucination in large language, image, video and au- dio foundation models. arXiv, 2024. 2

    2024

  30. [38]

    Trusting your evidence: Hallucinate less with context-aware decoding

    Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceed- ings of the 2024 Conference of the North American Chap- ter of the Association for Computational Linguis...

    2024

  31. [39]

    Octopus: Alleviating hal- lucination via dynamic contrastive decoding

    Wei Suo, Lijun Zhang, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, and Yanning Zhang. Octopus: Alleviating hal- lucination via dynamic contrastive decoding. arXiv preprint arXiv:2503.00361, 2025. 2, 3

    2025 arXiv

  32. [40]

    Sequence to sequence learning with neural networks

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, 2014. 2, 4

    2014

  33. [41]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text. arXiv preprint arXiv:2403.05530, 2024. 1

    2024 arXiv

  34. [42]

    Gemma: Open models based on gemini research and tech- nology

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi `ere, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and tech- nology. arXiv preprint arXiv:2403.08295, 2024. 1

    2024 arXiv

  35. [43]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram´e, et al. Gemma 2: Improving open language models at a practical size. arXiv, 2024

    2024

  36. [44]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 2

    2023 arXiv

  37. [45]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 1, 2

    2023 arXiv

  38. [46]

    Bimal: Bijective maximum likelihood approach to domain adaptation in se- mantic scene segmentation

    Thanh-Dat Truong, Chi Nhan Duong, Ngan Le, Son Lam Phung, Chase Rainwater, and Khoa Luu. Bimal: Bijective maximum likelihood approach to domain adaptation in se- mantic scene segmentation. In International Conference on Computer Vision, 2021. 3

    2021

  39. [47]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024

  40. [48]

    Mitigating hallucinations in large vision-language models with instruction contrastive decoding

    Xintong Wang, Jingheng Pan, Liang Ding, and Chris Bie- mann. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. In ACL (Find- ings), pages 15840–15853, 2024. 2, 3, 7, 8

    2024

  41. [49]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. 1

    2024 arXiv

  42. [50]

    MiniGPT-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Representa- tions, 2024. 1, 2 10

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.