Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images

Di Wang; Guimin Hu; Junchi Yao; Liangliang You; Lijie Hu; Shu Yang

REVIEW 5 major objections 7 minor 57 references

Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that behavioral hallucination in multimodal language models—describing actions that never occur across an image sequence—has two identifiable causes, and that a training-free two-stage projection method can cut it by…

desk verdict A plausible training-free idea for behavioral hallucination in image sequences, but the reported numbers are internally inconsistent and the detector is underspecified, so the paper is not reproducible as written. read the letter →

arxiv 2506.07184 v1 pith:LHCFKVJW submitted 2025-06-08 cs.AI cs.CLcs.CV

Liangliang You , Junchi Yao , Shu Yang , Guimin Hu , Lijie Hu , Di Wang This is my paper

classification cs.AIcs.CLcs.CV

keywords behavioralhallucinationmultimodallargelanguagemodelssequentialimagesdetectionorthogonalprojectionBEACHmetrictemporalwindowsnowballeffect

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that behavioral hallucinations in multimodal language models—cases where the model describes actions that never happen across a sequence of images—are not random errors but follow two identifiable mechanisms: prior-driven bias and the snowball effect. It proposes SHE, a training-free two-stage intervention that detects hallucinated behavior by checking cosine similarity between behavior text embeddings and image patch embeddings, and removes the offending visual direction by orthogonal projection. It also introduces BEACH, a metric for behavior hallucination severity. The claim is that SHE cuts BEACH scores by over 10% across standard benchmarks while keeping descriptive accuracy intact.

What carries the argument

The load-bearing mechanism is the combination of an adaptive temporal window and orthogonal projection in the joint embedding space. For each behavior token, SHE averages its intermediate text-encoder embeddings to get a behavior representation, compares it against image patch embeddings via cosine similarity across layers, and takes the maximum patch score as a confidence measure. The temporal window radius grows with the entropy of the behavior embedding, so uncertain behaviors pull features from neighboring frames to counter the snowball effect. Correction then subtracts, from the aggregated visual feature, its projection onto the behavior text embedding, scaled by a confidence-dependent coefficient so that low-confidence behaviors receive stronger correction.

What would settle it

Run SHE's detection stage on captions with human-annotated hallucinated behaviors and compare the set of tokens SHE flags as low-similarity against the human annotation; if the flagged set disagrees on a large fraction of instances, the projection step is correcting the wrong tokens and the reported BEACH gains should disappear.

Watch

Extended reading notes

Core claim

The paper establishes that behavioral hallucination in sequential images can be detected and corrected in the latent space without retraining. It first shows that hallucinated behaviors co-occur strongly with learned object–behavior associations, revealing a prior-driven bias, and that errors early in a sequence amplify later errors, revealing a snowball effect. It then shows that the same signal used for detection—low cosine similarity between averaged behavior-token embeddings and image patch embeddings—can drive correction: orthogonally projecting the aggregated visual features away from the hallucinated behavior embedding removes the spurious association. The paper reports that this two-stage procedure reduces BEACH_S and BEACH_I on four benchmarks across three model architectures, with over 10% reduction on BEACH while object hallucination metrics stay roughly flat.

Load-bearing premise

The whole method stands on the assumption that a hallucinated behavior is exactly the one whose text embedding has low cosine similarity with the image patch embeddings, yet the paper does not define how behavior tokens are selected or where the cutoff lies.

Editorial extensions

If this is right

Behavior hallucination becomes measurable separately from object hallucination through the proposed BEACH metric, giving both a per-caption rate and a per-behavior rate.
SHE works without additional training, fine-tuning, or external detectors; it only requires access to intermediate hidden-layer embeddings of the multimodal model.
The correction does not appear to trade object hallucination for behavior hallucination: CHAIR object metrics stay comparable while BEACH measures fall.
The two identified causes imply that early-frame errors are disproportionately costly, so interventions that stabilize early visual grounding should yield the largest hallucination reductions.
The method's operating point is a moderate correction strength and a temporal window coefficient around 0.4–0.6, with edits applied to middle-to-late network layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Extending beyond the paper, if the snowball effect is the dominant mechanism, SHE's gain should concentrate on corrections applied to early frames; this can be tested by restricting the adaptive window to the first segments of each sequence.
The paper leaves open how behavior tokens are selected from a caption; a direct audit is to compare SHE's flagged tokens against human-annotated hallucinated behaviors to see whether the cosine-similarity rule recovers the same set.
Because the projection removes the component aligned with the hallucinated behavior embedding, the same mechanism could plausibly be adapted to other hallucination types—such as attributes or relations—by swapping the text embedding used for projection; the paper does not explore this.
The co-occurrence analysis suggests that hallucinated behaviors cluster with learned object associations, implying that de-biasing training data on object–behavior pairs might reduce hallucination at the source, which SHE does not attempt.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea is worth a look: training-free detection of behavioral hallucinations via visual-textual alignment, with an adaptive temporal window to catch snowball effects and orthogonal projection to remove the hallucinated direction. The two-factor analysis (prior-driven bias, snowball effect) is a useful framing, and the BEACH metric, though a straightforward analog of CHAIR, fills a real gap for behavior-level evaluation. The empirical setup is broad: four datasets, three model families, and ablations over the key hyperparameters. That is real work, and the approach is cheap enough to be practically interesting if it works.

The problem is that the paper does not give the reader a way to verify it works. Table 1 and Table 4 report the same evaluation with different numbers—no explanation, no reference to one being a draft, no reconciliation. That alone would force a major revision. On top of that, the detection stage in Section 4.1 never specifies how behavior tokens are extracted from captions or what confidence threshold separates hallucinated from grounded behaviors. The limitation section concedes the method depends on threshold-based decisions and hyperparameters, but the text never states the threshold or its tuning protocol. Since the projection direction and the correction strength both depend on the detected set, the claimed >10% BEACH reduction could be a threshold artifact. The 76.7% early-frame perturbation claim in Section 3.2 is also unexplained (percentages do not sum sensibly), and the SSID dataset reference is a placeholder. These are not nitpicks; they are load-bearing omissions for a method whose entire selling point is the measured gain.

That said, the stress-test concern about circularity does not fully land. The evaluation against BEACH is external to the method, and the two-factor analysis is post hoc but plausible as motivation, not as proof. The method itself is not circular by construction.

Who is this for? A researcher working on inference-time hallucination correction for video or sequential-image MLLMs would want to know this idea exists. But they cannot reproduce it from this draft, and the internal inconsistency in the main table makes the central empirical claim unreliable.

Recommendation: do not desk-reject outright—the problem and the approach are legitimate and the experiments are extensive enough to deserve referee time. But accept with the explicit expectation of major revision: reconcile the tables, specify the detection pipeline (token extraction, threshold, sensitivity analysis), and fix the placeholder reference. My verdict would be skeptical until those are resolved.

Referee Report

5 major / 7 minor

Summary. The paper addresses behavioral hallucination in multimodal large language models (MLLMs) processing sequential images, a less-studied failure mode distinct from object hallucination. The authors identify two causes—prior-driven bias and the snowball effect—and propose SHE (Sequence Hallucination Eradication), a training-free two-stage framework: detection via visual-textual alignment using an adaptive temporal window, and mitigation via orthogonal projection of image features against hallucinated behavior embeddings. They also introduce a new metric, BEACH, to quantify behavioral hallucination severity. Experiments on Mementos, SSID, VWP, and Visual Storytelling with several MLLMs report reductions in BEACH and improvements in mAP relative to DeCo, VCD, and OPERA baselines.

Significance. The paper targets a genuinely understudied problem and proposes a conceptually interesting, training-free intervention that operates on internal representations. The introduction of BEACH as a dedicated evaluation metric is a useful contribution, and the evaluation spans multiple models and datasets. However, the empirical claims are currently not verifiable: the main results are presented in two tables with identical structure but different numbers, and the detection stage leaves the behavior-token extraction rule and the hallucination threshold unspecified. Because these omissions directly affect the measured outcomes, the significance of the method cannot be assessed until the reported numbers and the decision rule are clarified.

major comments (5)

[Tables 1 and 4] Tables 1 and 4 both report 'Behavior Hallucination Evaluation Results' for the same datasets, baselines, and model columns, but with different numerical values (e.g., Momentos/Qwen-2.5VL BEACH_S for DECO is 45.89 in Table 1 and 49.56 in Table 4). The paper does not explain which table is the final result, whether the numbers come from different runs, or why both are included. Since the abstract's 'over 10% on BEACH' claim rests on these numbers, the central empirical result is not reproducible as reported.
[Section 4.1] The detection stage defines eBeh by averaging embeddings of 'behavior tokens' but never specifies how behavior tokens are extracted from a caption; there is no parser, part-of-speech filter, LLM prompt, or annotation protocol described. Moreover, the text only states that behaviors with lower Confidence 'are considered likely to be hallucinated,' without giving a threshold, percentile, or learned classifier that converts Confidence into a binary decision. Since Section 4.2 projects away only the detected hallucinated behaviors and scales the correction strength α by the same Confidence, the entire pipeline depends on this unspecified decision rule, making the reported BEACH reductions unverifiable.
[Section 3.2] The perturbation experiment reports that noise injected into the first 10% of frames accounts for 76.7% of all behavioral hallucination perturbations, while middle frames contribute 42% and final frames have negligible impact; these percentages sum to more than 100% and the underlying counts and denominators are not provided. This numerical inconsistency weakens the snowball-effect evidence that motivates the adaptive temporal window in Section 4.1.
[Section 4.1] The adaptive temporal window radius is defined as τ = ⌈γ · Entropy(eBeh)⌉, but the entropy of an embedding vector is not defined anywhere in the paper, and the layer l_T used for behavior embeddings is only said to be 'discussed in the experimental part' without a concrete default or selection protocol in the main text. These missing definitions, together with the hyperparameters α_base and γ that the Limitation section admits require careful tuning, prevent replication of the method.
[Appendix A.5] The BEACH metric is defined in terms of 'hallucinated behaviors' and 'all annotated behaviors,' but the manuscript never specifies how hallucinated behaviors are annotated for the evaluation datasets, who performed the annotations, or how disagreements were resolved. Without this annotation protocol, BEACH cannot be independently computed, and the claimed improvements over baselines cannot be checked.

minor comments (7)

[Tables 1, 2, 4] The dataset name is spelled 'Momentos' in the tables and elsewhere as 'Mementos' (e.g., Section 5.1 and Reference [Wang et al., 2024c]); this should be made consistent.
[Figure 3] Figure 3(a) and Figure 3(b) are both labeled 'CoSc(BH)', though the discussion and the captions in Section 3.1 indicate that (b) should be CoS(BO).
[References] The reference for the SSID dataset is listed as 'Firstname Lastname. 2025', which is a placeholder and must be replaced with the actual author names and publication venue.
[Section 5.1] In the implementation details, the text states that for Qwen-VL architecture 'the beta value was set to 0.45,' but no variable β appears in the method description; this appears to be a leftover or typo.
[Section 3.2] The phrase 'Figure 4b shows shows that' contains a duplicated verb and should be corrected.
[Section 3.1] In the definition of CoS(BO), the text refers to O(oc,j) but the formula uses C(oc,j); the notation should be unified.
[Throughout] There are several spacing and formatting artifacts in the text, such as 'V olcano' and 'LLaV A', which should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SHE's detection and projection steps are defined independently of the evaluation metric, and the central claim rests on external baselines rather than on a self-citation chain.

full rationale

The paper's derivation chain is not circular in the sense defined by the review criteria. The detection stage (§4.1) computes cosine similarity between behavior-token embeddings and image-patch embeddings; the mitigation stage (§4.2) removes components aligned with detected hallucinated behaviors. These operations are specified by the paper's own equations, and the claimed reduction on BEACH is an empirical result against named baselines (DeCo, VCD, OPERA), not a quantity derived from the method's own definitions. The two proposed causes (prior-driven bias and snowball effect) are supported by separate co-occurrence and perturbation analyses, and the method's design is motivated by those analyses rather than being equivalent to them. The paper does contain non-circularity concerns that are outside this pass: Section 4.1 never specifies how behavior tokens are extracted or what similarity threshold separates hallucinated from non-hallucinated behaviors; the Limitation section itself concedes that the detection and mitigation steps rely on 'threshold-based decisions and hyperparameters.' These are reproducibility and correctness risks, not circular reductions. The placeholder reference 'Firstname Lastname' for SSID and the conflicting numbers between Table 1 and Table 4 also raise integrity concerns, but they do not make any equation equal to its own input. Self-citations in the introduction are contextual related-work citations and are not load-bearing for the central claim. Because no quoted step exhibits a prediction that reduces to its own input by construction, the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on an unspecified detection threshold, tuned hyperparameters (alpha_base, gamma), and the assumption that hallucination directions are linearly separable in embedding space. No code or data is provided.

free parameters (4)

alpha_base = [4,5]
Correction strength coefficient, tuned per model/architecture in Section 5.1 and A.7.
gamma = [0.4,0.6]
Temporal window radius coefficient, tuned in Section 5.1 and A.7.
hallucination detection threshold = not specified
Behaviors with low confidence scores are treated as hallucinated, but the cutoff is never given (Section 4.1).
layer selection l_T and editing layer = middle-to-late layers
Chosen via ablation in Section A.7; not a single number but a design choice.

assumptions (4)

domain assumption Behavioral hallucination annotations in the benchmarks are reliable and consistent with the definition in Section 1.
The BEACH metric and all experiments depend on the ground truth of which behaviors are hallucinated.
domain assumption The co-occurrence scores CoS(BH) and CoS(BO) measure prior-driven bias.
Section 3.1 interprets higher co-occurrence as evidence of learned associations, but no validation is provided.
domain assumption The attention distribution measurement (87% to text tokens) is representative of the model's reliance on text.
Appendix Figure 7 shows one aggregate statistic without specifying model, layer, or input distribution.
ad hoc to paper Orthogonal projection along the hallucinated behavior embedding removes hallucination-related signal without removing genuine visual information.
The correction formula in Section 4.2 assumes linear separability of hallucination directions in embedding space; no evidence supports this.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images." pith.science (2026). https://pith.science/paper/LHCFKVJW

@misc{pith2026250607184,
  author       = {Pith},
  title        = {Pith review of: Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LHCFKVJW}},
  note         = {Machine review of arXiv:2506.07184}
}

read the original abstract

While multimodal large language models excel at various tasks, they still suffer from hallucinations, which limit their reliability and scalability for broader domain applications. To address this issue, recent research mainly focuses on objective hallucination. However, for sequential images, besides objective hallucination, there is also behavioral hallucination, which is less studied. This work aims to fill in the gap. We first reveal that behavioral hallucinations mainly arise from two key factors: prior-driven bias and the snowball effect. Based on these observations, we introduce SHE (Sequence Hallucination Eradication), a lightweight, two-stage framework that (1) detects hallucinations via visual-textual alignment check using our proposed adaptive temporal window and (2) mitigates them via orthogonal projection onto the joint embedding space. We also propose a new metric (BEACH) to quantify behavioral hallucination severity. Empirical results on standard benchmarks demonstrate that SHE reduces behavioral hallucination by over 10% on BEACH while maintaining descriptive accuracy.

Figures

Figures reproduced from arXiv: 2506.07184 by the authors.

**Figure 1.** Behavior hallucinations indicate that the model generated behaviors not present in the image. There is no washing of a white bowl in the image; the bowl is simply placed on the table. Object hallucinations indicate that the output contains objects that are wrong or do not exist. and VCD, concentrate on static object hallucinations by processing frames independently and thus overlook inter-frame dependencies, tempor… view at source ↗

**Figure 2.** (a) The model correctly recognizes “making emotional jewelry” (highlighted in red), but then hallucinates by generating “holding a microphone.” (b) The model correctly identifies “Poker” (highlighted in blue), yet hallucinates by producing “have poker gaming.” (a) CoSc(BH) (b) CoSc(BH) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Distribution of CoSc(BH) and CoSc(BH) for hallucinatory and non-hallucinatory captions. quence. To summarize, our results show that hallucinations increase rapidly as the input sequence lengthens, especially for behavioral hallucinations. This trend occurs because the model processes each frame sequentially: an early mistake leads to more mistakes later. For objective hallucinations, errors are static—for example,… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: (a) Distribution of hallucination rates for behavioral hallucination (BH) and objective hallucination(OH) with different video lengths. (b) Distribution of hallucination rates for BH and OH for different lengths of the same videos. (c) Behavioral hallucination rates af…

**Figure 5.** Figure 5: (a) The probability of hallucination occurrence across the four models. (b) The probability of hallucinations being detected. (c) The probability of successful hallucination elimination. Step 2: Correcting Image Features. We orthogonally project each aggregated image…

**Figure 6.** Figure 6: (a) Hallucination reduction across different layers and γ value in Llava-OV model. (b) Hallucination reduction across different layers and γ value in Llava-Video model. (c) Hallucination reduction across different layers and γ value in Qwen-2.5vl model [PITH_FULL_IMAG…

**Figure 7.** Figure 7: Attention distribution. and sub-domains to maintain the representativeness of the dataset. For each video in the dataset, we utilize the provided question-answer pairs to prompt the models and generate responses. The process begins with question generation, where a s…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 5 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Muhammad Asif Ali, Nawal Daftardar, Mutayyaba Waheed, Jianbin Qin, and Di Wang. 2024. Mqa-keal: Multi-hop question answering under knowledge editing for arabic language. arXiv preprint arXiv:2409.12257

arXiv 2024
[4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025
[5]

Assaf Ben-Kish, Moran Yanuka, Morris Alper, Raja Giryes, and Hadar Averbuch-Elor. 2023. Mocha: Multi-objective reinforcement mitigating caption hallucinations. arXiv preprint arXiv:2312.03631

arXiv 2023
[6]

Xiaohui Chen, Satya Narayan Shukla, Mahmoud Azab, Aashu Singh, Qifan Wang, David Yang, ShengYun Peng, Hanchao Yu, Shen Yan, Xuewen Zhang, and Baosheng He. 2024 a . Compcap: Improving multimodal large language models with composite captions. arXiv preprint arXiv:2412.05243

work page Pith review arXiv 2024
[7]

Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. 2024 b . Halc: Object hallucination reduction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.00425

arXiv 2024
[8]

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Zhong Muyan, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2023. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238

arXiv 2023

Show all 57 references

[9]

Keyuan Cheng, Zijian Kan, Zhixian He, Zhuoran Zhang, Muhammad Asif Ali, Ke Xu, Lijie Hu, and Di Wang. 2025 a . Compke: Complex question answering under knowledge editing. arXiv preprint arXiv:2506.00829

2025 arXiv
[10]

Keyuan Cheng, Gang Lin, Haoyang Fei, Lu Yu, Muhammad Asif Ali, Lijie Hu, Di Wang, and 1 others. 2024. Multi-hop question answering under temporal knowledge editing. arXiv preprint arXiv:2404.00492

2024 arXiv
[11]

Keyuan Cheng, Xudong Shen, Yihao Yang, Tengyue Wang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, and Di Wang. 2025 b . Codemenv: Benchmarking large language models on code migration. arXiv preprint arXiv:2506.00894

2025 arXiv
[12]

Ailin Deng, Zhirui Chen, and Bryan Hooi. 2024. Seeing is believing: Mitigating hallucination in large vision-language models via clip-guided decoding. arXiv preprint arXiv:2402.15300

2024 arXiv
[13]

Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, and Xing Sun. 2024. Video-mme: The first-ever comprehensive evaluation b...

2024 arXiv
[14]

Xudong Hong, Asad Sayeed, Khushboo Mehra, Vera Demberg, and Bernt Schiele. 2023. https://arxiv.org/abs/2301.08571 Visual writing prompts: Character-grounded story generation with curated image sequences . arXiv preprint arXiv:2301.08571. Accessed: 2025-05-15

2023 arXiv
[15]

Lijie Hu, Liang Liu, Shu Yang, Xin Chen, Zhen Tan, Muhammad Asif Ali, Mengdi Li, and Di Wang. 2024. Understanding reasoning in chain-of-thought from the hopfieldian view. arXiv preprint arXiv:2410.03595

2024 arXiv
[16]

Hang Hua, Yunlong Tang, Chenliang Xu, and Jiebo Luo. 2025. V2xum-llm: Cross-modal video summarization with temporal prompt instruction tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3599--3607

2025
[17]

Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Con...

2024
[18]

Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell

Ting-Hao Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell. 2016. https://arxiv.org/pdf/16...

2016 arXiv
[19]

Jitesh Jain, Jianwei Yang, and Humphrey Shi. 2023. Vcoder: Versatile vision encoders for multimodal large language models. arXiv preprint arXiv:2312.14233

2023 arXiv
[20]

Cheongwoong Kang and Jaesik Choi. 2023. https://arxiv.org/abs/2310.08256 Impact of co-occurrence on factual knowledge of large language models . arXiv preprint arXiv:2310.08256

2023 arXiv
[21]

Firstname Lastname. 2025. https://ieeexplore.ieee.org/document/10177150 Sequential vision to language as story: A storytelling dataset and benchmarking . IEEE Journals & Magazine. Accessed: 2025-05-15

2025
[22]

Seongyun Lee, Sue Hyun Park, Yongrae Jo, and Minjoon Seo. 2023. Volcano: mitigating multimodal hallucination through self-feedback guided revision. arXiv preprint arXiv:2311.07362

2023 arXiv
[23]

Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2023. https://arxiv.org/abs/2311.16922 Mitigating object hallucinations in large vision-language models through visual contrastive decoding . arXiv preprint arXiv, 2311.16922

2023 arXiv
[24]

Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

2024
[25]

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024 a . Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

2024 arXiv
[26]

Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. 2024 b . Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895

2024 arXiv
[27]

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355

2023 arXiv
[28]

Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023. Mitigating hallucination in large multi-modal models via robust instruction tuning. arXiv preprint arXiv:2306.14565

2023 arXiv
[29]

Ziyu Liu, Zeyi Sun, Yuhang Zang, Wei Li, Pan Zhang, Xiaoyi Dong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. 2024. Rar: Retrieving and ranking augmented mllms for visual recognition. arXiv preprint arXiv:2403.13805

2024 arXiv
[30]

Fan Ma, Xiaojie Jin, Heng Wang, Yuchen Xian, Jiashi Feng, and Yi Yang. 2023. https://arxiv.org/pdf/2312.08870 Vista-llama: Reliable video narrator via equal distance to visual tokens . arXiv preprint arXiv:2312.08870

2023 arXiv
[31]

Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. arXiv preprint arXiv:1809.02156

2018 arXiv
[32]

Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. arXiv preprint arXiv:2306.05540

2023 arXiv
[33]

Zheng Sun, Shijun Shen, Sheng Cao, Hao Liu, Chong Li, Yi Shen, Chuang Gan, Li-Yi Gui, Yong-Xiang Wang, and Yi et al. Yang. 2023. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525

2023 arXiv
[34]

Andr \'e s Villa, Juan Carlos Le \'o n Alc \'a zar, Alvaro Soto, and Bernard Ghanem. 2023. Behind the magic, merlim: Multi-modal evaluation benchmark for large image-language models. arXiv preprint arXiv:2312.02219

2023 arXiv
[35]

Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, and Huajun Chen. 2024 a . Mllm can see? dynamic correction decoding for hallucination mitigation. arXiv preprint arXiv:2410.11779

2024 arXiv
[36]

Lei Wang, Jiabang He, Shenshen Li, Ning Liu, and Ee-Peng Lim. 2023. https://arxiv.org/abs/2312.01701 Mitigating fine-grained hallucination by fine-tuning large vision-language models with caption rewrites . arXiv preprint arXiv:2312.01701

2023 arXiv
[37]

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024 b . Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

2024 arXiv
[38]

Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, and 1 others. 2024 c . Mementos: A comprehensive benchmark for multimodal large language model reasoning over image sequences. arXiv preprint arXiv...

2024 arXiv
[39]

Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng. 2024 d . Videohallucer: Evaluating intrinsic and extrinsic hallucinations in large video-language models. arXiv preprint arXiv:2406.16338

2024 arXiv
[40]

Xiyang Wu, Tianrui Guan, Dianqi Li, Shuaiyi Huang, Xiaoyu Liu, Xijun Wang, Ruiqi Xian, Abhinav Shrivastava, Furong Huang, Jordan Lee Boyd-Graber, Tianyi Zhou, and Dinesh Manocha. 2024. Autohallusion: Automatic generation of hallucination benchmarks for vision-language models. ...

2024 arXiv
[41]

Shangyu Xing, Fei Zhao, Zhen Wu, Tuo An, Weihao Chen, Chunhui Li, Jianbing Zhang, and Xinyu Dai. 2024. Efuf: Efficient fine-grained unlearning framework for mitigating hallucinations in multimodal large language models. arXiv preprint arXiv:2402.09801

2024 arXiv
[42]

Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, and Mohan Kankanhalli. 2023. An llm can fool itself: A prompt-based adversarial attack. arXiv preprint arXiv:2310.13345

2023 arXiv
[43]

Siming Yan, Min Bai, Weifeng Chen, Xiong Zhou, Qixing Huang, and Li Erran Li. 2024. Vigor: Improving visual grounding of large vision language models with fine-grained reward modeling. arXiv preprint arXiv:2402.06118

2024
[44]

Shu Yang, Muhammad Asif Ali, Lu Yu, Lijie Hu, and Di Wang. 2024 a . Monal: Model autophagy analysis for modeling human-ai interactions. arXiv preprint arXiv:2402.11271

2024 arXiv
[45]

Shu Yang, Shenzhe Zhu, Ruoxuan Bao, Liang Liu, Yu Cheng, Lijie Hu, Mengdi Li, and Di Wang. 2024 b . What makes your model a low-empathy or warmth person: Exploring the origins of personality in llms. arXiv preprint arXiv:2410.10863

2024 arXiv
[46]

Shu Yang, Shenzhe Zhu, Zeyu Wu, Keyu Wang, Junchi Yao, Junchao Wu, Lijie Hu, Mengdi Li, Derek F Wong, and Di Wang. 2025. Fraud-r1: A multi-round benchmark for assessing the robustness of llm against augmented fraud and phishing inducements. arXiv preprint arXiv:2502.12904

2025 arXiv
[47]

Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421, 9(1):1

2023 arXiv
[48]

Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the ieee/cvf conference on computer vision and pattern recognit...

2024
[49]

Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549

2023 arXiv
[50]

Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2024. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 67(12):220105

2024
[51]

Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wentao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yueting Zhuang. 2023. Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. arXiv preprint arXiv:2311.13614

2023 arXiv
[52]

Hang Zhang, Xin Li, and Lidong Bing. 2023. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858

2023 arXiv
[53]

Lin Zhang, Lijie Hu, and Di Wang. 2025. Mechanistic unveiling of transformer circuits: Self-influence as a key to model reasoning. arXiv preprint arXiv:2502.09022

2025 arXiv
[54]

Ruiyang Zhang, Hu Zhang, and Zhedong Zheng. 2024 a . Vl-uncertainty: Detecting hallucination in large vision-language model via uncertainty estimation. arXiv preprint arXiv:2411.11919

2024 arXiv
[55]

Zhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng, Lijie Hu, and Di Wang. 2024 b . Locate-then-edit for multi-hop factual recall under knowledge editing. arXiv preprint arXiv:2410.06331

2024 arXiv
[57]

Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. 2023 b . Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839

2023 arXiv
[58]

Lanyun Zhu, Deyi Ji, Tianrun Chen, Peng Xu, Jieping Ye, and Jun Liu. 2024. Ibd: Alleviating hallucinations in large vision-language models via image-biased decoding. arXiv preprint arXiv:2402.18476

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Muhammad Asif Ali, Nawal Daftardar, Mutayyaba Waheed, Jianbin Qin, and Di Wang. 2024. Mqa-keal: Multi-hop question answering under knowledge editing for arabic language. arXiv preprint arXiv:2409.12257

arXiv 2024

[4] [4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025

[5] [5]

Assaf Ben-Kish, Moran Yanuka, Morris Alper, Raja Giryes, and Hadar Averbuch-Elor. 2023. Mocha: Multi-objective reinforcement mitigating caption hallucinations. arXiv preprint arXiv:2312.03631

arXiv 2023

[6] [6]

Xiaohui Chen, Satya Narayan Shukla, Mahmoud Azab, Aashu Singh, Qifan Wang, David Yang, ShengYun Peng, Hanchao Yu, Shen Yan, Xuewen Zhang, and Baosheng He. 2024 a . Compcap: Improving multimodal large language models with composite captions. arXiv preprint arXiv:2412.05243

work page Pith review arXiv 2024

[7] [7]

Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. 2024 b . Halc: Object hallucination reduction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.00425

arXiv 2024

[8] [8]

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Zhong Muyan, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2023. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238

arXiv 2023