Pith. sign in

REVIEW 2 major objections 5 minor 92 references

MESH, a new benchmark, measures hallucinations in large video models by testing the three perceptual levels at which humans watch video—setting, characters, and their actions—and shows that models hallucinate most on fine details and long c

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:28 UTC pith:QAN6UD3K

load-bearing objection MESH is a genuinely novel hallucination benchmark with a sensible perception-based hierarchy, but the unverified absence of trap instances makes its difficulty trends partly suspect. the 2 major comments →

arxiv 2509.08538 v2 pith:QAN6UD3K submitted 2025-09-10 cs.CV cs.AI

MESH -- Understanding Videos Like Human: Measuring Hallucinations in Large Video Models

Garry Yang , Zizhe Chen , Man Hon Wong , Haoyu Lei , Yongqiang Chen , Zhenguo Li , Kaiwen Zhou , James Cheng This is my paper
classification cs.CV cs.AI
keywords video hallucinationlarge video modelsbenchmarkmise-en-scènevideo question answeringsubject-action alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MESH is a benchmark for measuring when large video models hallucinate, organized the way humans actually watch video: first the setting and its objects, then the characters and their features, then the characters' actions and dialogue. The paper argues that existing video-hallucination benchmarks lack this perception-based, bottom-up structure and therefore miss the places where models fail. Built on TVQA+ annotations, MESH generates binary and multiple-choice questions that pair real content with "trap" items absent from or mismatched to the video. Across many LVMs, the benchmark shows accuracy is high for basic objects and coarse features but falls off sharply for fine details and for aligning multiple subjects' actions in longer clips. The authors also report that MESH scores line up with general video QA benchmarks, suggesting the hallucination levels it measures matter for real video understanding.

Core claim

MESH's central claim: hallucination in video models grows predictably as questions move from coarse to fine along a setting–character–stage ladder. Models that recognize basic objects and coarse features fail on fine character details, and fail most on stage questions that bind several subjects to their actions across many frames. MESH exposes this by building positive targets from TVQA+'s spatial, subtitle, and action annotations, and negative traps absent or mismatched by construction: objects from other locations, swapped features, imported actions. The authors claim this difficulty ordering is stable across sixteen open and closed models, separates models that exploit multi-frame tokens

What carries the argument

The load-bearing object is the mise-en-scène decomposition: every clip is probed at three levels—Setting (environment and objects), Character (each subject encoded as an eight-feature vector: gender, garment type and color, glasses, sleeve type, collar, pocket, garment shade), and Stage (subject-action pairs and dialogue attribution). Difficulty is graded by granularity masks that hide or reveal features and by clip length. Negatives are built by substitution and then filtered by text-embedding cosine similarity so traps stay distinguishable from targets. This target-trap machinery, in binary and multi-choice formats, is what ranks hallucination difficulty rather than just counting errors.

Load-bearing premise

The benchmark's ground truth assumes each trap item is genuinely absent from (or mismatched to) its video, resting on sparse TVQA+ annotations, LLM-generated object sets, and a text-embedding filter, with no reported human-agreement or error rates for verification.

What would settle it

Hand-check a random sample of rejected Setting traps (say 200 'no' questions) against the full video frames rather than just the annotated frames: if a visible object such as a bed or bottle turns up in an unannotated frame, then refusing the trap is not evidence of non-hallucination and Setting accuracy numbers would be inflated. Likewise, re-derive the eight-feature character vectors for the same clips with two independent annotation teams and measure agreement; low agreement on features like collar or pocket presence would weaken the fine-grained difficulty ladder.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • MESH's basic-versus-advanced score gap predicts performance on general video QA benchmarks (Video-MME, MLVU), so cutting fine-grained hallucinations should improve high-level video understanding.
  • Model scale helps on fine details in long videos but not on coarse questions, indicating fine-grained perception, not raw parameter count, is the current bottleneck.
  • The benchmark separates models that leverage multi-frame tokens from those that are distracted by more frames: stronger models gain accuracy from longer clips while weaker ones lose up to 20 points.
  • The annotation pipeline transfers to other datasets, demonstrated on UCF101 and a concatenated UCF101-COMBINE, reproducing the same coarse-to-fine difficulty trend.
  • Aggressive token reduction (e.g., LLaMA-VID's two tokens per frame) correlates with the largest hallucination rates, linking architectural design choices to hallucination.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to use MESH's target/trap pairs as contrastive training data—teaching models to reject near-miss characters and imported actions—and check whether VQA gains follow; the paper reports correlation but does not test causation.
  • The trap-absence assumption is the risk point: traps come from sparse TVQA+ spatial groundings plus LLM-generated object sets, so a trap object may appear in a frame that was never annotated. Re-verifying negative questions against full videos, and reporting inter-annotator agreement on character features, would firm up or refute the ground truth.
  • Similar-action and role-mix traps are the hardest categories, suggesting they isolate the subject-action binding step and could serve as a dedicated stress test for future models.
  • Closed-source general models underperform open video-specific models on stage questions; a follow-up could determine whether that gap comes from missing video fine-tuning or from frame-sampling strategy, which the paper's data cannot separate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces MESH, a video hallucination benchmark built on TVQA+. It organizes questions along a bottom-up mise-en-scène taxonomy — Setting (objects), Character (coarse-to-fine person features), and Stage (subject–action and dialogue pairs) — and generates both binary and four-option multiple-choice questions pairing a target instance with trap instances. The benchmark is evaluated on a wide range of open- and closed-source LVMs. The main reported findings are that LVMs perform well on basic object/coarse-feature questions but degrade markedly on fine-grained character details and on subject–action alignment in longer clips; that SA/MI and CI/CO categories are harder than EA/CO/AO; and that MESH scores correlate with V-MME and MLVU. The paper also reports image-vs-video ablations, temporal-order experiments, and a generalization study on UCF101 and UCF101-COMBINE.

Significance. If the benchmark's ground-truth labels are trustworthy, MESH is a useful diagnostic resource: it is perception-motivated, publicly released, evaluated on 20+ models with diverse backbones, and it includes balance metrics (OB/COB, JSD) that go beyond raw accuracy. The attempt to separate setting/character/stage and to construct fine-grained traps is a genuine contribution. However, the central validity of every negative question depends on trap absence or mismatch being verified, and the paper currently provides only weak, indirect filters for this. Because false-negative labels could be correlated with the very trends the paper highlights, the empirical conclusions are not yet fully load-bearing.

major comments (2)
  1. [3.3.1–3.3.3, B.2.1, B.4.6] The validity of each trap question relies on the trap instance being truly absent from (or mismatched to) the video, but the construction pipeline never verifies this. Setting traps are sampled from object sets of other space labels (Eq. 3), yet the paper itself states that TVQA+ object annotations are sparse (B.2.1). Character traps are feature vectors from other videos, filtered only by feature-group dissimilarity and an embedding similarity criterion (Eq. 5); a trap descriptor could match an unannotated bystander or a real character with the same 8-feature vector. Stage traps are produced by substituting subjects/actions, and the only quality gate is text-embedding cosine distance > 0.65 (B.4.6). Human verification is reported for location names and character features, but not for trap absence, and no inter-annotator agreement or error rate is given. This is not cosmetic: false-negati
  2. [4.3.2 / Table 7] The claim that MESH results 'align with' V-MME and MLVU is supported only by six models and no quantitative measure. The ordering is not even monotone: e.g., LLaVA-OV-72B has a higher Advanced MESH score than Aria-23B but a lower VMME score, and GPT-4o has a lower Advanced score than LLaVA-OV-72B yet a higher VMME score. Without a Spearman/Kendall correlation, confidence intervals, or an explicit regression, the alignment claim is not established. Since this is one of the three advertised findings, please compute a correlation statistic or clearly label the comparison as qualitative.
minor comments (5)
  1. [Section 3.3.3] Typo: 'devide' should be 'divide'.
  2. [Appendix B.4.5] In the MI example, the swapped result is given twice as 'A man in a blue jacket without glasses is sitting'; the second should refer to the other subject (e.g., the man in a gray t-shirt) for the role swap to be coherent.
  3. [Table 2 vs Appendix C.1.2] There are small numeric inconsistencies in the Stage Action binary counts: Table 2 reports Yes/EA = 2491, while Appendix C.1.2 reports 2488. Please reconcile.
  4. [Notation / Table 3] Model naming is inconsistent: LLaVA-NV-32B, LLaVA-NeXT-Video-32B, and LLaVA-NeXT-Video-7B are used interchangeably; stage tables also mix 'COV/AOV/MIV' with the main text's 'CO/AO/MI'. Please standardize.
  5. [Appendix B.4.6] The embedding model 'text-embedding-3-small' is used as a quality gate but no version or reference is given; also the 0.65 threshold is not justified. Please document the model version and provide a sensitivity analysis or rationale for the threshold.

Circularity Check

0 steps flagged

No significant circularity: the benchmark's measurements are empirical evaluations of LVM behavior, not derivations from its own construction.

full rationale

The paper constructs a video-hallucination benchmark (MESH) and reports LVM accuracies across settings, character-feature granularities, action/dialogue categories, and clip lengths. I walked the construction chain in Sections 3.2-3.3 and Appendices B.2-B.4. No parameter is fitted to the evaluated models, and no headline result is derived from an equation that already contains that result. The difficulty ladder (coarse/medium/fine, basic/advanced) is set by construction—masking more character features or choosing harder trap types—but this only defines the question granularity; the reported finding that LVMs do worse on fine-grained or longer-video questions is an empirical measurement of model behavior, not a logical consequence of the benchmark definitions. Target/trap instances come from TVQA+ annotations and LLM-generated object/feature/action sets; the trap selections (Eqs. 3, 5, 8-10, Algorithms 2-5) use similarity filters and space/feature/action dissimilarity rules, but those rules do not encode the measured accuracies. The one self-citation by an author (ref. [52]) concerns graph contrastive learning and is not load-bearing. A potential concern that trap absence is never fully verified would be an annotation-validity or label-noise issue, not circularity: answering 'no' to a trap measures whether the model rejects that instance, and whether the instance truly is absent affects benchmark validity, but the paper does not reduce its predictions to its inputs by definition. Therefore no circular step can be exhibited, and the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The paper contains no fitted parameters in the derivation sense, but three hand-chosen design values shape the central findings: the 0.65 text-embedding threshold that decides whether a trap is kept, the feature-mask granularity ladder (coarse at 3 features up to fine at 8) that defines the difficulty axis and therefore pre-figures the 'fine details are harder' result, and the frame-sampling budgets (8/16/32/64, 1 fps dialogues) that determine the length effects. The axioms are unvalidated domain assumptions: bottom-up film-theory cognition, sparse TVQA+ annotation completeness, LLM annotation accuracy after unquantified human checks, and embedding similarity as perceptual similarity.

free parameters (3)
  • Semantic similarity threshold for trap retention = 0.65 cosine distance
    Appendix B.4.6; traps whose target-similarity exceeds 0.65 are discarded; this cut shapes which traps enter the benchmark and directly affects measured difficulty and accuracy.
  • Granularity mask levels = coarse=3 features, medium=5, mixed/fine=8 (with random masking)
    Appendix B.3.3; the difficulty ladder is defined by how many of the 8 character features are unmasked, so the headline finding that finer questions are harder is partly encoded by construction.
  • Frame sampling budgets = 8/16/32/64 frames for character; 32 default elsewhere; 1 fps for dialogue
    Sections 4.2.1-4.2.3; the finding that longer inputs hurt fine-grained answer quality is co-determined by these hand-chosen budgets and by whether frames are centered or uniform.
axioms (4)
  • domain assumption Humans understand videos bottom-up through Setting, then Characters, then Stage
    Section 1 and Appendix A; taken from film theory (Bordwell and Thompson [7]); not empirically validated in the paper, yet it justifies the benchmark's hierarchy and the claim that MESH 'aligns with human video understanding'.
  • domain assumption TVQA+ spatial grounding plus LLM-generated object lists are complete enough to declare trap objects absent
    Section 3.3.1 Eq. (3), Appendix B.2.1 (paper concedes video object annotations are sparse); traps are selected from other scene labels and assumed never to appear unannotated in the target clip.
  • domain assumption LLM annotation outputs (DeepSeek-v3 objects, GPT-4o features, Qwen2-VL locations) are correct after human verification
    Sections 3.3.2, 3.3.3, Appendices B.2.3, B.3.2; no annotator counts, inter-annotator agreement, or correction rates are reported, so the quality gate is stated but unmeasured.
  • ad hoc to paper Text-embedding cosine distance is a valid proxy for how distinguishable a trap is from a target
    Appendix B.4.6; the 0.65 threshold in text-embedding space replaces human judgment of perceptual distinguishability without any validation study.
invented entities (2)
  • Mise-en-scene hallucination taxonomy (Setting/Character/Stage) no independent evidence
    purpose: organize hallucination question generation into a bottom-up, perception-aligned hierarchy
    Introduced in Section 1 and Appendix A; an analytical construct from film studies; the paper provides no independent evidence that human video cognition actually follows this order.
  • Stage trap categories CO, CI, AO, SA, MI no independent evidence
    purpose: generate negative subject-action instances (character out/in video, action out of video, similar action, mixed swap)
    Section 3.3.3, Appendix B.4.5; new analytic categories; the observed difficulty ordering (SA, MI hardest) is presented as empirical but is partly a function of how these traps are constructed.

pith-pipeline@v1.3.0-alltime-deepseek · 53476 in / 18838 out tokens · 186082 ms · 2026-08-04T20:28:07.694240+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of MESH -- Understanding Videos Like Human: Measuring Hallucinations in Large Video Models." pith.science (2026). https://pith.science/paper/QAN6UD3K

@misc{pith2026250908538,
  author       = {Pith},
  title        = {Pith review of: MESH -- Understanding Videos Like Human: Measuring Hallucinations in Large Video Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QAN6UD3K}},
  note         = {Machine review of arXiv:2509.08538}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Video Models (LVMs) build on the semantic capabilities of Large Language Models (LLMs) and vision modules by integrating temporal information to better understand dynamic video content. Despite their progress, LVMs are prone to hallucinations-producing inaccurate or irrelevant descriptions. Current benchmarks for video hallucination depend heavily on manual categorization of video content, neglecting the perception-based processes through which humans naturally interpret videos. We introduce MESH, a benchmark designed to evaluate hallucinations in LVMs systematically. MESH uses a Question-Answering framework with binary and multi-choice formats incorporating target and trap instances. It follows a bottom-up approach, evaluating basic objects, coarse-to-fine subject features, and subject-action pairs, aligning with human video understanding. We demonstrate that MESH offers an effective and comprehensive approach for identifying hallucinations in videos. Our evaluations show that while LVMs excel at recognizing basic objects and features, their susceptibility to hallucinations increases markedly when handling fine details or aligning multiple actions involving various subjects in longer videos.

Figures

Figures reproduced from arXiv: 2509.08538 by Garry Yang, Haoyu Lei, James Cheng, Kaiwen Zhou, Man Hon Wong, Yongqiang Chen, Zhenguo Li, Zizhe Chen.

Figure 1
Figure 1. Figure 1: LVMs understand videos. “Is there a comprehensive benchmark for mea￾suring hallucination in video understanding?” Most existing benchmarks either adopt caption-based compar￾isons or adapt the binary question format from POPE [40]. They evaluate LVMs on various aspects, including subject recognition or temporal details [15, 19, 37, 39, 67, 78, 81, 84]. However, most exist￾ing approaches depend on manual cat… view at source ↗
Figure 2
Figure 2. Figure 2: Space Grounding and Subtitle/Action Annotations from TVQA+ and example questions [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Constructing setting hallucination questions [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Constructing character hallucination questions [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Constructing stage hallucination questions [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Character Hallucination for Binary/MC tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Dialogue Hallucination for Binary/MC tasks [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 7
Figure 7. Figure 7: Action Hallucination for Binary/MC tasks [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Left: "A man wearing long-sleeved collared pink button-up shirt with pockets and not wearing glasses appears in the [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Left: Representative frame from clip a depicting A domestic living room environment, Right: Representative frame [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Pipeline for labeling the locations in the video. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Pipeline for labeling the character features in the video. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: An example of spatial-temporal hallucination question. The 1, 2, and 3 in the question can be permuted or substituted [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Using YOLO to generate bounding boxes for the characters in UCF101 dataset [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Answer Distribution for binary task and multi-choice task in Setting and Character Hallucination. [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Category and answer distribution for binary task and multi-choice task in Stage Hallucination (Action). [PITH_FULL_IMAGE:figures/full_fig_p023_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Category and answer distribution for binary task and multi-choice task in Stage Hallucination (Dialogue). [PITH_FULL_IMAGE:figures/full_fig_p023_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Close-source models on Character for Binary/MC tasks. [PITH_FULL_IMAGE:figures/full_fig_p025_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: An example of one video and part of its corresponding hallucination questions. [PITH_FULL_IMAGE:figures/full_fig_p040_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Analysis of video question answering and description tasks reveals that LVMs exhibit a tendency to avoid generating [PITH_FULL_IMAGE:figures/full_fig_p041_20.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

92 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    Aakanksha and A. N. Rajagopalan. 2023. Improving Robustness of Semantic Segmentation to Motion-Blur Using Class-Centric Augmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. IEEE, 10470–10479. doi:10.1109/CVPR52729.2023.01009

    arXiv 2023

  2. [2]

    Anthropic. 2024. Introducing Claude 3.5 Sonnet. https://www.anthropic.com/ news/claude-3-5-sonnet

    2024

  3. [3]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.16609 2023

  4. [4]

    Hritik Bansal, Yonatan Bitton, Idan Szpektor, Kai-Wei Chang, and Aditya Grover

  5. [5]

    2020.The visual story: Creating the visual structure of film, TV, and digital media

    Bruce Block. 2020.The visual story: Creating the visual structure of film, TV, and digital media. Routledge

    2020

  6. [6]

    2005.Figures traced in light: On cinematic staging

    David Bordwell. 2005.Figures traced in light: On cinematic staging. Univ of California Press

    2005

  7. [7]

    Bordwell and K

    D. Bordwell and K. Thompson. 2008.Film Art: An Introduction. McGraw Hill. https://books.google.com.hk/books?id=g6ZAAQAAIAAJ

    2008

  8. [8]

    Paulo Vinicius Koerich Borges, Nicola Conci, and Andrea Cavallaro. 2013. Video- based human behavior understanding: A survey.IEEE transactions on circuits and systems for video technology23, 11 (2013), 1993–2008

    2013

  9. [9]

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, S...

  10. [10]

    Guangyi Chen, Xiao Liu, Guangrun Wang, Kun Zhang, Philip H. S. Torr, Xiao- Ping Zhang, and Yansong Tang. 2023. Tem-adapter: Adapting Image-Text Pre- training for Video Question Answer. InIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023. IEEE, 13899–13909. doi:10.1109/ICCV51070.2023.01282

    arXiv 2023

  11. [11]

    arXiv:2403.17297 [cs.CL]

    InternLM2 Technical Report. arXiv:2403.17297 [cs.CL]

    Pith/arXiv arXiv

  12. [12]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 24185–24198

    2024

  13. [13]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al . 2024. Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling.arXiv preprint arXiv:2412.05271(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. https://lmsys.org/blog/2023-03-30-vicuna/

    2023

  15. [15]

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. 2024. VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs.arXiv preprint arXiv:2406.07476(2024). https://arxiv.org/abs/ 2406.07476

    Pith/arXiv arXiv 2024

  16. [16]

    2013.Human information processing: Vision, memory, and attention.American Psychological Association

    Charles Ed Chubb, Barbara A Dosher, Zhong-Lin Ed Lu, and Richard M Shiffrin. 2013.Human information processing: Vision, memory, and attention.American Psychological Association

    2013

  17. [17]

    Kankanhalli

    Wey Yeh Choong, Yangyang Guo, and Mohan S. Kankanhalli. 2024. VidHal: Benchmarking Temporal Hallucinations in Vision LLMs.CoRRabs/2411.16771 (2024). arXiv:2411.16771 doi:10.48550/ARXIV.2411.16771

    Pith/arXiv arXiv doi:10.48550/arxiv.2411.16771 2024

  18. [18]

    LMDeploy Contributors. 2023. LMDeploy: A Toolkit for Compressing, Deploying, and Serving LLM. https://github.com/InternLM/lmdeploy

    2023

  19. [20]

    Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), Vol. 1. Ieee, 886–893

    2005

  20. [21]

    Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M. Asano. 2025. Lost in Time: A New Temporal Benchmark for VideoLLMs. arXiv:2410.07752 [cs.CV] https://arxiv.org/abs/2410.07752

    Pith/arXiv arXiv 2025

  21. [22]

    Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. 2025. Videoagent: A memory-augmented multimodal agent for video understanding. InEuropean Conference on Computer Vision. Springer, 75–92

    2025

  22. [23]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huaj...

    Pith/arXiv arXiv 2024

  23. [24]

    2002.Mise-en-scène: Film style and interpre- tation

    John Gibbs and John Edward Gibbs. 2002.Mise-en-scène: Film style and interpre- tation. Vol. 10. Wallflower Press

    2002

  24. [25]

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun. 2024. Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis.CoRRabs/...

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.21075 2024

  25. [26]

    Jiaming Han, Renrui Zhang, Wenqi Shao, Peng Gao, Peng Xu, Han Xiao, Kaipeng Zhang, Chris Liu, Song Wen, Ziyu Guo, Xudong Lu, Shuai Ren, Yafei Wen, Xiaoxin Chen, Xiangyu Yue, Hongsheng Li, and Yu Qiao. 2023. ImageBind-LLM: Multi- modality Instruction Tuning.CoRRabs/2309.03905 (2023). arXiv:2309.03905 doi:10.48550/ARXIV.2309.03905

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.03905 2023

  26. [29]

    Berthold KP Horn and Brian G Schunck. 1981. Determining optical flow.Artificial intelligence17, 1-3 (1981), 185–203

    1981

  27. [30]

    Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau,...

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.21276 2024

  28. [31]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Hao- tian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. A Survey on Hallucination in Large Language Models: Princi- ples, Taxonomy, Challenges, and Open Questions.CoRRabs/2311.05232 (2023). arXiv:2311.05232 doi:10.48550/ARXIV.2311.05232

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.05232 2023

  29. [32]

    A Comprehensive Survey on Visual Question Answering Datasets and Algorithms

    Raihan Kabir, Naznin Haque, Md. Saiful Islam, and Marium-E-Jannat. 2024. A Comprehensive Survey on Visual Question Answering Datasets and Algorithms. CoRRabs/2411.11150 (2024). arXiv:2411.11150 doi:10.48550/ARXIV.2411.11150

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2411.11150 2024

  30. [33]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation.ACM Comput. Surv.55, 12 (2023), 248:1–248:38. doi:10.1145/3571730

    doi:10.1145/3571730 2023

  31. [34]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAtten- tion. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  32. [35]

    Serpil Kocdar, Nejdet Karadag, and Murat Dogan Sahin. 2016. Analysis of the Difficulty and Discrimination Indices of Multiple-Choice Questions According to Cognitive Levels in an Open and Distance Learning Context.Turkish Online Journal of Educational Technology-TOJET15, 4 (2016), 16–24

    2016

  33. [36]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326(2024)

    Pith/arXiv arXiv 2024

  34. [37]

    Berg, and Mohit Bansal

    Jie Lei, Licheng Yu, Tamara L. Berg, and Mohit Bansal. 2020. TVQA+: Spatio- Temporal Grounding for Video Question Answering. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Li...

    doi:10.18653/v1/ 2020

  35. [38]

    Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Guoyin Wang, Bei Chen, and Junnan Li. 2024. Aria: An Open Multimodal Native Mixture-of-Experts Model.arXiv preprint arXiv:2410.05993(2024)

    Pith/arXiv arXiv 2024

  36. [39]

    Chaoyu Li, Eun Woo Im, and Pooyan Fazli. 2024. VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding. arXiv:2412.03735 [cs.CV] https://arxiv.org/abs/2412.03735

    Pith/arXiv arXiv 2024

  37. [40]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating Object Hallucination in Large Vision-Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computati...

    doi:10.18653/v1/2023.emnlp-main.20 2023

  38. [42]

    Yanwei Li, Chengyao Wang, and Jiaya Jia. 2025. Llama-vid: An image is worth 2 tokens in large language models. InEuropean Conference on Computer Vision. Springer, 323–340

    2025

  39. [43]

    Yanyang Li, Shuo Liang, Michael R Lyu, and Liwei Wang. 2024. Making long-context language models better multi-hop reasoners.arXiv preprint arXiv:2408.03246(2024)

    Pith/arXiv arXiv 2024

  40. [44]

    Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han. 2023. VILA: On Pre-training for Visual Language Models. arXiv:2312.07533 [cs.CV]

    Pith/arXiv arXiv 2023

  41. [45]

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. 2024. Video-LLaVA: Learning United Visual Representation by Alignment Before Pro- jection. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Ed...

    doi:10.18653/v1/2024.emnlp-main.342 2024

  42. [46]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. InComputer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V (Lecture Notes in Computer Science, Vol. 8693), David J. Fl...

    doi:10.1007/978-3-319-10602-1_48 2014

  43. [47]

    Kevin Lin, Faisal Ahmed, Linjie Li, Chung-Ching Lin, Ehsan Azarnasab, Zhengyuan Yang, Jianfeng Wang, Lin Liang, Zicheng Liu, Yumao Lu, Ce Liu, and Lijuan Wang. 2023. MM-VID: Advancing Video Understanding with GPT- 4V(ision).CoRRabs/2310.19773 (2023). arXiv:2310.19773 doi:10.48550/ARXIV. 2310.19773

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.19773 2023

  44. [48]

    Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2023. HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Chal- lenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models.CoRR abs/2310.14566 (2023). arXiv:2310.14566 doi:10.48550/ARXIV.2310.14566

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.14566 2023

  45. [49]

    Yuanze Lin, Xun Guo, and Yan Lu. 2021. Self-Supervised Video Representation Learning with Meta-Contrastive Network. In2021 IEEE/CVF International Con- ference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. IEEE, 8219–8229. doi:10.1109/ICCV48922.2021.00813

    arXiv 2021

  46. [50]

    Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. 2024. TempCompass: Do Video LLMs Really Understand Videos?. InFindings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association...

    doi:10.18653/v1/2024.findings-acl.517 2024

  47. [51]

    Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, and Xirong Li. 2024. PhD: A Prompted Visual Hallucination Evaluation Dataset.CoRRabs/2403.11116 (2024). arXiv:2403.11116 doi:10.48550/ ARXIV.2403.11116

    Pith/arXiv arXiv doi:10.48550/arxiv.2403.11116 2024

  48. [52]

    Kaili Ma, Garry Yang, Han Yang, Yongqiang Chen, and James Cheng. 2023. Calibrating and Improving Graph Contrastive Learning.Trans. Mach. Learn. Res. 2023 (2023). https://openreview.net/forum?id=LdSP6cvTS4

    2023

  49. [53]

    Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao

  50. [54]

    Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution.arXiv preprint arXiv:2409.12961(2024)

    Pith/arXiv arXiv 2024

  51. [55]

    O’Connor

    Mayug Maniparambil, Raiymbek Akshulakov, Yasser Abdelaziz Dahou Dji- lali, Mohamed El Amine Seddik, Sanath Narayan, Karttikeya Mangalam, and Noel E. O’Connor. 2024. Do Vision and Language Encoders Represent the World Similarly?. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, W A, USA, June 16-22, 2024. IEEE, 14334–14...

    arXiv 2024

  52. [56]

    Rawat, and Thomas B

    Neelu Madan, Andreas Møgelmose, Rajat Modi, Yogesh S. Rawat, and Thomas B. Moeslund. 2024. Foundation Models for Video Understanding: A Survey.CoRR abs/2405.03770 (2024). arXiv:2405.03770 doi:10.48550/ARXIV.2405.03770

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.03770 2024

  53. [57]

    2016.From Human Attention to Computational Attention

    Matei Mancas, Vincent P Ferrera, Nicolas Riche, and John G Taylor. 2016.From Human Attention to Computational Attention. Vol. 2. Springer

    2016

  54. [58]

    Thong Nguyen, Yi Bin, Junbin Xiao, Leigang Qu, Yicong Li, Jay Zhangjie Wu, Cong-Duy Nguyen, See-Kiong Ng, and Anh Tuan Luu. 2024. Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives. InFindings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, ...

    work page doi:10.18653/v1/2024.findin 2024

  55. [59]

    1999.Foundations of statistical natural language processing

    Christopher Manning and Hinrich Schutze. 1999.Foundations of statistical natural language processing. MIT press

    1999

  56. [60]

    Shervin Minaee, Tomás Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large Language Models: A Survey.CoRRabs/2402.06196 (2024). arXiv:2402.06196 doi:10.48550/ARXIV.2402. 06196

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.06196 2024

  57. [61]

    Viorica Patraucean, Lucas Smaira, Ankush Gupta, Adrià Recasens, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Joseph Heyward, Mateusz Ma- linowski, Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, An- toine Miech, Alexandre Fréchette, Hanna Klimczak, Raphael Koster, Junlin Zhang, Stephanie Winkler, Yusuf Aytar, Simon Osindero, Dima Damen, An- d...

    2023

  58. [62]

    Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, and Li Yuan. 2023. Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models.CoRRabs/2311.16103 (2023). arXiv:2311.16103 doi:10.48550/ARXIV.2311.16103

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.16103 2023

  59. [63]

    OpenAI. 2023. GPT-4 Technical Report.CoRRabs/2303.08774 (2023). arXiv:2303.08774 doi:10.48550/ARXIV.2303.08774

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.08774 2023

  60. [64]

    G. Rose. 2007.Visual Methodologies: An Introduction to the Interpretation of Visual Materials. SAGE Publications. https://books.google.com.hk/books?id= gnUPNcnYjcIC

    2007

  61. [65]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Mod- els From Natural Language Supervision. InProceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 Jul...

    2021

  62. [66]

    Girshick, and Ali Farhadi

    Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi

  63. [67]

    Darshana Saravanan, Darshan Singh S, Varun Gupta, Zeeshan Khan, Vineet Gandhi, and Makarand Tapaswi. 2024. VELOCITI: Can Video-Language Mod- els Bind Semantic Concepts through Time?CoRRabs/2406.10889 (2024). arXiv:2406.10889 doi:10.48550/ARXIV.2406.10889

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2406.10889 2024

  64. [68]

    Yan Shu, Peitian Zhang, Zheng Liu, Minghao Qin, Junjie Zhou, Tiejun Huang, and Bo Zhao. 2024. Video-xl: Extra-long vision language model for hour-scale video understanding.arXiv preprint arXiv:2409.14485(2024)

    Pith/arXiv arXiv 2024

  65. [69]

    Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. A Comprehensive Survey of Hallucination in Large Lan- guage, Image, Video and Audio Foundation Models. InFindings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nu...

    2024

  66. [70]

    Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. Unveiling Hallucination in Text, Image, Video, and Audio Foundation Models: A Comprehensive Survey.CoRRabs/2405.09589 (2024). arXiv:2405.09589 doi:10.48550/ARXIV.2405.09589

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.09589 2024

  67. [71]

    Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, and Rita Cucchiara. 2023. From Show to Tell: A Survey on Deep Learning-Based Image Captioning.IEEE Trans. Pattern Anal. Mach. Intell.45, 1 (2023), 539–559. doi:10.1109/TPAMI.2022.3148210

    arXiv 2023

  68. [72]

    Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, Ali Vosoughi, Chao Huang, Zeliang Zhang, Feng Zheng, Jianguo Zhang, Ping Luo, Jiebo Luo, and Chenliang Xu

  69. [74]

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. 2012. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. arXiv:1212.0402 [cs.CV] https://arxiv.org/abs/1212.0402

    Pith/arXiv arXiv 2012

  70. [75]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lam- ple. 2023. LLaMA: Open and Efficient Foundation Language Models.CoRR abs/2302.13971 (2023). arXiv:2302.13971 doi:10.48550/ARXIV....

    Pith/arXiv arXiv doi:10.48550/arxiv.2302.13971 2023

  71. [76]

    2013.The Oxford handbook of sound and image in digital media

    Carol Vernallis, Amy Herzog, and John Richardson. 2013.The Oxford handbook of sound and image in digital media. OUP USA

    2013

  72. [77]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution. arXiv:2409.12191 [cs.CV] https://...

    Pith/arXiv arXiv 2024

  73. [78]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530(2024)

    Pith/arXiv arXiv 2024

  74. [79]

    2009.Multimodal Sig- nal Processing: Theory and applications for human-computer interaction

    Jean-Philippe Thiran, Ferran Marques, and Hervé Bourlard. 2009.Multimodal Sig- nal Processing: Theory and applications for human-computer interaction. Academic Press

    2009

  75. [80]

    Chenyu Yang, Xuan Dong, Xizhou Zhu, Weijie Su, Jiahao Wang, Hao Tian, Zhe Chen, Wenhai Wang, Lewei Lu, and Jifeng Dai. 2024. PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision- Language Models. arXiv:2412.09613 [cs.CV] https://arxiv.org/abs/2412.09613

    Pith/arXiv arXiv 2024

  76. [81]

    Dongjie Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. 2024. Vript: A Video Is Worth Thousands of Words. CoRRabs/2406.06040 (2024). arXiv:2406.06040 doi:10.48550/ARXIV.2406.06040

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.06040 2024

  77. [82]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and En- hong Chen. 2023. A Survey on Multimodal Large Language Models.CoRR abs/2306.13549 (2023). arXiv:2306.13549 doi:10.48550/ARXIV.2306.13549

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.13549 2023

  78. [83]

    Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng

  79. [84]

    arXiv:2406.16338 doi:10

    VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models.CoRRabs/2406.16338 (2024). arXiv:2406.16338 doi:10. 48550/ARXIV.2406.16338

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.16338 2024

  80. [85]

    T Wolf. 2019. Huggingface’s transformers: State-of-the-art natural language processing.arXiv preprint arXiv:1910.03771(2019)

    Pith/arXiv arXiv 2019

Showing first 80 references.