Pith. sign in

REVIEW 2 cited by

NeMoBench, an automatically generated benchmark with 31,378 QA pairs, shows that video LLMs struggle with temporal grounding of relevant clips hidden in long montages.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 13:51 UTC pith:ARIN6D55

arxiv 2509.24563 v3 pith:ARIN6D55 submitted 2025-09-29 cs.CV cs.CL

NeMo: Needle in a Montage for Video-Language Understanding

Zi-Yuan Hu , Shuo Liang , Duo Zheng , Yanyang Li , Yeyao Tao , Shijia Huang , Wei Feng , Jia Qin
show 5 more authors
Jianguang Yu Jing Huang Meng Fang Yin Li Liwei Wang
This is my paper
classification cs.CV cs.CL
keywords datanemobenchtaskunderstandingneedlepipelinetemporalvideo-language
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Video language models are often tested on short clips, but real videos are long and full of distractions. The needle in a haystack idea, common in text AI, hides one small relevant detail inside a huge amount of irrelevant text. NeMo adapts this to video. A needle is a short clip showing a specific object or scene, and the montage is a long synthetic video built by concatenating that clip with many other similar clips from the same source. The model is asked to output the start and end timestamps of the needle. This requires both remembering a detail across a long context and pinpointing exactly when it appears.

To avoid expensive manual labeling, the authors built a pipeline that first converts each video into a structured representation of scenes and objects using off-the-shelf vision models. A frozen large multimodal model (GPT-4o) then writes a natural language question about the selected object or scene, and the known timestamps become the answer. A self-verification step filters out low-quality questions. They evaluated 20 models and found that open-source VideoLLMs score near zero on long montages, while the best closed-source model (Gemini-1.5-Pro-002) reaches about 65% recall on short montages but still falls far behind human experts, who score above 90%. Model rankings are stable between the noisy full set and the manually cleaned subset, supporting the pipeline's reliability for ranking purposes.

Core claim

The paper's central assertion, stated in the abstract and Sec. 5, is that NeMoBench 'can reliably and automatically generate high-quality evaluation data' and that 'even the latest VideoLLMs exhibit a significant performance gap when compared to humans.' If true, the benchmark provides a scalable, contamination-resistant test of long-context recall and temporal grounding.

Load-bearing premise

The montage is constructed from 'loosely related' clips sampled from the same video source, with negative scenes excluded only by object-tag mismatch (Sec. 3.2.2, Stage 2). The benchmark's validity assumes this synthetic haystack does not leak the needle's location through detectable artifacts such as scene cuts, subtitle discontinuities, or object-tag mismatches. If models solve the task by exploiting such shortcuts rather than true temporal grounding, the benchmark would not measure what it claims. The paper's own shortcut baseline (Sec. 5.3), which segments the montage and grounds sequentially, improves open-source scores, suggesting such artifacts may exist.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The benchmark is a synthetic evaluation protocol. Its validity rests on the fidelity of the video representation, the reliability of GPT-4o as question generator and verifier, the absence of shortcut cues in the montage, the assumption of low contamination, and the standardness of the evaluation metrics. These are domain assumptions, not derivable facts, and the paper provides only partial empirical support for them.

free parameters (4)
  • minimum target scene duration = >5 seconds
    Hand-chosen threshold for selecting target scenes in the pipeline (Sec. 3.2.2 and Appendix D.2). Affects benchmark difficulty and the set of eligible needles.
  • minimum object visibility duration = >=4 seconds
    Hand-chosen threshold for object needles; objects visible less than 4s are excluded (Appendix D.2).
  • average visible area fraction range = 5%-25% of frame
    Hand-chosen range to pick visually prominent objects, affecting needle discrimination difficulty (Appendix D.2).
  • montage duration cutoffs = 2.5min and 15min
    Categorize montages into short/medium/long; hand-chosen boundaries (Sec. 3.1).
axioms (5)
  • domain assumption The automatic pipeline (RAM++, UniRef, DEVA) accurately segments scenes, tags objects, and tracks objects, so the video representation is a faithful summary of the video content.
    The entire benchmark construction relies on these pre-trained models; errors propagate to QA pairs (Appendix D.1).
  • domain assumption GPT-4o can reliably generate clearly worded, unambiguous needle grounding questions from visual prompts, and its self-verification reliably filters invalid samples.
    Question quality and filtering directly affect benchmark validity; Table 3 shows 82.5% question accuracy, so this is not perfectly reliable.
  • domain assumption Negative scenes sampled from the same video source with no target object tag constitute a challenging haystack that does not leak the needle location through artifacts such as scene cuts.
    If montage concatenation creates detectable boundaries, models could solve the task by artifact detection rather than temporal grounding; the shortcut baseline in Sec. 5.3 suggests such shortcuts exist.
  • domain assumption The videos from Phoenix TV are not included in VideoLLM training sets; the check covers a list of common datasets but not all possible sources.
    The paper claims reduced contamination risk but acknowledges only a curated check (Sec. 3.2.2, Stage 1).
  • standard math Evaluation metrics (Recall@1x, tIoU=0.7/0.5, Average mAP) are appropriate for multi-target temporal grounding.
    Standard metrics from Ego4D; generally accepted in the field (Sec. C.3).

pith-pipeline@v1.3.0-alltime-deepseek · 38471 in / 12768 out tokens · 95537 ms · 2026-08-04T13:51:33.868252+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of NeMo: Needle in a Montage for Video-Language Understanding." pith.science (2026). https://pith.science/paper/ARIN6D55

@misc{pith2026250924563,
  author       = {Pith},
  title        = {Pith review of: NeMo: Needle in a Montage for Video-Language Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ARIN6D55}},
  note         = {Machine review of arXiv:2509.24563}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for video-language understanding. Inspired by the needle in a haystack test widely used by LLMs, we introduce a novel task of Needle in a Montage (NeMo), which is designed to assess the temporal understanding capabilities of advanced VideoLLMs. Specifically, the proposed task focuses on two fundamental abilities critical for temporal understanding, i.e., retrieval-style long-context recall and temporal grounding. To generate video question answering data for our task, we develop a scalable automated data generation pipeline that facilitates high-quality data synthesis. Built upon the proposed pipeline, we present NeMoBench, a video-language benchmark centered on our task. Specifically, our full set of NeMoBench features 31,378 automatically generated question-answer (QA) pairs from 13,486 videos with various durations ranging from seconds to hours. Experiments demonstrate that our pipeline can reliably and automatically generate high-quality evaluation data, enabling NeMoBench to be continuously updated with the latest videos. We evaluate 20 state-of-the-art models on our benchmark, providing extensive results and key insights into their capabilities and limitations. Our project page is available at: https://lavi-lab.github.io/NeMoBench.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. M$^3$Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

    cs.CV 2026-06 unverdicted novelty 7.0

    M³Eval is a new cognitively-grounded benchmark that evaluates memory dimensions in multi-modal video models and reports consistent model weaknesses in disentanglement, interference, spatial-temporal grounding, and sym...

  2. Cambrian-S: Towards Spatial Supersensing in Video

    cs.CV 2025-11 unverdicted novelty 6.0

    Cambrian-S introduces VSI-SUPER benchmarks for long-horizon spatial recall and counting, shows data scaling yields 30% gains on existing tests, and demonstrates a self-supervised next-latent predictor using surprise o...

Reference graph

Works this paper leans on

98 extracted references · 23 linked inside Pith · cited by 2 Pith papers

  1. [1]

    , Bai , S

    botherref Bai , J. , Bai , S. , Yang , S. , Wang , S. , Tan , S. , Wang , P. , Lin , J. , Zhou , C. , Zhou , J. : Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 (2023) botherref

    Pith/arXiv arXiv 2023

  2. [2]

    , Chen , K

    botherref Bai , S. , Chen , K. , Liu , X. , Wang , J. , Ge , W. , Song , S. , Dang , K. , Wang , P. , Wang , S. , Tang , J. , Zhong , H. , Zhu , Y. , Yang , M. , Li , Z. , Wan , J. , Wang , P. , Ding , W. , Fu , Z. , Xu , Y. , Ye , J. , Zhang , X. , Xie , T. , Cheng , Z. , Zhang , H. , Yang , Z. , Xu , H. , Lin , J. : Qwen2.5-vl technical report. CoRR abs...

    Pith/arXiv arXiv 2025

  3. [3]

    , Eyzaguirre , C

    bchapter Buch , S. , Eyzaguirre , C. , Gaidon , A. , Wu , J. , Fei - Fei , L. , Niebles , J.C. : Revisiting the "video" in video-language understanding . In: CVPR ( 2022 ) bchapter

    2022

  4. [4]

    , Mann , B

    bchapter Brown , T.B. , Mann , B. , Ryder , N. , Subbiah , M. , Kaplan , J. , Dhariwal , P. , : Language models are few-shot learners . In: NeurIPS ( 2020 ) bchapter

    2020

  5. [5]

    , Nagrani , A

    bchapter Bain , M. , Nagrani , A. , Varol , G. , Zisserman , A. : Frozen in time: A joint video and image encoder for end-to-end retrieval . In: ICCV ( 2021 ) bchapter

    2021

  6. [6]

    , Gupta , A

    bchapter Chandrasegaran , K. , Gupta , A. , Hadzic , L.M. , Kota , T. , He , J. , Eyzaguirre , C. , Durante , Z. , Li , M. , Wu , J. , Fei - Fei , L. : Hourvideo: 1-hour video-language understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  7. [7]

    , Liu , Y

    bchapter Chen , G. , Liu , Y. , Huang , Y. , Pei , B. , Xu , J. , He , Y. , Lu , T. , Wang , Y. , Wang , L. : Cg-bench: Clue-grounded question answering benchmark for long video understanding . In: ICLR ( 2025 ) bchapter

    2025

  8. [8]

    , Liao , Y

    bchapter Chen , J. , Liao , Y. , Lin , H. , Yu , Y. , Chen , Y. , Wang , Y.F. : Rextime: A benchmark suite for reasoning-across-time in videos . In: NeurIPS ( 2024 ) bchapter

    2024

  9. [9]

    , Lin , Y

    bchapter Chen , X. , Lin , Y. , Zhang , Y. , Huang , W. : Autoeval-video: An automatic benchmark for assessing large vision language models in open-ended video question answering . In: ECCV ( 2024 ) bchapter

    2024

  10. [10]

    , Oh , S.W

    bchapter Cheng , H.K. , Oh , S.W. , Price , B. , Schwing , A.G. , Lee , J. : Tracking anything with decoupled video segmentation . In: ICCV ( 2023 ) bchapter

    2023

  11. [11]

    , Siarohin , A

    bchapter Chen , T. , Siarohin , A. , Menapace , W. , Deyneka , E. , Chao , H. , Jeon , B.E. , Fang , Y. , Lee , H. , Ren , J. , Yang , M. , Tulyakov , S. : Panda-70m: Captioning 70m videos with multiple cross-modality teachers . In: CVPR ( 2024 ) bchapter

    2024

  12. [12]

    , Tan , R

    botherref Cai , M. , Tan , R. , Zhang , J. , Zou , B. , Zhang , K. , Yao , F. , Zhu , F. , Gu , J. , Zhong , Y. , Shang , Y. , Dou , Y. , Park , J. , Gao , J. , Lee , Y.J. , Yang , J. : Temporalbench: Benchmarking fine-grained temporal understanding for multimodal video models. CoRR abs/2410.10818 (2024) botherref

    Pith/arXiv arXiv 2024

  13. [13]

    , Wei , X

    bchapter Chen , L. , Wei , X. , Li , J. , Dong , X. , Zhang , P. , Zang , Y. , Chen , Z. , Duan , H. , Bin , L. , Tang , Z. , Yuan , L. , Qiao , Y. , Lin , D. , Zhao , F. , Wang , J. : Sharegpt4video: Improving video understanding and generation with better captions . In: NeurIPS ( 2024 ) bchapter

    2024

  14. [14]

    , Wu , J

    botherref Chen , Z. , Wu , J. , Wang , W. , Su , W. , Chen , G. , Xing , S. , Zhong , M. , Zhang , Q. , Zhu , X. , Lu , L. , Li , B. , Luo , P. , Lu , T. , Qiao , Y. , Dai , J. : Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. CoRR abs/2312.14238 (2023) botherref

    Pith/arXiv arXiv 2023

  15. [15]

    , Xue , F

    bchapter Chen , Y. , Xue , F. , Li , D. , Hu , Q. , Zhu , L. , Li , X. , Fang , Y. , Tang , H. , Yang , S. , Liu , Z. , He , Y. , Yin , H. , Molchanov , P. , Kautz , J. , Fan , L. , Zhu , Y. , Lu , Y. , Han , S. : Longvila: Scaling long-context visual language models for long videos . In: ICLR ( 2025 ) bchapter

    2025

  16. [16]

    , Dai , Y

    bchapter Fu , C. , Dai , Y. , Luo , Y. , Li , L. , Ren , S. , Zhang , R. , Wang , Z. , Zhou , C. , Shen , Y. , Zhang , M. , Chen , P. , Li , Y. , Lin , S. , Zhao , S. , Li , K. , Xu , T. , Zheng , X. , Chen , E. , Shan , C. , He , R. , Sun , X. : Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis . In: CVPR ...

    2025

  17. [17]

    , Hua , W

    bchapter Fan , L. , Hua , W. , Li , L. , Ling , H. , Zhang , Y. : Nphardeval: Dynamic benchmark on reasoning ability of large language models via complexity classes . In: ACL ( 2024 ) bchapter

    2024

  18. [18]

    , Mao , K

    bchapter Fang , X. , Mao , K. , Duan , H. , Zhao , X. , Li , Y. , Lin , D. , Chen , K. : Mmbench-video: A long-form multi-shot benchmark for holistic video understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  19. [19]

    , Wang , W

    bchapter Fang , Y. , Wang , W. , Xie , B. , Sun , Q. , Wu , L. , Wang , X. , Huang , T. , Wang , X. , Cao , Y. : EVA: exploring the limits of masked visual representation learning at scale . In: CVPR ( 2023 ) bchapter

    2023

  20. [20]

    u nd , I. , Yianilos , P. , Mueller - Freitag , M. , Hoppe , F. , Thurau , C. , Bax , I. , Memisevic , R. : The

    bchapter Goyal , R. , Kahou , S.E. , Michalski , V. , Materzynska , J. , Westphal , S. , Kim , H. , Haenel , V. , Fr \" u nd , I. , Yianilos , P. , Mueller - Freitag , M. , Hoppe , F. , Thurau , C. , Bax , I. , Memisevic , R. : The "something something" video database for learning and evaluating visual common sense . In: ICCV ( 2017 ) bchapter

    2017

  21. [21]

    , Sun , C

    bchapter Gao , J. , Sun , C. , Yang , Z. , Nevatia , R. : Tall: Temporal activity localization via language query . In: ICCV ( 2017 ) bchapter

    2017

  22. [22]

    , Westbury , A

    bchapter Grauman , K. , Westbury , A. , Byrne , E. , Chavis , Z. , Furnari , A. , Girdhar , R. , Hamburger , J. , Jiang , H. , Liu , M. , Liu , X. , : Ego4d: Around the world in 3,000 hours of egocentric video . In: CVPR ( 2022 ) bchapter

    2022

  23. [23]

    , Escorcia , V

    bchapter Heilbron , F.C. , Escorcia , V. , Ghanem , B. , Niebles , J.C. : Activitynet: A large-scale video benchmark for human activity understanding . In: CVPR ( 2015 ) bchapter

    2015

  24. [24]

    , Feng , W

    bchapter He , X. , Feng , W. , Zheng , K. , Lu , Y. , Zhu , W. , Li , J. , Fan , Y. , Wang , J. , Li , L. , Yang , Z. , Lin , K. , Wang , W.Y. , Wang , L. , Wang , X.E. : Mmworld: Towards multi-discipline multi-faceted world model evaluation in videos . In: ICLR ( 2025 ) bchapter

    2025

  25. [25]

    , Huang , Y.-J

    botherref Huang , X. , Huang , Y.-J. , Zhang , Y. , Tian , W. , Feng , R. , Zhang , Y. , Xie , Y. , Li , Y. , Zhang , L. : Open-set image tagging with multi-grained text supervision. arXiv preprint arXiv:2310.15200 (2023) botherref

    Pith/arXiv arXiv 2023

  26. [26]

    , Sun , S

    botherref Hsieh , C. , Sun , S. , Kriman , S. , Acharya , S. , Rekesh , D. , Jia , F. , Zhang , Y. , Ginsburg , B. : RULER: what's the real context size of your long-context language models? CoRR abs/2404.06654 (2024) botherref

    Pith/arXiv arXiv 2024

  27. [27]

    , Wang , X

    bchapter Huang , B. , Wang , X. , Chen , H. , Song , Z. , Zhu , W. : Vtimellm: Empower LLM to grasp video moments . In: CVPR ( 2024 ) bchapter

    2024

  28. [28]

    , Wu , P

    botherref Hu , K. , Wu , P. , Pu , F. , Xiao , W. , Zhang , Y. , Yue , X. , Li , B. , Liu , Z. : Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. CoRR abs/2501.13826 (2025) botherref

    Pith/arXiv arXiv 2025

  29. [29]

    , Zhong , Y

    bchapter Hu , Z. , Zhong , Y. , Huang , S. , Lyu , M.R. , Wang , L. : Enhancing temporal modeling of video llms via time gating . In: EMNLP (Findings) ( 2024 ) bchapter

    2024

  30. [30]

    , Gimeno , F

    bchapter Jaegle , A. , Gimeno , F. , Brock , A. , Vinyals , O. , Zisserman , A. , Carreira , J. : Perceiver: General perception with iterative attention . In: ICML ( 2021 ) bchapter

    2021

  31. [31]

    , Song , Y

    bchapter Jang , Y. , Song , Y. , Yu , Y. , Kim , Y. , Kim , G. : Tgif-qa: Toward spatio-temporal reasoning in visual question answering . In: CVPR ( 2017 ) bchapter

    2017

  32. [32]

    : Needle in a haystack-pressure testing llms

    botherref Kamradt , G. : Needle in a haystack-pressure testing llms. https://github.com/gkamradt/LLMTest_NeedleInAHaystack (2023) botherref

    2023

  33. [33]

    , Bulatov , A

    bchapter Kuratov , Y. , Bulatov , A. , Anokhin , P. , Rodkin , I. , Sorokin , D. , Sorokin , A.Y. , Burtsev , M. : Babilong: Testing the limits of llms with long context reasoning-in-a-haystack . In: NeurIPS ( 2024 ) bchapter

    2024

  34. [34]

    , Bulatov , A

    botherref Kuratov , Y. , Bulatov , A. , Anokhin , P. , Sorokin , D. , Sorokin , A.Y. , Burtsev , M. : In search of needles in a 11m haystack: Recurrent memory finds what llms miss. CoRR abs/2402.10790 (2024) botherref

    Pith/arXiv arXiv 2024

  35. [35]

    , Berg , T.L

    botherref Lei , J. , Berg , T.L. , Bansal , M. : Detecting moments and highlights in videos via natural language queries. NeurIPS (2021) botherref

    2021

  36. [36]

    , Chen , X

    botherref Li , Y. , Chen , X. , Hu , B. , Wang , L. , Shi , H. , Zhang , M. : Videovista: A versatile benchmark for video understanding and reasoning. CoRR abs/2406.11303 (2024) botherref

    Pith/arXiv arXiv 2024

  37. [37]

    , Dong , Y

    bchapter Liu , Z. , Dong , Y. , Liu , Z. , Hu , W. , Lu , J. , Rao , Y. : Oryx MLLM: on-demand spatial-temporal understanding at arbitrary resolution . In: ICLR ( 2025 ) bchapter

    2025

  38. [38]

    , Ge , Y

    bchapter Li , B. , Ge , Y. , Ge , Y. , Wang , G. , Wang , R. , Zhang , R. , Shan , Y. : Seed-bench: Benchmarking multimodal large language models . In: CVPR ( 2024 ) bchapter

    2024

  39. [39]

    , He , Y

    botherref Li , K. , He , Y. , Wang , Y. , Li , Y. , Wang , W. , Luo , P. , Wang , Y. , Wang , L. , Qiao , Y. : Videochat: Chat-centric video understanding. CoRR abs/2305.06355 (2023) botherref

    Pith/arXiv arXiv 2023

  40. [40]

    , Han , P

    botherref Liu , Z. , Han , P. , Yu , H. , Li , H. , You , J. : Time-r1: Towards comprehensive temporal reasoning in llms. CoRR abs/2505.13508 (2025) botherref

    Pith/arXiv arXiv 2025

  41. [41]

    , Li , S

    bchapter Liu , Y. , Li , S. , Liu , Y. , Wang , Y. , Ren , S. , Li , L. , Chen , S. , Sun , X. , Hou , L. : Tempcompass: Do video llms really understand videos? In: ACL (Findings) ( 2024 ) bchapter

    2024

  42. [42]

    , Ma , Z

    bchapter Liu , Y. , Ma , Z. , Qi , Z. , Wu , Y. , Shan , Y. , Chen , C.W. : E.T. bench: Towards open-ended event-level video-language understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  43. [43]

    , Wang , Y

    bchapter Li , K. , Wang , Y. , He , Y. , Li , Y. , Wang , Y. , Liu , Y. , Wang , Z. , Xu , J. , Chen , G. , Lou , P. , Wang , L. , Qiao , Y. : Mvbench: A comprehensive multi-modal video understanding benchmark . In: CVPR ( 2024 ) bchapter

    2024

  44. [44]

    , Wong , T.L

    bchapter Li , Y. , Wong , T.L. , Hung , C.T. , Zhao , J. , Zheng , D. , Liu , K.W. , Lyu , M.R. , Wang , L. : C\( ^ 2 \)leva: Toward comprehensive and contamination-free language model evaluation . In: ACL (Findings) ( 2025 ) bchapter

    2025

  45. [45]

    , Wu , H

    bchapter Luo , Z. , Wu , H. , Li , D. , Ma , J. , Kankanhalli , M.S. , Li , J. : Videoautoarena: An automated arena for evaluating large multimodal models in video analysis through user simulation . In: CVPR ( 2025 ) bchapter

    2025

  46. [46]

    , Yin , H

    botherref Lin , J. , Yin , H. , Ping , W. , Lu , Y. , Molchanov , P. , Tao , A. , Mao , H. , Kautz , J. , Shoeybi , M. , Han , S. : Vila: On pre-training for visual language models. CVPR (2024) botherref

    2024

  47. [47]

    , Ye , Y

    bchapter Lin , B. , Ye , Y. , Zhu , B. , Cui , J. , Ning , M. , Jin , P. , Yuan , L. : Video-llava: Learning united visual representation by alignment before projection . In: EMNLP ( 2024 ) bchapter

    2024

  48. [48]

    , Zhang , P

    bchapter Lin , K.Q. , Zhang , P. , Chen , J. , Pramanick , S. , Gao , D. , Wang , A.J. , Yan , R. , Shou , M.Z. : Univtg: Towards unified video-language temporal grounding . In: ICCV ( 2023 ) bchapter

    2023

  49. [49]

    , Zhang , Y

    botherref Li , B. , Zhang , Y. , Guo , D. , Zhang , R. , Li , F. , Zhang , H. , Zhang , K. , Zhang , P. , Li , Y. , Liu , Z. , Li , C. : Llava-onevision: Easy visual task transfer. Trans. Mach. Learn. Res. (2025) botherref

    2025

  50. [50]

    , Zheng , X

    botherref Luo , Y. , Zheng , X. , Yang , X. , Li , G. , Lin , H. , Huang , J. , Ji , J. , Chao , F. , Luo , J. , Ji , R. : Video-rag: Visually-aligned retrieval-augmented long video comprehension. CoRR abs/2411.13093 (2024) botherref

    arXiv 2024

  51. [51]

    , Zhang , Y

    botherref Li , S. , Zhang , Y. , Zhao , Y. , Wang , Q. , Jia , F. , Liu , Y. , Wang , T. : Vlm-eval: A general evaluation on video large language models. CoRR abs/2311.11865 (2023) botherref

    Pith/arXiv arXiv 2023

  52. [52]

    , Akshulakov , R

    bchapter Mangalam , K. , Akshulakov , R. , Malik , J. : Egoschema: A diagnostic benchmark for very long-form video language understanding . In: NeurIPS ( 2023 ) bchapter

    2023

  53. [53]

    , Rasheed , H.A

    bchapter Maaz , M. , Rasheed , H.A. , Khan , S. , Khan , F. : Video-chatgpt: Towards detailed video understanding via large vision and language models . In: ACL ( 2024 ) bchapter

    2024

  54. [54]

    , Zhu , B

    botherref Ning , M. , Zhu , B. , Xie , Y. , Lin , B. , Cui , J. , Yuan , L. , Chen , D. , Yuan , L. : Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models. CoRR abs/2311.16103 (2023) botherref

    Pith/arXiv arXiv 2023

  55. [55]

    CoRR abs/2303.08774 (2023) botherref

    botherref OpenAI : GPT-4 technical report. CoRR abs/2303.08774 (2023) botherref

    Pith/arXiv arXiv 2023

  56. [56]

    https://openai.com/index/hello-gpt-4o/

    botherref OpenAI : Hello GPT-4o. https://openai.com/index/hello-gpt-4o/. Accessed: 2024-07-29 (2024) botherref

    2024

  57. [57]

    , Smaira , L

    bchapter Patraucean , V. , Smaira , L. , Gupta , A. , Recasens , A. , Markeeva , L. , Banarse , D. , Koppula , S. , Heyward , J. , Malinowski , M. , Yang , Y. , Doersch , C. , Matejovicova , T. , Sulsky , Y. , Miech , A. , Fr \' e chette , A. , Klimczak , H. , Koster , R. , Zhang , J. , Winkler , S. , Aytar , Y. , Osindero , S. , Damen , D. , Zisserman , ...

    2023

  58. [58]

    , Li , J

    bchapter Qian , L. , Li , J. , Wu , Y. , Ye , Y. , Fei , H. , Chua , T. , Zhuang , Y. , Tang , S. : Momentor: Advancing video large language model with fine-grained temporal reasoning . In: ICML ( 2024 ) bchapter

    2024

  59. [59]

    , Kim , J.W

    bchapter Radford , A. , Kim , J.W. , Hallacy , C. , Ramesh , A. , Goh , G. , Agarwal , S. , Sastry , G. , Askell , A. , Mishkin , P. , Clark , J. , Krueger , G. , Sutskever , I. : Learning transferable visual models from natural language supervision . In: ICML ( 2021 ) bchapter

    2021

  60. [60]

    , Rohrbach , M

    botherref Regneri , M. , Rohrbach , M. , Wetzel , D. , Thater , S. , Schiele , B. , Pinkal , M. : Grounding action descriptions in videos. Trans. Assoc. Comput. Linguistics (2013) botherref

    2013

  61. [61]

    , Savinov , N

    botherref Reid , M. , Savinov , N. , Teplyashin , D. , Lepikhin , D. , Lillicrap , T.P. , Alayrac , J. , et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. CoRR abs/2403.05530 (2024) botherref

    Pith/arXiv arXiv 2024

  62. [62]

    , Yao , L

    bchapter Ren , S. , Yao , L. , Li , S. , Sun , X. , Hou , L. : Timechat: A time-sensitive multimodal large language model for long video understanding . In: CVPR ( 2024 ) bchapter

    2024

  63. [63]

    , Chai , W

    bchapter Song , E. , Chai , W. , Wang , G. , Zhang , Y. , Zhou , H. , Wu , F. , Chi , H. , Guo , X. , Ye , T. , Zhang , Y. , Lu , Y. , Hwang , J. , Wang , G. : Moviechat: From dense token to sparse memory for long video understanding . In: CVPR ( 2024 ) bchapter

    2024

  64. [64]

    , Di , S

    bchapter Shi , Y. , Di , S. , Chen , Q. , Xie , W. : Enhancing video-llm reasoning via agent-of-thoughts distillation . In: CVPR ( 2025 ) bchapter

    2025

  65. [65]

    , Xiong , Y

    bchapter Shen , X. , Xiong , Y. , Zhao , C. , Wu , L. , Chen , J. , Zhu , C. , Liu , Z. , Xiao , F. , Varadarajan , B. , Bordes , F. , Liu , Z. , Xu , H. , J. Kim , H. , Soran , B. , Krishnamoorthi , R. , Elhoseiny , M. , Chandra , V. : Longvu: Spatiotemporal adaptive compression for long video-language understanding . In: ICML ( 2025 ) bchapter

    2025

  66. [66]

    , Zheng , M

    botherref Song , M. , Zheng , M. , Luo , X. : Counting-stars: A simple, efficient, and reasonable strategy for evaluating long-context large language models. CoRR abs/2403.11802 (2024) botherref

    Pith/arXiv arXiv 2024

  67. [67]

    , Lavril , T

    botherref Touvron , H. , Lavril , T. , Izacard , G. , Martinet , X. , Lachaux , M. , Lacroix , T. , Rozi \` e re , B. , Goyal , N. , Hambro , E. , Azhar , F. , Rodriguez , A. , Joulin , A. , Grave , E. , Lample , G. : Llama: Open and efficient foundation language models. CoRR abs/2302.13971 (2023) botherref

    Pith/arXiv arXiv 2023

  68. [68]

    , Menick , J

    bchapter Tsimpoukelli , M. , Menick , J. , Cabi , S. , Eslami , S.M.A. , Vinyals , O. , Hill , F. : Multimodal few-shot learning with frozen language models . In: NeurIPS ( 2021 ) bchapter

    2021

  69. [69]

    , Bai , S

    botherref Wang , P. , Bai , S. , Tan , S. , Wang , S. , Fan , Z. , Bai , J. , Chen , K. , Liu , X. , Wang , J. , Ge , W. , Fan , Y. , Dang , K. , Du , M. , Ren , X. , Men , R. , Liu , D. , Zhou , C. , Zhou , J. , Lin , J. : Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. CoRR abs/2409.12191 (2024) botherref

    Pith/arXiv arXiv 2024

  70. [70]

    , He , Z

    bchapter Wang , W. , He , Z. , Hong , W. , Cheng , Y. , Zhang , X. , Qi , J. , Huang , S. , Xu , B. , Dong , Y. , Ding , M. , Tang , J. : Lvbench: An extreme long video understanding benchmark . In: ICCV ( 2025 ) bchapter

    2025

  71. [71]

    , He , Y

    bchapter Wang , Y. , He , Y. , Li , Y. , Li , K. , Yu , J. , Ma , X. , Li , X. , Chen , G. , Chen , X. , Wang , Y. , Luo , P. , Liu , Z. , Wang , Y. , Wang , L. , Qiao , Y. : Internvid: A large-scale video-text dataset for multimodal understanding and generation . In: ICLR ( 2024 ) bchapter

    2024

  72. [72]

    , Li , D

    bchapter Wu , H. , Li , D. , Chen , B. , Li , J. : Longvideobench: A benchmark for long-context interleaved video-language understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  73. [73]

    , Hang , T

    bchapter Xue , H. , Hang , T. , Zeng , Y. , Sun , Y. , Liu , B. , Yang , H. , Fu , J. , Guo , B. : Advancing high-resolution video-language representation with large-scale video transcriptions . In: CVPR ( 2022 ) bchapter

    2022

  74. [74]

    , Shang , X

    bchapter Xiao , J. , Shang , X. , Yao , A. , Chua , T.-S. : Next-qa: Next phase of question-answering to explaining temporal actions . In: CVPR ( 2021 ) bchapter

    2021

  75. [75]

    , Zhao , Z

    bchapter Xu , D. , Zhao , Z. , Xiao , J. , Wu , F. , Zhang , H. , He , X. , Zhuang , Y. : Video question answering via gradually refined attention over appearance and motion . In: ACM Multimedia ( 2017 ) bchapter

    2017

  76. [76]

    , Gan , C

    bchapter Yi , K. , Gan , C. , Li , Y. , Kohli , P. , Wu , J. , Torralba , A. , Tenenbaum , J.B. : CLEVRER: collision events for video representation and reasoning . In: ICLR ( 2020 ) bchapter

    2020

  77. [77]

    , Wu , Y

    bchapter Yu , J. , Wu , Y. , Chu , M. , Ren , Z. , Huang , Z. , Chu , P. , Zhang , R. , He , Y. , Li , Q. , Li , S. , Li , Z. , Tu , Z. , He , C. , Qiao , Y. , Wang , Y. , Wang , Y. , Wang , L. : Vrbench: A benchmark for multi-step reasoning in long narrative videos . In: ICCV ( 2025 ) bchapter

    2025

  78. [78]

    , Wang , Z

    bchapter Ye , J. , Wang , Z. , Sun , H. , Chandrasegaran , K. , Durante , Z. , Eyzaguirre , C. , Bisk , Y. , Niebles , J.C. , Adeli , E. , Fei - Fei , L. , Wu , J. , Li , M. : Re-thinking temporal search for long-form video understanding . In: CVPR ( 2025 ) bchapter

    2025

  79. [79]

    , Wang , J

    botherref Yuan , L. , Wang , J. , Sun , H. , Zhang , Y. , Lin , Y. : Tarsier2: Advancing large vision-language models from detailed video description to comprehensive video understanding. CoRR abs/2501.07888 (2025) botherref

    Pith/arXiv arXiv 2025

  80. [80]

    , Xu , D

    bchapter Yu , Z. , Xu , D. , Yu , J. , Yu , T. , Zhao , Z. , Zhuang , Y. , Tao , D. : Activitynet-qa: A dataset for understanding complex web videos via question answering . In: AAAI ( 2019 ) bchapter

    2019

Showing first 80 references.