Pith. sign in

REVIEW 2 cited by

NeMo: Needle in a Montage for Video-Language Understanding

T0 review · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read NeMoBench, an automatically generated benchmark with 31,378 QA pairs, shows that video LLMs struggle with temporal grounding of relevant clips hidden in long montages.

arxiv 2509.24563 v3 pith:ARIN6D55 submitted 2025-09-29 cs.CV cs.CL

Zi-Yuan Hu , Shuo Liang , Duo Zheng , Yanyang Li , Yeyao Tao , Shijia Huang , Wei Feng , Jia Qin
show 5 more authors
Jianguang Yu Jing Huang Meng Fang Yin Li Liwei Wang
This is my paper
classification cs.CVcs.CL
keywords datanemobenchtaskunderstandingneedlepipelinetemporalvideo-language
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video language models are often tested on short clips, but real videos are long and full of distractions. The needle in a haystack idea, common in text AI, hides one small relevant detail inside a huge amount of irrelevant text. NeMo adapts this to video. A needle is a short clip showing a specific object or scene, and the montage is a long synthetic video built by concatenating that clip with many other similar clips from the same source. The model is asked to output the start and end timestamps of the needle. This requires both remembering a detail across a long context and pinpointing exactly when it appears.

To avoid expensive manual labeling, the authors built a pipeline that first converts each video into a structured representation of scenes and objects using off-the-shelf vision models. A frozen large multimodal model (GPT-4o) then writes a natural language question about the selected object or scene, and the known timestamps become the answer. A self-verification step filters out low-quality questions. They evaluated 20 models and found that open-source VideoLLMs score near zero on long montages, while the best closed-source model (Gemini-1.5-Pro-002) reaches about 65% recall on short montages but still falls far behind human experts, who score above 90%. Model rankings are stable between the noisy full set and the manually cleaned subset, supporting the pipeline's reliability for ranking purposes.

Extended reading notes

Core claim

The paper's central assertion, stated in the abstract and Sec. 5, is that NeMoBench 'can reliably and automatically generate high-quality evaluation data' and that 'even the latest VideoLLMs exhibit a significant performance gap when compared to humans.' If true, the benchmark provides a scalable, contamination-resistant test of long-context recall and temporal grounding.

Load-bearing premise

The montage is constructed from 'loosely related' clips sampled from the same video source, with negative scenes excluded only by object-tag mismatch (Sec. 3.2.2, Stage 2). The benchmark's validity assumes this synthetic haystack does not leak the needle's location through detectable artifacts such as scene cuts, subtitle discontinuities, or object-tag mismatches. If models solve the task by exploiting such shortcuts rather than true temporal grounding, the benchmark would not measure what it claims. The paper's own shortcut baseline (Sec. 5.3), which segments the montage and grounds sequentially, improves open-source scores, suggesting such artifacts may exist.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The benchmark is a synthetic evaluation protocol. Its validity rests on the fidelity of the video representation, the reliability of GPT-4o as question generator and verifier, the absence of shortcut cues in the montage, the assumption of low contamination, and the standardness of the evaluation metrics. These are domain assumptions, not derivable facts, and the paper provides only partial empirical support for them.

free parameters (4)
  • minimum target scene duration = >5 seconds
    Hand-chosen threshold for selecting target scenes in the pipeline (Sec. 3.2.2 and Appendix D.2). Affects benchmark difficulty and the set of eligible needles.
  • minimum object visibility duration = >=4 seconds
    Hand-chosen threshold for object needles; objects visible less than 4s are excluded (Appendix D.2).
  • average visible area fraction range = 5%-25% of frame
    Hand-chosen range to pick visually prominent objects, affecting needle discrimination difficulty (Appendix D.2).
  • montage duration cutoffs = 2.5min and 15min
    Categorize montages into short/medium/long; hand-chosen boundaries (Sec. 3.1).
assumptions (5)
  • domain assumption The automatic pipeline (RAM++, UniRef, DEVA) accurately segments scenes, tags objects, and tracks objects, so the video representation is a faithful summary of the video content.
    The entire benchmark construction relies on these pre-trained models; errors propagate to QA pairs (Appendix D.1).
  • domain assumption GPT-4o can reliably generate clearly worded, unambiguous needle grounding questions from visual prompts, and its self-verification reliably filters invalid samples.
    Question quality and filtering directly affect benchmark validity; Table 3 shows 82.5% question accuracy, so this is not perfectly reliable.
  • domain assumption Negative scenes sampled from the same video source with no target object tag constitute a challenging haystack that does not leak the needle location through artifacts such as scene cuts.
    If montage concatenation creates detectable boundaries, models could solve the task by artifact detection rather than temporal grounding; the shortcut baseline in Sec. 5.3 suggests such shortcuts exist.
  • domain assumption The videos from Phoenix TV are not included in VideoLLM training sets; the check covers a list of common datasets but not all possible sources.
    The paper claims reduced contamination risk but acknowledges only a curated check (Sec. 3.2.2, Stage 1).
  • standard math Evaluation metrics (Recall@1x, tIoU=0.7/0.5, Average mAP) are appropriate for multi-target temporal grounding.
    Standard metrics from Ego4D; generally accepted in the field (Sec. C.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeMo: Needle in a Montage for Video-Language Understanding." pith.science (2026). https://pith.science/paper/ARIN6D55

@misc{pith2026250924563,
  author       = {Pith},
  title        = {Pith review of: NeMo: Needle in a Montage for Video-Language Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ARIN6D55}},
  note         = {Machine review of arXiv:2509.24563}
}
read the original abstract

Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for video-language understanding. Inspired by the needle in a haystack test widely used by LLMs, we introduce a novel task of Needle in a Montage (NeMo), which is designed to assess the temporal understanding capabilities of advanced VideoLLMs. Specifically, the proposed task focuses on two fundamental abilities critical for temporal understanding, i.e., retrieval-style long-context recall and temporal grounding. To generate video question answering data for our task, we develop a scalable automated data generation pipeline that facilitates high-quality data synthesis. Built upon the proposed pipeline, we present NeMoBench, a video-language benchmark centered on our task. Specifically, our full set of NeMoBench features 31,378 automatically generated question-answer (QA) pairs from 13,486 videos with various durations ranging from seconds to hours. Experiments demonstrate that our pipeline can reliably and automatically generate high-quality evaluation data, enabling NeMoBench to be continuously updated with the latest videos. We evaluate 20 state-of-the-art models on our benchmark, providing extensive results and key insights into their capabilities and limitations. Our project page is available at: https://lavi-lab.github.io/NeMoBench.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. M$^3$Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    M³Eval is a new cognitively-grounded benchmark that evaluates memory dimensions in multi-modal video models and reports consistent model weaknesses in disentanglement, interference, spatial-temporal grounding, and sym...

  2. Cambrian-S: Towards Spatial Supersensing in Video

    cs.CV 2025-11 unverdicted novelty 6.0 of 10

    Cambrian-S introduces VSI-SUPER benchmarks for long-horizon spatial recall and counting, shows data scaling yields 30% gains on existing tests, and demonstrates a self-supervised next-latent predictor using surprise o...

Reference graph

Works this paper leans on

98 extracted references · 23 linked inside Pith · cited by 2 Pith papers

  1. [1]

    , Bai , S

    botherref Bai , J. , Bai , S. , Yang , S. , Wang , S. , Tan , S. , Wang , P. , Lin , J. , Zhou , C. , Zhou , J. : Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 (2023) botherref

    arXiv 2023

  2. [2]

    , Chen , K

    botherref Bai , S. , Chen , K. , Liu , X. , Wang , J. , Ge , W. , Song , S. , Dang , K. , Wang , P. , Wang , S. , Tang , J. , Zhong , H. , Zhu , Y. , Yang , M. , Li , Z. , Wan , J. , Wang , P. , Ding , W. , Fu , Z. , Xu , Y. , Ye , J. , Zhang , X. , Xie , T. , Cheng , Z. , Zhang , H. , Yang , Z. , Xu , H. , Lin , J. : Qwen2.5-vl technical report. CoRR abs...

    arXiv 2025

  3. [3]

    , Eyzaguirre , C

    bchapter Buch , S. , Eyzaguirre , C. , Gaidon , A. , Wu , J. , Fei - Fei , L. , Niebles , J.C. : Revisiting the "video" in video-language understanding . In: CVPR ( 2022 ) bchapter

    2022

  4. [4]

    , Mann , B

    bchapter Brown , T.B. , Mann , B. , Ryder , N. , Subbiah , M. , Kaplan , J. , Dhariwal , P. , : Language models are few-shot learners . In: NeurIPS ( 2020 ) bchapter

    2020

  5. [5]

    , Nagrani , A

    bchapter Bain , M. , Nagrani , A. , Varol , G. , Zisserman , A. : Frozen in time: A joint video and image encoder for end-to-end retrieval . In: ICCV ( 2021 ) bchapter

    2021

  6. [6]

    , Gupta , A

    bchapter Chandrasegaran , K. , Gupta , A. , Hadzic , L.M. , Kota , T. , He , J. , Eyzaguirre , C. , Durante , Z. , Li , M. , Wu , J. , Fei - Fei , L. : Hourvideo: 1-hour video-language understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  7. [7]

    , Liu , Y

    bchapter Chen , G. , Liu , Y. , Huang , Y. , Pei , B. , Xu , J. , He , Y. , Lu , T. , Wang , Y. , Wang , L. : Cg-bench: Clue-grounded question answering benchmark for long video understanding . In: ICLR ( 2025 ) bchapter

    2025

  8. [8]

    , Liao , Y

    bchapter Chen , J. , Liao , Y. , Lin , H. , Yu , Y. , Chen , Y. , Wang , Y.F. : Rextime: A benchmark suite for reasoning-across-time in videos . In: NeurIPS ( 2024 ) bchapter

    2024

Show all 98 references
  1. [9]

    , Lin , Y

    bchapter Chen , X. , Lin , Y. , Zhang , Y. , Huang , W. : Autoeval-video: An automatic benchmark for assessing large vision language models in open-ended video question answering . In: ECCV ( 2024 ) bchapter

    2024

  2. [10]

    , Oh , S.W

    bchapter Cheng , H.K. , Oh , S.W. , Price , B. , Schwing , A.G. , Lee , J. : Tracking anything with decoupled video segmentation . In: ICCV ( 2023 ) bchapter

    2023

  3. [11]

    , Siarohin , A

    bchapter Chen , T. , Siarohin , A. , Menapace , W. , Deyneka , E. , Chao , H. , Jeon , B.E. , Fang , Y. , Lee , H. , Ren , J. , Yang , M. , Tulyakov , S. : Panda-70m: Captioning 70m videos with multiple cross-modality teachers . In: CVPR ( 2024 ) bchapter

    2024

  4. [12]

    , Tan , R

    botherref Cai , M. , Tan , R. , Zhang , J. , Zou , B. , Zhang , K. , Yao , F. , Zhu , F. , Gu , J. , Zhong , Y. , Shang , Y. , Dou , Y. , Park , J. , Gao , J. , Lee , Y.J. , Yang , J. : Temporalbench: Benchmarking fine-grained temporal understanding for multimodal video models...

    2024 arXiv

  5. [13]

    , Wei , X

    bchapter Chen , L. , Wei , X. , Li , J. , Dong , X. , Zhang , P. , Zang , Y. , Chen , Z. , Duan , H. , Bin , L. , Tang , Z. , Yuan , L. , Qiao , Y. , Lin , D. , Zhao , F. , Wang , J. : Sharegpt4video: Improving video understanding and generation with better captions . In: Neur...

    2024

  6. [14]

    , Wu , J

    botherref Chen , Z. , Wu , J. , Wang , W. , Su , W. , Chen , G. , Xing , S. , Zhong , M. , Zhang , Q. , Zhu , X. , Lu , L. , Li , B. , Luo , P. , Lu , T. , Qiao , Y. , Dai , J. : Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. Co...

    2023 arXiv

  7. [15]

    , Xue , F

    bchapter Chen , Y. , Xue , F. , Li , D. , Hu , Q. , Zhu , L. , Li , X. , Fang , Y. , Tang , H. , Yang , S. , Liu , Z. , He , Y. , Yin , H. , Molchanov , P. , Kautz , J. , Fan , L. , Zhu , Y. , Lu , Y. , Han , S. : Longvila: Scaling long-context visual language models for long ...

    2025

  8. [16]

    , Dai , Y

    bchapter Fu , C. , Dai , Y. , Luo , Y. , Li , L. , Ren , S. , Zhang , R. , Wang , Z. , Zhou , C. , Shen , Y. , Zhang , M. , Chen , P. , Li , Y. , Lin , S. , Zhao , S. , Li , K. , Xu , T. , Zheng , X. , Chen , E. , Shan , C. , He , R. , Sun , X. : Video-mme: The first-ever comp...

    2025

  9. [17]

    , Hua , W

    bchapter Fan , L. , Hua , W. , Li , L. , Ling , H. , Zhang , Y. : Nphardeval: Dynamic benchmark on reasoning ability of large language models via complexity classes . In: ACL ( 2024 ) bchapter

    2024

  10. [18]

    , Mao , K

    bchapter Fang , X. , Mao , K. , Duan , H. , Zhao , X. , Li , Y. , Lin , D. , Chen , K. : Mmbench-video: A long-form multi-shot benchmark for holistic video understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  11. [19]

    , Wang , W

    bchapter Fang , Y. , Wang , W. , Xie , B. , Sun , Q. , Wu , L. , Wang , X. , Huang , T. , Wang , X. , Cao , Y. : EVA: exploring the limits of masked visual representation learning at scale . In: CVPR ( 2023 ) bchapter

    2023

  12. [20]

    u nd , I. , Yianilos , P. , Mueller - Freitag , M. , Hoppe , F. , Thurau , C. , Bax , I. , Memisevic , R. : The

    bchapter Goyal , R. , Kahou , S.E. , Michalski , V. , Materzynska , J. , Westphal , S. , Kim , H. , Haenel , V. , Fr \" u nd , I. , Yianilos , P. , Mueller - Freitag , M. , Hoppe , F. , Thurau , C. , Bax , I. , Memisevic , R. : The "something something" video database for lear...

    2017

  13. [21]

    , Sun , C

    bchapter Gao , J. , Sun , C. , Yang , Z. , Nevatia , R. : Tall: Temporal activity localization via language query . In: ICCV ( 2017 ) bchapter

    2017

  14. [22]

    , Westbury , A

    bchapter Grauman , K. , Westbury , A. , Byrne , E. , Chavis , Z. , Furnari , A. , Girdhar , R. , Hamburger , J. , Jiang , H. , Liu , M. , Liu , X. , : Ego4d: Around the world in 3,000 hours of egocentric video . In: CVPR ( 2022 ) bchapter

    2022

  15. [23]

    , Escorcia , V

    bchapter Heilbron , F.C. , Escorcia , V. , Ghanem , B. , Niebles , J.C. : Activitynet: A large-scale video benchmark for human activity understanding . In: CVPR ( 2015 ) bchapter

    2015

  16. [24]

    , Feng , W

    bchapter He , X. , Feng , W. , Zheng , K. , Lu , Y. , Zhu , W. , Li , J. , Fan , Y. , Wang , J. , Li , L. , Yang , Z. , Lin , K. , Wang , W.Y. , Wang , L. , Wang , X.E. : Mmworld: Towards multi-discipline multi-faceted world model evaluation in videos . In: ICLR ( 2025 ) bchapter

    2025

  17. [25]

    , Huang , Y.-J

    botherref Huang , X. , Huang , Y.-J. , Zhang , Y. , Tian , W. , Feng , R. , Zhang , Y. , Xie , Y. , Li , Y. , Zhang , L. : Open-set image tagging with multi-grained text supervision. arXiv preprint arXiv:2310.15200 (2023) botherref

    2023 arXiv

  18. [26]

    , Sun , S

    botherref Hsieh , C. , Sun , S. , Kriman , S. , Acharya , S. , Rekesh , D. , Jia , F. , Zhang , Y. , Ginsburg , B. : RULER: what's the real context size of your long-context language models? CoRR abs/2404.06654 (2024) botherref

    2024 arXiv

  19. [27]

    , Wang , X

    bchapter Huang , B. , Wang , X. , Chen , H. , Song , Z. , Zhu , W. : Vtimellm: Empower LLM to grasp video moments . In: CVPR ( 2024 ) bchapter

    2024

  20. [28]

    , Wu , P

    botherref Hu , K. , Wu , P. , Pu , F. , Xiao , W. , Zhang , Y. , Yue , X. , Li , B. , Liu , Z. : Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. CoRR abs/2501.13826 (2025) botherref

    2025 arXiv

  21. [29]

    , Zhong , Y

    bchapter Hu , Z. , Zhong , Y. , Huang , S. , Lyu , M.R. , Wang , L. : Enhancing temporal modeling of video llms via time gating . In: EMNLP (Findings) ( 2024 ) bchapter

    2024

  22. [30]

    , Gimeno , F

    bchapter Jaegle , A. , Gimeno , F. , Brock , A. , Vinyals , O. , Zisserman , A. , Carreira , J. : Perceiver: General perception with iterative attention . In: ICML ( 2021 ) bchapter

    2021

  23. [31]

    , Song , Y

    bchapter Jang , Y. , Song , Y. , Yu , Y. , Kim , Y. , Kim , G. : Tgif-qa: Toward spatio-temporal reasoning in visual question answering . In: CVPR ( 2017 ) bchapter

    2017

  24. [32]

    : Needle in a haystack-pressure testing llms

    botherref Kamradt , G. : Needle in a haystack-pressure testing llms. https://github.com/gkamradt/LLMTest_NeedleInAHaystack (2023) botherref

    2023

  25. [33]

    , Bulatov , A

    bchapter Kuratov , Y. , Bulatov , A. , Anokhin , P. , Rodkin , I. , Sorokin , D. , Sorokin , A.Y. , Burtsev , M. : Babilong: Testing the limits of llms with long context reasoning-in-a-haystack . In: NeurIPS ( 2024 ) bchapter

    2024

  26. [34]

    , Bulatov , A

    botherref Kuratov , Y. , Bulatov , A. , Anokhin , P. , Sorokin , D. , Sorokin , A.Y. , Burtsev , M. : In search of needles in a 11m haystack: Recurrent memory finds what llms miss. CoRR abs/2402.10790 (2024) botherref

    2024 arXiv

  27. [35]

    , Berg , T.L

    botherref Lei , J. , Berg , T.L. , Bansal , M. : Detecting moments and highlights in videos via natural language queries. NeurIPS (2021) botherref

    2021

  28. [36]

    , Chen , X

    botherref Li , Y. , Chen , X. , Hu , B. , Wang , L. , Shi , H. , Zhang , M. : Videovista: A versatile benchmark for video understanding and reasoning. CoRR abs/2406.11303 (2024) botherref

    2024 arXiv

  29. [37]

    , Dong , Y

    bchapter Liu , Z. , Dong , Y. , Liu , Z. , Hu , W. , Lu , J. , Rao , Y. : Oryx MLLM: on-demand spatial-temporal understanding at arbitrary resolution . In: ICLR ( 2025 ) bchapter

    2025

  30. [38]

    , Ge , Y

    bchapter Li , B. , Ge , Y. , Ge , Y. , Wang , G. , Wang , R. , Zhang , R. , Shan , Y. : Seed-bench: Benchmarking multimodal large language models . In: CVPR ( 2024 ) bchapter

    2024

  31. [39]

    , He , Y

    botherref Li , K. , He , Y. , Wang , Y. , Li , Y. , Wang , W. , Luo , P. , Wang , Y. , Wang , L. , Qiao , Y. : Videochat: Chat-centric video understanding. CoRR abs/2305.06355 (2023) botherref

    2023 arXiv

  32. [40]

    , Han , P

    botherref Liu , Z. , Han , P. , Yu , H. , Li , H. , You , J. : Time-r1: Towards comprehensive temporal reasoning in llms. CoRR abs/2505.13508 (2025) botherref

    2025 arXiv

  33. [41]

    , Li , S

    bchapter Liu , Y. , Li , S. , Liu , Y. , Wang , Y. , Ren , S. , Li , L. , Chen , S. , Sun , X. , Hou , L. : Tempcompass: Do video llms really understand videos? In: ACL (Findings) ( 2024 ) bchapter

    2024

  34. [42]

    , Ma , Z

    bchapter Liu , Y. , Ma , Z. , Qi , Z. , Wu , Y. , Shan , Y. , Chen , C.W. : E.T. bench: Towards open-ended event-level video-language understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  35. [43]

    , Wang , Y

    bchapter Li , K. , Wang , Y. , He , Y. , Li , Y. , Wang , Y. , Liu , Y. , Wang , Z. , Xu , J. , Chen , G. , Lou , P. , Wang , L. , Qiao , Y. : Mvbench: A comprehensive multi-modal video understanding benchmark . In: CVPR ( 2024 ) bchapter

    2024

  36. [44]

    , Wong , T.L

    bchapter Li , Y. , Wong , T.L. , Hung , C.T. , Zhao , J. , Zheng , D. , Liu , K.W. , Lyu , M.R. , Wang , L. : C\( ^ 2 \)leva: Toward comprehensive and contamination-free language model evaluation . In: ACL (Findings) ( 2025 ) bchapter

    2025

  37. [45]

    , Wu , H

    bchapter Luo , Z. , Wu , H. , Li , D. , Ma , J. , Kankanhalli , M.S. , Li , J. : Videoautoarena: An automated arena for evaluating large multimodal models in video analysis through user simulation . In: CVPR ( 2025 ) bchapter

    2025

  38. [46]

    , Yin , H

    botherref Lin , J. , Yin , H. , Ping , W. , Lu , Y. , Molchanov , P. , Tao , A. , Mao , H. , Kautz , J. , Shoeybi , M. , Han , S. : Vila: On pre-training for visual language models. CVPR (2024) botherref

    2024

  39. [47]

    , Ye , Y

    bchapter Lin , B. , Ye , Y. , Zhu , B. , Cui , J. , Ning , M. , Jin , P. , Yuan , L. : Video-llava: Learning united visual representation by alignment before projection . In: EMNLP ( 2024 ) bchapter

    2024

  40. [48]

    , Zhang , P

    bchapter Lin , K.Q. , Zhang , P. , Chen , J. , Pramanick , S. , Gao , D. , Wang , A.J. , Yan , R. , Shou , M.Z. : Univtg: Towards unified video-language temporal grounding . In: ICCV ( 2023 ) bchapter

    2023

  41. [49]

    , Zhang , Y

    botherref Li , B. , Zhang , Y. , Guo , D. , Zhang , R. , Li , F. , Zhang , H. , Zhang , K. , Zhang , P. , Li , Y. , Liu , Z. , Li , C. : Llava-onevision: Easy visual task transfer. Trans. Mach. Learn. Res. (2025) botherref

    2025

  42. [50]

    , Zheng , X

    botherref Luo , Y. , Zheng , X. , Yang , X. , Li , G. , Lin , H. , Huang , J. , Ji , J. , Chao , F. , Luo , J. , Ji , R. : Video-rag: Visually-aligned retrieval-augmented long video comprehension. CoRR abs/2411.13093 (2024) botherref

    2024

  43. [51]

    , Zhang , Y

    botherref Li , S. , Zhang , Y. , Zhao , Y. , Wang , Q. , Jia , F. , Liu , Y. , Wang , T. : Vlm-eval: A general evaluation on video large language models. CoRR abs/2311.11865 (2023) botherref

    2023 arXiv

  44. [52]

    , Akshulakov , R

    bchapter Mangalam , K. , Akshulakov , R. , Malik , J. : Egoschema: A diagnostic benchmark for very long-form video language understanding . In: NeurIPS ( 2023 ) bchapter

    2023

  45. [53]

    , Rasheed , H.A

    bchapter Maaz , M. , Rasheed , H.A. , Khan , S. , Khan , F. : Video-chatgpt: Towards detailed video understanding via large vision and language models . In: ACL ( 2024 ) bchapter

    2024

  46. [54]

    , Zhu , B

    botherref Ning , M. , Zhu , B. , Xie , Y. , Lin , B. , Cui , J. , Yuan , L. , Chen , D. , Yuan , L. : Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models. CoRR abs/2311.16103 (2023) botherref

    2023 arXiv

  47. [55]

    CoRR abs/2303.08774 (2023) botherref

    botherref OpenAI : GPT-4 technical report. CoRR abs/2303.08774 (2023) botherref

    2023 arXiv

  48. [56]

    https://openai.com/index/hello-gpt-4o/

    botherref OpenAI : Hello GPT-4o. https://openai.com/index/hello-gpt-4o/. Accessed: 2024-07-29 (2024) botherref

    2024

  49. [57]

    , Smaira , L

    bchapter Patraucean , V. , Smaira , L. , Gupta , A. , Recasens , A. , Markeeva , L. , Banarse , D. , Koppula , S. , Heyward , J. , Malinowski , M. , Yang , Y. , Doersch , C. , Matejovicova , T. , Sulsky , Y. , Miech , A. , Fr \' e chette , A. , Klimczak , H. , Koster , R. , Zh...

    2023

  50. [58]

    , Li , J

    bchapter Qian , L. , Li , J. , Wu , Y. , Ye , Y. , Fei , H. , Chua , T. , Zhuang , Y. , Tang , S. : Momentor: Advancing video large language model with fine-grained temporal reasoning . In: ICML ( 2024 ) bchapter

    2024

  51. [59]

    , Kim , J.W

    bchapter Radford , A. , Kim , J.W. , Hallacy , C. , Ramesh , A. , Goh , G. , Agarwal , S. , Sastry , G. , Askell , A. , Mishkin , P. , Clark , J. , Krueger , G. , Sutskever , I. : Learning transferable visual models from natural language supervision . In: ICML ( 2021 ) bchapter

    2021

  52. [60]

    , Rohrbach , M

    botherref Regneri , M. , Rohrbach , M. , Wetzel , D. , Thater , S. , Schiele , B. , Pinkal , M. : Grounding action descriptions in videos. Trans. Assoc. Comput. Linguistics (2013) botherref

    2013

  53. [61]

    , Savinov , N

    botherref Reid , M. , Savinov , N. , Teplyashin , D. , Lepikhin , D. , Lillicrap , T.P. , Alayrac , J. , et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. CoRR abs/2403.05530 (2024) botherref

    2024 arXiv

  54. [62]

    , Yao , L

    bchapter Ren , S. , Yao , L. , Li , S. , Sun , X. , Hou , L. : Timechat: A time-sensitive multimodal large language model for long video understanding . In: CVPR ( 2024 ) bchapter

    2024

  55. [63]

    , Chai , W

    bchapter Song , E. , Chai , W. , Wang , G. , Zhang , Y. , Zhou , H. , Wu , F. , Chi , H. , Guo , X. , Ye , T. , Zhang , Y. , Lu , Y. , Hwang , J. , Wang , G. : Moviechat: From dense token to sparse memory for long video understanding . In: CVPR ( 2024 ) bchapter

    2024

  56. [64]

    , Di , S

    bchapter Shi , Y. , Di , S. , Chen , Q. , Xie , W. : Enhancing video-llm reasoning via agent-of-thoughts distillation . In: CVPR ( 2025 ) bchapter

    2025

  57. [65]

    , Xiong , Y

    bchapter Shen , X. , Xiong , Y. , Zhao , C. , Wu , L. , Chen , J. , Zhu , C. , Liu , Z. , Xiao , F. , Varadarajan , B. , Bordes , F. , Liu , Z. , Xu , H. , J. Kim , H. , Soran , B. , Krishnamoorthi , R. , Elhoseiny , M. , Chandra , V. : Longvu: Spatiotemporal adaptive compress...

    2025

  58. [66]

    , Zheng , M

    botherref Song , M. , Zheng , M. , Luo , X. : Counting-stars: A simple, efficient, and reasonable strategy for evaluating long-context large language models. CoRR abs/2403.11802 (2024) botherref

    2024 arXiv

  59. [67]

    , Lavril , T

    botherref Touvron , H. , Lavril , T. , Izacard , G. , Martinet , X. , Lachaux , M. , Lacroix , T. , Rozi \` e re , B. , Goyal , N. , Hambro , E. , Azhar , F. , Rodriguez , A. , Joulin , A. , Grave , E. , Lample , G. : Llama: Open and efficient foundation language models. CoRR ...

    2023 arXiv

  60. [68]

    , Menick , J

    bchapter Tsimpoukelli , M. , Menick , J. , Cabi , S. , Eslami , S.M.A. , Vinyals , O. , Hill , F. : Multimodal few-shot learning with frozen language models . In: NeurIPS ( 2021 ) bchapter

    2021

  61. [69]

    , Bai , S

    botherref Wang , P. , Bai , S. , Tan , S. , Wang , S. , Fan , Z. , Bai , J. , Chen , K. , Liu , X. , Wang , J. , Ge , W. , Fan , Y. , Dang , K. , Du , M. , Ren , X. , Men , R. , Liu , D. , Zhou , C. , Zhou , J. , Lin , J. : Qwen2-vl: Enhancing vision-language model's perceptio...

    2024 arXiv

  62. [70]

    , He , Z

    bchapter Wang , W. , He , Z. , Hong , W. , Cheng , Y. , Zhang , X. , Qi , J. , Huang , S. , Xu , B. , Dong , Y. , Ding , M. , Tang , J. : Lvbench: An extreme long video understanding benchmark . In: ICCV ( 2025 ) bchapter

    2025

  63. [71]

    , He , Y

    bchapter Wang , Y. , He , Y. , Li , Y. , Li , K. , Yu , J. , Ma , X. , Li , X. , Chen , G. , Chen , X. , Wang , Y. , Luo , P. , Liu , Z. , Wang , Y. , Wang , L. , Qiao , Y. : Internvid: A large-scale video-text dataset for multimodal understanding and generation . In: ICLR ( 2...

    2024

  64. [72]

    , Li , D

    bchapter Wu , H. , Li , D. , Chen , B. , Li , J. : Longvideobench: A benchmark for long-context interleaved video-language understanding . In: NeurIPS ( 2024 ) bchapter

    2024

  65. [73]

    , Hang , T

    bchapter Xue , H. , Hang , T. , Zeng , Y. , Sun , Y. , Liu , B. , Yang , H. , Fu , J. , Guo , B. : Advancing high-resolution video-language representation with large-scale video transcriptions . In: CVPR ( 2022 ) bchapter

    2022

  66. [74]

    , Shang , X

    bchapter Xiao , J. , Shang , X. , Yao , A. , Chua , T.-S. : Next-qa: Next phase of question-answering to explaining temporal actions . In: CVPR ( 2021 ) bchapter

    2021

  67. [75]

    , Zhao , Z

    bchapter Xu , D. , Zhao , Z. , Xiao , J. , Wu , F. , Zhang , H. , He , X. , Zhuang , Y. : Video question answering via gradually refined attention over appearance and motion . In: ACM Multimedia ( 2017 ) bchapter

    2017

  68. [76]

    , Gan , C

    bchapter Yi , K. , Gan , C. , Li , Y. , Kohli , P. , Wu , J. , Torralba , A. , Tenenbaum , J.B. : CLEVRER: collision events for video representation and reasoning . In: ICLR ( 2020 ) bchapter

    2020

  69. [77]

    , Wu , Y

    bchapter Yu , J. , Wu , Y. , Chu , M. , Ren , Z. , Huang , Z. , Chu , P. , Zhang , R. , He , Y. , Li , Q. , Li , S. , Li , Z. , Tu , Z. , He , C. , Qiao , Y. , Wang , Y. , Wang , Y. , Wang , L. : Vrbench: A benchmark for multi-step reasoning in long narrative videos . In: ICCV...

    2025

  70. [78]

    , Wang , Z

    bchapter Ye , J. , Wang , Z. , Sun , H. , Chandrasegaran , K. , Durante , Z. , Eyzaguirre , C. , Bisk , Y. , Niebles , J.C. , Adeli , E. , Fei - Fei , L. , Wu , J. , Li , M. : Re-thinking temporal search for long-form video understanding . In: CVPR ( 2025 ) bchapter

    2025

  71. [79]

    , Wang , J

    botherref Yuan , L. , Wang , J. , Sun , H. , Zhang , Y. , Lin , Y. : Tarsier2: Advancing large vision-language models from detailed video description to comprehensive video understanding. CoRR abs/2501.07888 (2025) botherref

    2025 arXiv

  72. [80]

    , Xu , D

    bchapter Yu , Z. , Xu , D. , Yu , J. , Yu , T. , Zhao , Z. , Zhuang , Y. , Tao , D. : Activitynet-qa: A dataset for understanding complex web videos via question answering . In: AAAI ( 2019 ) bchapter

    2019

  73. [81]

    , Yang , B

    botherref Yang , A. , Yang , B. , Hui , B. , Zheng , B. , Yu , B. , et al.: Qwen2 technical report. CoRR abs/2407.10671 (2024) botherref

    2024 arXiv

  74. [82]

    , Yang , B

    botherref Yang , A. , Yang , B. , Zhang , B. , Hui , B. , Zheng , B. , Yu , B. , Li , C. , Liu , D. , Huang , F. , Wei , H. , Lin , H. , Yang , J. , Tu , J. , Zhang , J. , Yang , J. , Yang , J. , Zhou , J. , Lin , J. , Dang , K. , Lu , K. , Bao , K. , Yang , K. , Yu , L. , Li ...

    2024 arXiv

  75. [83]

    , Yu , T

    botherref Yao , Y. , Yu , T. , Zhang , A. , Wang , C. , Cui , J. , Zhu , H. , Cai , T. , Li , H. , Zhao , W. , He , Z. , Chen , Q. , Zhou , H. , Zou , Z. , Zhang , H. , Hu , S. , Zheng , Z. , Zhou , J. , Cai , J. , Han , X. , Zeng , G. , Li , D. , Liu , Z. , Sun , M. : Minicpm...

    2024 arXiv

  76. [84]

    , Attarian , M

    bchapter Zeng , A. , Attarian , M. , Ichter , B. , Choromanski , K.M. , Wong , A. , Welker , S. , Tombari , F. , Purohit , A. , Ryoo , M.S. , Sindhwani , V. , Lee , J. , Vanhoucke , V. , Florence , P. : Socratic models: Composing zero-shot multimodal reasoning with language . ...

    2023

  77. [85]

    , Cho , J

    bchapter Zala , A. , Cho , J. , Kottur , S. , Chen , X. , Oguz , B. , Mehdad , Y. , Bansal , M. : Hierarchical video-moment retrieval and step-captioning . In: CVPR ( 2023 ) bchapter

    2023

  78. [86]

    , Chiang , W

    bchapter Zheng , L. , Chiang , W. , Sheng , Y. , Zhuang , S. , Wu , Z. , Zhuang , Y. , Lin , Z. , Li , Z. , Li , D. , Xing , E.P. , Zhang , H. , Gonzalez , J.E. , Stoica , I. : Judging llm-as-a-judge with mt-bench and chatbot arena . In: NeurIPS ( 2023 ) bchapter

    2023

  79. [87]

    , Hu , Z

    bchapter Zhong , Y. , Hu , Z. , Lyu , M.R. , Wang , L. : Beyond embeddings: The promise of visual table in visual reasoning . In: EMNLP ( 2024 ) bchapter

    2024

  80. [88]

    , Kong , T

    bchapter Zheng , D. , Kong , T. , Jing , Y. , Wang , J. , Wang , X. : Towards unifying reference expression generation and comprehension . In: EMNLP ( 2022 ) bchapter

    2022

  81. [89]

    , Li , X

    bchapter Zhang , H. , Li , X. , Bing , L. : Video-llama: An instruction-tuned audio-visual language model for video understanding . In: EMNLP (Demos) ( 2023 ) bchapter

    2023

  82. [90]

    , Lu , H

    bchapter Zhao , Z. , Lu , H. , Huo , Y. , Du , Y. , Yue , T. , Guo , L. , Wang , B. , Chen , W. , Liu , J. : Needle in A video haystack: A scalable synthetic evaluator for video mllms . In: ICLR ( 2025 ) bchapter

    2025

  83. [91]

    , Lin , B

    bchapter Zhu , B. , Lin , B. , Ning , M. , Yan , Y. , Cui , J. , Wang , H. , Pang , Y. , Jiang , W. , Zhang , J. , Li , Z. , Zhang , C. , Li , Z. , Liu , W. , Yuan , L. : Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment . In...

    2024

  84. [92]

    , Luo , T

    botherref Zou , H. , Luo , T. , Xie , G. , Zhang , V. , Lv , F. , Wang , G. , Chen , J. , Wang , Z. , Zhang , H. , Zhang , H. : From seconds to hours: Reviewing multimodal large language models on comprehensive long video understanding. CoRR abs/2409.18938 (2024) botherref

    2024 arXiv

  85. [93]

    , Mustafa , B

    bchapter Zhai , X. , Mustafa , B. , Kolesnikov , A. , Beyer , L. : Sigmoid loss for language image pre-training . In: ICCV ( 2023 ) bchapter

    2023

  86. [94]

    , Shu , Y

    bchapter Zhou , J. , Shu , Y. , Zhao , B. , Wu , B. , Liang , Z. , Xiao , S. , Qin , M. , Yang , X. , Xiong , Y. , Zhang , B. , Huang , T. , Liu , Z. : MLVU: benchmarking multi-task long video understanding . In: CVPR ( 2025 ) bchapter

    2025

  87. [95]

    , Wu , J

    botherref Zhang , Y. , Wu , J. , Li , W. , Li , B. , MA , Z. , Liu , Z. , Li , C. : LL a VA -video: Video instruction tuning with synthetic data. Trans. Mach. Learn. Res. (2025) botherref

    2025

  88. [96]

    , Zhang , K

    botherref Zhang , P. , Zhang , K. , Li , B. , Zeng , G. , Yang , J. , Zhang , Y. , Wang , Z. , Tan , H. , Li , C. , Liu , Z. : Long context transfer from language to vision. Trans. Mach. Learn. Res. (2025) botherref

    2025

  89. [97]

    , Zhang , H

    bchapter Zhao , Y. , Zhang , H. , Xie , L. , Hu , T. , Gan , G. , Long , Y. , Hu , Z. , Chen , W. , Li , C. , Xu , Z. , Wang , C. , Shangguan , Z. , Liang , Z. , Liu , Y. , Zhao , C. , Cohan , A. : MMVU: measuring expert-level multi-discipline video understanding . In: CVPR ( ...

    2025

  90. [98]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.