Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

How Important are Videos for Training Video LLMs?

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Real videos barely help video LLMs reason about time.

desk verdict A cheap pseudo-video baseline nearly matches expensive video training on TVBench—that result is real and worth attention—but the 'image-trained' claim is overstated because their proxy got one step of temporal training. read the letter →

arxiv 2506.06928 v1 pith:RAVOX4WE submitted 2025-06-07 cs.CV

George Lydakis , Alexander Hermans , Ali Athar , Daan de Geus , Bastian Leibe This is my paper
classification cs.CV
keywords videolargelanguagemodelstemporalreasoningtrainingimage-onlypseudo-videoTVBenchLongVUmultiple-choicequestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks how much of a video LLM's temporal reasoning comes from training on actual videos rather than from text and image pretraining. It shows that two LongVU-based models that never saw real video score far above chance on the TVBench temporal benchmark, and that a simple baseline finetuned on pseudo-videos—ordered still images with generated temporal questions—matches or exceeds real-video finetuning on one backbone. This matters because video training is computationally expensive, and if still-image sequences capture most of the benefit, the bottleneck likely lies in how real video data is used rather than in a lack of video data.

What carries the argument

The central object is the pseudo-video: a sequence formed by sampling S captioned images, repeating each image F times with a mild affine transformation, and then auto-generating multiple-choice questions about relative scene order or absolute counting/localization. This construction gives fully known temporal evolution and controllable question difficulty. A second load-bearing mechanism is the image-trained proxy: because the pure image checkpoint does not emit multiple-choice format, the authors train it for one step on pseudo-videos with question R1 and treat the result as sufficiently close to an image-only model for evaluation.

What would settle it

Train the one-step proxy on pseudo-videos with captions randomly mismatched to their frames; if TVBench accuracy stays above chance, the one step is not teaching frame-caption correspondence and the proxy assumption is safe, whereas if accuracy drops to chance, the proxy itself is the source of the apparent image-only temporal ability.

Watch

Extended reading notes

Core claim

The paper argues that much of the temporal reasoning exhibited by Video LLMs is acquired from text-and-image training, not from exposure to videos. Using LongVU models with Llama3.2 and Qwen2 backbones, the authors find that image-trained checkpoints score well above chance on TVBench, and that finetuning on pseudo-videos built from annotated COCO images—with repeated frames, mild perturbations, and generated ordering and counting questions—reaches 53.5% average accuracy for Llama3.2, above the 51.2% of the video-trained model, while Qwen2's pseudo-video model trails the video-trained one by about 4 percentage points. Because pseudo-videos contain none of the complexity of real motion, the authors conclude that current video training schemes underuse the temporal information in real video.

Load-bearing premise

The claim rests on treating a model trained for one step on pseudo-videos as a faithful proxy for image-only training; if that single step imparts non-trivial temporal reasoning skills, the above-chance result is not evidence from image training alone.

Editorial extensions

If this is right

  • If temporal reasoning mostly comes from image and text data, then large amounts of video training may be replaceable by cheaper image-sequence training without much accuracy loss.
  • Video-trained models do not reliably benefit from additional pseudo-video finetuning, and some tasks show performance drops, suggesting task-specific forgetting rather than general temporal improvement.
  • The small gap between video-trained models and the pseudo-video baseline points to a bottleneck in video data quality, video captioning, training setup, or architecture.
  • Training with relative ordering questions is more effective than training with absolute temporal reasoning questions for pseudo-video finetuning.
  • Increasing the number of scenes per pseudo-video helps tasks like Scene Transition, while training on more than about 100,000 pseudo-videos can lead to overfitting on a specific question type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The one-step proxy could be tested by shuffling captions or frame order during that single training step; if above-chance TVBench accuracy persists, the temporal gains stem from language priors rather than from image sequence structure.
  • Inference: The paper's inability to run pseudo-video training as an intermediate stage between image and real-video training leaves open the possibility that such an ordering would improve real-video models, which would indicate that current pipelines order their training phases suboptimally.
  • Inference: The CLEVRER/CLEVR confound suggests that retraining image-text models without CLEVR-source images would drop Object Count and Moving Direction scores to chance, which would show that those seemingly temporal skills are actually static-scene recognition.
  • Inference: Extending the pseudo-video generator with object masks and scripted motion patterns could probe whether models can learn true dynamics; if they still fail to beat the static baseline, the bottleneck is architectural rather than purely data-driven.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper asks how much of the temporal reasoning ability of modern Video LLMs actually comes from training on real videos. Using the LongVU architecture with Llama3.2-3B and Qwen2-7B backbones, the authors evaluate image-trained and video-trained checkpoints on the TVBench temporal-reasoning benchmark. They report that models trained only on images and text (with a one-step pseudo-video adaptation to fix the output format) score significantly above chance on TVBench, and that a simple pseudo-video baseline—sequences of captioned COCO images with affine perturbations and automatically generated questions about scene order—achieves accuracy close to, or for Llama3.2 even exceeding, the video-trained models. The paper concludes that current video training schemes underutilize the temporal information in real videos and encourages research into more efficient data usage.

Significance. If the central comparison holds, the paper makes a striking and useful contribution: it suggests that a large part of the temporal reasoning signal that Video LLMs currently extract from expensive video datasets can be reproduced by ordering static captioned images. The pseudo-video training scheme is simple, reproducible, and the ablation tables (Tables 5-8) provide a valuable map of the design space. The paper also asks the right question about data efficiency in a field where video training is becoming increasingly costly. However, the force of the paper is weakened by two issues: the 'image-trained' proxy in Section 4.1 actually received one step of pseudo-video training with a temporal ordering question, and the pseudo-video baseline was explicitly tuned on TVBench, which overlaps in task structure with the training questions. These issues need to be addressed before the claims as stated are fully supported.

major comments (4)
  1. [Sec. 4.1] The 'image-trained' proxy is not actually image-only. The text states that the checkpoint was trained for 'as little as one step on samples consisting of multiple frames' and specifically 'on pseudo videos with question R1', where R1 asks the model to select the correct order of scene captions. This single step is the first time the model receives a temporal supervision signal (that frames have an order) and simultaneously learns the MCQA output format. The shuffled-frame drop in Table 3 shows that the proxy is sensitive to frame order, but that sensitivity could be learned entirely from the one-step R1 adaptation rather than from image-text pretraining. The paper provides no control for the temporal content of the adaptation step; for example, one could adapt the image checkpoint with a non-temporal task (e.g., single-image captioning or an MCQA task without ordering) for one step and then compare. Without such a control, the abstract's claim of 'image-trained versions ... perform significantly above chance' and Section 5's statement that the models were 'trained on just images and text' are not supported.
  2. [Table 3 / Sec. 4.1] The phrase 'significantly above chance level' is used repeatedly, but no error bars, confidence intervals, or significance tests are reported. TVBench accuracy is a point estimate on a finite test set; for tasks with 25% chance level, a few dozen questions could make the difference between 30% and 35% non-significant. The authors should report per-task sample sizes and either Wilson confidence intervals or a binomial test for each model against chance, and also for the image-versus-video comparisons in Tables 3 and 4. Without this, the 'significant' claim is not substantiated.
  3. [Sec. 4.2 / Table 4] The pseudo-video baseline is effectively tuned on TVBench. The paper reports results for the 'best pseudo-video configuration, as found empirically through experiments', and Tables 5-8 show a search over question types (R1-R4, A1-A2), number of scenes S, frames per scene F, and dataset size N. Several of the training questions are structurally very close to TVBench tasks: R1 asks for the order of scene captions (analogous to Scene Transition and Action Sequence), R2 asks before/after relations, R3 asks first/last, and R4 asks successor/predecessor. This task overlap, combined with selection of the best configuration on the test benchmark, gives the pseudo-video baseline an advantage that the video-trained models (trained on generic video-caption data without such tuning) do not enjoy. This weakens the claim that pseudo-videos 'definitely do not capture the complexity of real videos' yet match video training. The authors should either pre-register the configuration, use a held-out validation set, or include a control in which pseudo-videos are trained with a broader, less benchmark-aligned question set.
  4. [Table 3 / Shuffled-frame evaluation] The shuffled-frame evaluation is used to argue that the image-trained proxy 'do learn some form of temporal reasoning'. However, because the proxy has already seen one step of R1 training (which teaches that frame order is task-relevant), the shuffled-frame drop may simply reflect that learned signal, not emergent temporal reasoning from image-text pretraining. To make this argument, the authors should compare the shuffled-frame sensitivity of the one-step R1-adapted model with that of a model adapted on a non-temporal task. If the non-temporal control shows no drop, then the effect is attributable to the adaptation step, and the 'image-only temporal reasoning' conclusion is unsupported.
minor comments (4)
  1. [Abstract] The claim 'image-trained versions of two LLMs ... perform significantly above chance' should be qualified to explicitly state that these models received one step of pseudo-video training with a temporal question, so readers are not misled.
  2. [Sec. 4.2] There is a typo: 'LLama3.2' appears in the text where 'Llama3.2' is meant.
  3. [Table 3] For the UA task, shuffled-frame evaluation sometimes yields accuracy equal to or higher than the ordered evaluation (e.g., Llama3.2 image-trained: 35.4 ordered vs 36.6 shuffled; Qwen2 image-trained: 36.6 both). This is not discussed and complicates the 'drop indicates temporal reasoning' interpretation; the authors should comment on it.
  4. [Sec. 4.2 / Table 5] The A1 configuration with S=6 yields below-chance accuracy on AC (20.3 vs 25.0 chance), suggesting that some question types can actively degrade performance. The paper says 'relative questions perform better than absolute ones' but does not discuss the negative effect of A1 on AC; a brief comment would help.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-chain circularity; the 'image-only' proxy is contaminated by one step of R1 pseudo-video training, a validity confound rather than a circular reduction.

  1. other [Sec 4.1, Table 3 (proxy definition for image-trained evaluation)]
    "This issue can be resolved by training the models for as little as one step on samples consisting of multiple frames. We therefore present results when training for one step on pseudo videos with question R1. Given the short duration of training and the fact that these models never see real videos, we deem these configurations to be sufficiently close to LLMs that have only been trained on images and thus use them as their proxy for our evaluation."

    The first claim ('image-trained versions... perform significantly above chance') is attributed to image-text pretraining, but the evaluated checkpoint has received one step of pseudo-video training with R1, a question about scene ordering. That step supplies temporal supervision (frames have an order) and the MCQA format, so the above-chance TVBench scores in Table 3 may partly reflect this adaptation rather than pure image-only training. This is a definitional/validity issue rather than a formal equivalence by construction, because TVBench is an external benchmark and the step is not fitted to it; the paper is transparent about the step, but the load-bearing attribution to 'image-only' is not isolated.

full rationale

The paper is empirical and self-contained against an external benchmark (TVBench); its main second finding—pseudo-video finetuning reaches performance close to or above video finetuning—does not depend on the image-only proxy and is not circular. No self-citation chain or imported uniqueness theorem is load-bearing. The only concern is in Sec 4.1: the 'image-trained' proxy is trained for one step on R1 pseudo-videos, so the claim that image-only training yields above-chance temporal reasoning is not cleanly supported; one step of ordering supervision could teach the relevant behavior. The paper itself flags this ('we deem these configurations to be sufficiently close') and the CLEVR-exclusion experiment it would take to isolate the effect is stated to exceed compute resources. This is a confound and an openly stated limitation, not a reduction of a prediction to its inputs by construction. Accordingly, the circularity score is low (2).

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on standard ML hyperparameters (S, F, N, LR) chosen by search, a stated proxy assumption for image-only evaluation, and the external validity of TVBench. No new physical entities are introduced; the pseudo-video is a synthetic data construction whose only support is the reported experiments.

free parameters (4)
  • max scenes S = 4 for R1, 6 for R3 (Llama best config)
    Number of distinct images sampled per pseudo-video; chosen via empirical search (Tables 5, 7), with S=4 best for R1 on Llama.
  • max frames per scene F = 20 for R1, 5 for R3 (Llama best config)
    Number of repetitions of each image; affects pseudo-video length; optimal value is question-dependent (Table 6).
  • number of pseudo videos N = 100,000
    Training set size; larger N (500K) degrades average performance on some tasks (Table 8).
  • learning rate schedule = peak 5e-6, linear warmup 3%, cosine decay
    Modified from LongVU's schedule; authors note it led to faster convergence in preliminary experiments, a hand-chosen hyperparameter.
assumptions (4)
  • domain assumption COCO captions are sufficiently noise-free to serve as ground truth for scene descriptions
    The pseudo-video question generator assumes the caption uniquely and correctly describes each scene (Sec 3, Fig 1).
  • domain assumption TVBench is a valid measure of temporal reasoning that cannot be solved via single-frame or text-only shortcuts
    The paper relies on TVBench's design as stated in Sec 2 and cites [4]; if TVBench is partially gameable, the gap measurements are distorted.
  • ad hoc to paper One step of pseudo-video training makes an image checkpoint behaviorally equivalent to a purely image-trained model
    Stated in Sec 4.1 as a proxy; this is the weakest premise, since the effect of the one step is not measured.
  • domain assumption Mild affine perturbations preserve scene identity and temporal order for question answering
    The pseudo-video scheme repeats a static image with affine transforms and assumes the model can still identify the scene and its order (Sec 3).
invented entities (1)
  • Pseudo-video
    purpose: A training sample formed by concatenating repeated perturbed images with known scene order, used to generate temporal MCQA questions
    This is a synthetic data construction introduced by the paper; its usefulness is evaluated only on TVBench, so it has no external falsifiable handle outside the reported experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Important are Videos for Training Video LLMs?." pith.science (2026). https://pith.science/paper/RAVOX4WE

@misc{pith2026250606928,
  author       = {Pith},
  title        = {Pith review of: How Important are Videos for Training Video LLMs?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RAVOX4WE}},
  note         = {Machine review of arXiv:2506.06928}
}
read the original abstract

Research into Video Large Language Models (LLMs) has progressed rapidly, with numerous models and benchmarks emerging in just a few years. Typically, these models are initialized with a pretrained text-only LLM and finetuned on both image- and video-caption datasets. In this paper, we present findings indicating that Video LLMs are more capable of temporal reasoning after image-only training than one would assume, and that improvements from video-specific training are surprisingly small. Specifically, we show that image-trained versions of two LLMs trained with the recent LongVU algorithm perform significantly above chance level on TVBench, a temporal reasoning benchmark. Additionally, we introduce a simple finetuning scheme involving sequences of annotated images and questions targeting temporal capabilities. This baseline results in temporal reasoning performance close to, and occasionally higher than, what is achieved by video-trained LLMs. This suggests suboptimal utilization of rich temporal features found in real video by current models. Our analysis motivates further research into the mechanisms that allow image-trained LLMs to perform temporal reasoning, as well as into the bottlenecks that render current video training schemes inefficient.

Figures

Figures reproduced from arXiv: 2506.06928 by the authors.

Figure 1
Figure 1. A comparison of the standard video-based training scheme for Video LLMs (top) and our proposed pseudo-video training scheme [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Accuracy Without Grounding: Diagnosing Visual Dependency Dissociation in Video LLM Benchmarks

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A benchmark audit shows that video LLM accuracy and true visual grounding are separable, and that most apparent video understanding comes from frame diversity, not temporal order.

Reference graph

Works this paper leans on

31 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Qwen2.5-VL Technical Report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-VL Technical Report. arXiv preprint arXiv:2502.13923, 2025. 1, 2, 4

    arXiv 2025

  2. [2]

    ShareGPT4Video: Improving Video Under- standing and Generation with Better Captions

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. ShareGPT4Video: Improving Video Under- standing and Generation with Better Captions. In NeurIPS,

    work page

  3. [3]

    VideoLLaMA 2: Advancing Spatial- Temporal Modeling and Audio Understanding in Video- LLMs

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. VideoLLaMA 2: Advancing Spatial- Temporal Modeling and Audio Understanding in Video- LLMs. arXiv preprint arXiv:2406.07476, 2024. 1, 2

    arXiv 2024

  4. [4]

    Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M. Asano. Lost in Time: A New Temporal Benchmark for VideoLLMs. arXiv preprint arXiv:2410.07752, 2025. 1, 2, 4, 5

    arXiv 2025

  5. [5]

    Video-MME: The First- Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-MME: The First- Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. arXiv preprint arXiv:2405.21075,

    arXiv

  6. [6]

    The Llama 3 Herd of Models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Ab- hinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783, 2024. 4

    arXiv 2024

  7. [7]

    MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

    Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim. MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding. In CVPR, 2024. 1, 2

    work page 2024

  8. [8]

    LoRA: Low- Rank Adaptation of Large Language Models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. LoRA: Low- Rank Adaptation of Large Language Models. InICLR, 2022. 1

    work page 2022

Show all 31 references
  1. [9]

    TGIF-QA: Toward Spatio-Temporal Reason- ing in Visual Question Answering

    Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. TGIF-QA: Toward Spatio-Temporal Reason- ing in Visual Question Answering. In CVPR, 2017. 2

    2017

  2. [10]

    CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning. In CVPR,

  3. [11]

    JUWELS Cluster and Booster: Exascale Pathfinder with Modular Supercomputing Architecture at Juelich Supercomputing Centre

    J ¨ulich Supercomputing Centre. JUWELS Cluster and Booster: Exascale Pathfinder with Modular Supercomputing Architecture at Juelich Supercomputing Centre. Journal of large-scale research facilities, 7(A138), 2021. 8

    2021

  4. [12]

    LLaV A-OneVision: Easy Visual Task Trans- fer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. LLaV A-OneVision: Easy Visual Task Trans- fer. arXiv preprint arXiv:2408.03326, 2024. 5

    2024 arXiv

  5. [13]

    MVBench: A Comprehensive Multi-modal Video Under- standing Benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. MVBench: A Comprehensive Multi-modal Video Under- standing Benchmark. In CVPR, 2024. 1, 2

    2024

  6. [14]

    Temporal Preference Optimization for Long-Form Video Understanding

    Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy. Temporal Preference Optimization for Long-Form Video Understanding. arXiv preprint arXiv:2501.13919, 2025. 1, 2

    2025 arXiv

  7. [15]

    LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models. In ECCV, 2024. 1, 2

    2024

  8. [16]

    Video-LLaV A: Learning United Visual Representation by Alignment Before Projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-LLaV A: Learning United Visual Representation by Alignment Before Projection. In EMNLP,

  9. [17]

    Microsoft COCO: Common Objects in Context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In ECCV, 2014. 5

    2014

  10. [18]

    Oryx MLLM: On-Demand Spatial- Temporal Understanding at Arbitrary Resolution

    Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx MLLM: On-Demand Spatial- Temporal Understanding at Arbitrary Resolution. arXiv preprint arXiv:2409.12961, 2024. 1, 2

    2024 arXiv

  11. [19]

    Video-ChatGPT: Towards Detailed Video Under- standing via Large Vision and Language Models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fa- had Khan. Video-ChatGPT: Towards Detailed Video Under- standing via Large Vision and Language Models. In ACL,

  12. [20]

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael ...

    2024

  13. [21]

    Learning Transferable Visual Models From Natural Language Super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning Transferable Visual Models From Natural Language Super- vision. In ICLR, 2021. 1

    2021

  14. [22]

    Kim, Bilge Soran, Raghuraman Krishnamoor- thi, Mohamed Elhoseiny, and Vikas Chandra

    Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Bal- akrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoor- thi, Mohamed Elhoseiny, and Vikas Chandra. LongVU: Spat...

    arXiv

  15. [23]

    Tarsier: Recipes for Training and Evalu- ating Large Video Description Models

    Jiawei Wang, Liping Yuan, Yuchen Zhang, and Hao- miao Sun. Tarsier: Recipes for Training and Evalu- ating Large Video Description Models. arXiv preprint arXiv:2407.00634, 2024. 1, 2

    2024 arXiv

  16. [24]

    LongVideoBench: A Benchmark for Long-context Inter- leaved Video-Language Understanding

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. LongVideoBench: A Benchmark for Long-context Inter- leaved Video-Language Understanding. In NeurIPS, 2024. 2

    2024

  17. [25]

    MSR-VTT: A Large Video Description Dataset for Bridging Video and Language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. MSR-VTT: A Large Video Description Dataset for Bridging Video and Language. In CVPR, 2016. 2 9

    2016

  18. [26]

    PLLaV A : Parameter-free LLaV A Extension from Images to Videos for Video Dense Captioning

    Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. PLLaV A : Parameter-free LLaV A Extension from Images to Videos for Video Dense Captioning. arXiv preprint arXiv:2404.16994, 2024. 1, 2

    2024 arXiv

  19. [27]

    CLEVRER: CoLlision Events for Video REpresentation and Reasoning

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Ji- ajun Wu, Antonio Torralba, and Joshua B Tenenbaum. CLEVRER: CoLlision Events for Video REpresentation and Reasoning . In ICLR, 2019. 5

    2019

  20. [28]

    ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answer- ing

    Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answer- ing. In AAAI, 2019. 2

    2019

  21. [29]

    Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding

    Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding. arXiv preprint arXiv:2501.07888 ,

    arXiv

  22. [30]

    Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding. In EMNLP, 2023. 2

    2023

  23. [31]

    Video Instruction Tuning With Synthetic Data

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Zi- wei Liu, and Chunyuan Li. Video Instruction Tuning With Synthetic Data. arXiv preprint arXiv:2410.02713, 2024. 2 10

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.