Pith. sign in

REVIEW 3 major objections 3 minor 122 references

VideoChat3 claims a 4B, fully open video model can beat larger open rivals by compressing visual tokens 16x before they reach the language model, while also handling streaming video.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:39 UTC pith:IBJWLJI4

load-bearing objection Serious open systems paper with a genuine architecture/data contribution, but the temporal-grounding and streaming comparisons need a fairness check before the headline numbers can be trusted. the 3 major comments →

arxiv 2607.14935 v1 pith:IBJWLJI4 submitted 2026-07-16 cs.CV

VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

Xinhao Li , Yuhan Zhu , Xiangyu Zeng , Yuhao Dong , Haoning Wu , Zhiqiu Zhang , Yuandong Yang , Changlian Ma
show 19 more authors
Qingyu Zhang Yansong Shi Xinyu Chen Haoran Chen Zizheng Huang Jun Zhang Kun Ouyang Lin Sui Ziang Yan Yicheng Xu Chenting Wang Yinan He Hongjie Zhang Yi Wang Yu Qiao Yali Wang Ziwei Liu Kai Chen Limin Wang
This is my paper
classification cs.CV
keywords video multimodal large language modelsefficient video tokenization3D vision transformertemporal groundingstreaming video understandinginstruction tuningopen-source datasetslong video understanding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that a video multimodal language model can be both broadly capable and computationally cheap if video redundancy is removed inside the vision encoder rather than by sparse frame sampling. VideoChat3, at 4B parameters, uses an inflated 3D vision transformer that chunks frames, applies spatiotemporal attention, and temporally pools, achieving a 16x spatiotemporal compression before tokens reach the language model. The authors report that this is enough to surpass prior open models of equal or larger size across general, long-form, and streaming benchmarks: 18 of 19 direct comparisons over a leading 4B baseline, matching or beating another 4B open model on most benchmarks, with roughly half the visual tokens and substantially lower long-video latency. A second mechanism, adaptive frame resolution, lets the model spend more visual budget only after a 'standby' cue during streaming. The claim matters because it would mean efficiency and generality need not trade off, and because the full stack—weights, code, training strategy, and datasets—is released.

Core claim

The central discovery, stated on the paper's own terms, is that compressing video early—inside a vision tokenizer that models local space-time before the LLM—preserves motion evidence while cutting LLM context cost. I3D-ViT inflates a pretrained image ViT's 2D attention into chunk-wise 3D spatiotemporal attention with learned temporal positions, then pools over T frames; with T=4 and 2x2 spatial pixel-shuffle downsampling, the visual sequence shrinks 16x. This changes the compute profile: the vision encoder's cost grows roughly linearly in frames, while the LLM's quadratic context cost drops, so long-video inference becomes faster and lighter. The paper further shows the same tokenizer suppo

What carries the argument

The load-bearing object is the Inflated 3D Vision Transformer (I3D-ViT), a visual tokenizer created by inflating a pretrained image ViT's 2D self-attention into chunk-wise spatiotemporal attention over T-frame windows, adding learned temporal positional embeddings, and then applying chunk-wise temporal pooling. Combined with 2x2 pixel-shuffle spatial downsampling and T=4, it gives a 16x spatiotemporal compression ratio, halving the visual tokens a baseline would produce while preserving motion cues. The second mechanism is Adaptive Frame Resolution: at each streaming step the LLM emits a response-state token (Silence, Standby, Response), and a deterministic controller sets the next window's

Load-bearing premise

The comparative claims stand or fall on the assumption that every model was evaluated under identical protocols with no training/test overlap; the paper's own tables show temporal-grounding training data derived from the same datasets as the temporal-grounding evaluation splits, and streaming baselines are mostly quoted while VideoChat3 uses a modified bounding-box output range.

What would settle it

Delete the TimeLens-100K (and any same-source grounding) samples from the Stage-2/3 mixtures, retrain, and re-measure the three TimeLens splits plus VUE-TR and MomentSeeker; if the 6-20 point grounding margins shrink to noise, the temporal-grounding claim is an artifact of train/test overlap. In parallel, re-run the streaming baselines under the same [0,1]-vs-[0,1000] bounding-box normalization and identical evaluation code; if the ODVBench and OVO-Timing gaps disappear, the streaming claim is protocol-driven.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The efficiency claim is concrete: with 256 to 2048 input frames, VideoChat3 produces exactly half the visual tokens of the compared baseline under equal ViT patches per frame, and at 2048 frames the reported end-to-end latency drops from 44.4 s to 20.4 s while FLOPs fall by more than 60%.
  • Because the model is fully open (weights, code, training strategy, and all three datasets), any research group can replicate the training pipeline and extend it without reverse engineering.
  • The streaming training recipe—state tokens with a balanced transition mask and a deterministic low/high pixel-budget controller—is the paper's answer to 'when to answer' and can be applied to other video MLLMs.
  • The evidence-grounded annotation pipeline increases supervision density from sparse academic labels and is reported to improve temporal perception and grounding consistently, suggesting data quality rather than raw scale drives much of the gain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the 16x early-compression design is the real driver, it should transfer to other base LLMs and vision encoders; a controlled swap experiment would reveal how much of the gain is architectural rather than data-specific.
  • The adaptive frame-resolution controller could be lifted from streaming into offline long-video pipelines: a salience estimate, rather than a state token, would decide which segments get the high 448² pixel budget, potentially cutting cost without a real-time loop.
  • Given the TimeLens training/evaluation overlap noted in Sections 4.3 and 5.1, the most informative independent check is a zero-shot run on temporal-grounding benchmarks whose source datasets were never in the training mixture; the reported margins may not transfer.
  • A cheaper-to-reproduce variant—distilling the synthetic long-video and streaming supervision into a smaller open model—would test whether the state-transition masking and adaptive budget still yield proactive behavior at lower cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper presents VideoChat3, a 4B-parameter fully open video-centric MLLM, with two main contributions: an Inflated 3D Vision Transformer (I3D-ViT) that performs chunk-wise spatiotemporal attention and temporal pooling to reduce visual tokens, and an Adaptive Frame Resolution mechanism for streaming video that toggles per-frame pixel budgets based on predicted response-state tokens. The authors also release three training datasets (VideoChat3-Academic2M, VideoChat3-LV116K, VideoChat3-OL617K), a four-stage training curriculum, model weights, and code. Empirically, the paper claims strong generalization across temporal perception, long-video QA, temporal grounding, and streaming/proactive benchmarks, with particular gains over Qwen3-VL-4B and Molmo2-4B, plus efficiency improvements from reduced visual-token counts.

Significance. If the reported results are taken at face value, this is a substantial systems-and-data contribution: it demonstrates that early spatiotemporal compression in the visual tokenizer can preserve broad video understanding while cutting inference cost, and it provides fully released assets for reproducible research. The ablation isolating the state-transition mask and the dynamic perception-window policy is well designed and informative. However, the paper's strongest empirical claims—especially in temporal grounding and streaming—rest on benchmark comparisons whose train/eval disjointness and protocol symmetry are not established. The contribution is therefore potentially significant but currently conditional on resolving these evaluation concerns.

major comments (3)
  1. [Sections 4.3/4.4 vs 5.1, Table 2] TimeLens-100K [14] is included in both the Stage-2 and Stage-3 training mixtures, and temporal grounding is evaluated on the TimeLens suite [14] built from Charades-STA, ActivityNet Captions, and QVHighlights. No evidence is provided that the training and evaluation splits are disjoint at the video or annotation level. Because the largest differentials in Table 2 are on the three TimeLens columns (+9.7/+6.4/+8.3 over Qwen3-VL-4B, and +16.4/+29.8/+34.3 over VideoChat-Flash-7B), these gains are load-bearing for the claim of surpassing prior open models in temporal grounding. Please provide exact overlap statistics between TimeLens-100K training samples and each evaluation split, and re-report results after excluding overlapping items or on independent held-out grounding benchmarks.
  2. [Section 4.4 vs 5.2, Table 3] Stage-3 training explicitly includes StreamForest [19], while the streaming evaluation includes ODVBench [19], which is the benchmark introduced in the StreamForest paper. The paper does not establish that the StreamForest training data and the ODVBench evaluation clips are disjoint. Similarly, River [87] appears to be an author-created benchmark, and the Stage-3 mixture includes StreamForest/Streamo/Seeker data; no exclusion analysis is reported. Given that Table 3 claims best results on four of six streaming metrics, this potential overlap must be quantified and the results recomputed on non-overlapping data before the streaming claims can be accepted.
  3. [Appendix Table 11, Table 3] The evaluation protocol for VideoChat3 is not symmetric with the baselines. For ODVBench and OVBench, the paper modifies the bounding-box output range from [0,1] to [0,1000] when evaluating VideoChat3, while most baseline results are quoted from their original papers. Likewise, Table 12 applies custom low/high token budgets and high-budget frame counts to the proactive-response benchmarks. Since the streaming comparison is cross-paper, such protocol differences could explain part of the reported improvements. Please run at least the strongest open baselines (e.g., StreamForest, StreamingVLM, Qwen3-VL-4B) under the same evaluation harness and output-format conversions, and report all models with matched token/frame budgets.
minor comments (3)
  1. [Throughout] The PDF contains recurring ligature artifacts such as 'efficiency', 'sufficient', and 'official'. These should be corrected in the camera-ready version.
  2. [Table 9 / Section 5.2] Adding VideoChat3-OL617K substantially decreases the ProactiveVQA WEB score (39.0 to 28.4). The text says gains extend across streaming benchmarks without acknowledging this drop. Please add a sentence discussing this trade-off.
  3. [Section 3 / 7] The term 'fully open' should be qualified: the model weights and training assets are released, but a large fraction of the supervision is generated by proprietary models (Qwen3-VL-235B-A22B and Gemini). The assets are open, but the annotation pipeline is not fully reproducible without access to those closed APIs.

Circularity Check

3 steps flagged

Several headline metrics are evaluated on benchmarks that also appear in the training mixture (MotionBench, TimeLens/Charades/ActivityNet, StreamForest/ODVBench), so those columns are partly fitted rather than independent; the architecture and most external benchmarks remain independent.

specific steps
  1. fitted input called prediction [Section 4.3 (Stage-2 data mixture) vs Section 5.1 / Table 2 (MotionBench val [56])]
    "Vript [28], PE-Video [37], TGIF [53], CLEVRER [54], Something-Something-v2 [43], EgoIT [55], MotionBench [56], ActivityNet [41], LaSOT/GOT-10k [57, 58], and related tracking/localization datasets broaden coverage over generic video captioning, action recognition, procedural/egocentric understanding, object tracking, physical/causal reasoning, and fine motion discrimination."

    MotionBench is explicitly placed in the Stage-2 training mixture, and Table 2 then reports 'MotionBench val' as an evaluation column. The model parameters are fitted on MotionBench data, so the reported MotionBench score is a fitted quantity rather than a prediction on an untouched distribution. No split-disjointness statement is provided, making the +3.1 advantage over Qwen3-VL-4B on this column non-independent.

  2. fitted input called prediction [Section 4.3/4.4 (Stage-2/3 data: 'TimeLens-100K [14]'; Stage-0/2 include Charades [42] and ActivityNet [41]) vs Section 5.1 / Table 2 (TimeLens suite)]
    "We further evaluate temporal grounding on the TimeLens suite [14], covering Charades-STA [72], ActivityNet Captions [73], and QVHighlights [74]"

    The Stage-2 and Stage-3 training mixtures explicitly include TimeLens-100K [14], and Stage-0/Stage-2 also include Charades [42] and ActivityNet [41] video sources. The temporal-grounding evaluation is then run on the TimeLens suite, which the paper itself defines as Charades-STA, ActivityNet Captions, and QVHighlights. Since TimeLens-100K shares citation [14] with the evaluation suite and no split-disjointness analysis is reported, the large TimeLens gains (+9.7/+6.4/+8.3 over Qwen3-VL-4B; +16.4/+29.8/+34.3 over VideoChat-Flash-7B) are construction-dependent unless the released splits are provably disjoint.

  3. fitted input called prediction [Section 4.4 (Stage-3 data: StreamForest [19]) and Figure 8 vs Section 5.2 / Table 3 (ODVBench [19])]
    "Additional video datasets, including TimeLens-100K [14], Molmo2 [5], Vript [28], LLaVA-Video [26], CinePile [51], MotionBench [56], TGIF/TVQA-style QA [49, 53], AVA/EgoQA [59, 60], and StreamForest [19], provide complementary coverage of subtitle QA, dense captioning, motion understanding, tracking, spatial grounding, and driving-scene reasoning."

    Stage-3 trains on StreamForest [19], and Figure 8 shows that VideoChat3-OL617K contains 272,424 StreamForest General and 46,538 StreamForest Drive instances. The streaming evaluation then reports ODVBench [19] as a headline metric (72.3, +12.4 over StreamForest). ODVBench comes from the same cited work as the training data, so the ODVBench advantage is not independent of the fitted streaming data. This is compounded by Appendix Table 11, which modifies the bounding-box output range to [0,1000] for ODVBench/OVBench while most baselines are quoted from original papers.

full rationale

The core architecture and efficiency results are self-contained and not circular: I3D-ViT's 16x compression is a design property, and the efficiency table (Table 4) is a direct measurement independent of any benchmark fit. The general-video understanding claim is also substantially supported by external benchmarks not present in the training mixtures, e.g., Video-MME, LVBench, TempCompass, TOMATO, MMVU, VUE-TR, MomentSeeker, OVOBench, and StreamingBench. The circularity is partial and concentrated in the columns where a benchmark or its constituent source data is listed in the training recipe: MotionBench, the TimeLens suite / Charades / ActivityNet, and StreamForest / ODVBench. These contaminated columns happen to include some of the largest reported differentials, so the headline 'surpasses prior open-source models' is partly fitted rather than fully independent. Self-citation alone (TimeLens, StreamForest, River, etc.) is not treated as circular; it becomes a problem only because the same cited resources appear on both the training and evaluation sides without disclosed split disjointness. Score 4 reflects partial, not wholesale, circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 1 invented entities

The central claims rest on domain assumptions about transfer of pretrained visual features, trustworthiness of LLM-generated annotations, teacher-forced state supervision transferring to inference, and VLM-based clue verification. The comparative results also depend on several hand-set architecture and evaluation hyperparameters, plus unreported data-filtering thresholds.

free parameters (6)
  • Temporal chunk size T = 4
    Default temporal pooling factor; combined with 2x2 spatial merging it yields the 16x compression quoted throughout. No sensitivity sweep is reported. Section 2.2.
  • Streaming resolution quotas B_low and B_high = 224^2 / 448^2 pixels
    Hand-selected controller thresholds. The ablation compares fixed low, fixed high, and dynamic policies, but does not sweep the quota values. Section 2.3.
  • Standby high-budget frame count = 3 for OVO-Timing, 1 for ProactiveVQA
    Per-benchmark evaluation hyperparameter that directly affects proactive-response scores. Appendix Table 12.
  • State-supervision continuation ratio = equal number of continuation and transition positions (2:2:1 target)
    Balancing factor for the state-token loss. No alternative ratios are ablated. Section 4.4.
  • Benchmark token and pixel budgets = 80K/128K visual tokens; 448^2-768^2 pixels per frame
    Per-benchmark evaluation configurations chosen by the authors. They are applied to VideoChat3 and Qwen3-VL, but most other baselines are quoted from original papers. Appendix Tables 10-12.
  • Data-pipeline filtering thresholds = not reported
    Minimum/maximum segment durations, semantic-similarity filters, and generic-caption filters determine LV116K and OL617K quality, but their exact values are not specified. Sections 3.3-3.4.
axioms (4)
  • domain assumption Inflating a pretrained image ViT to chunk-wise 3D attention preserves image-level perception while improving video perception.
    This is the central architectural bet. It is empirically supported by Table 5, but not derived; image benchmarks are close while video benchmarks improve.
  • domain assumption LLM-rewritten academic annotations, after judge-model verification, remain faithful to the original labels.
    The quality of the 2M-sample academic dataset rests on Qwen3-VL-235B rewriting and judging. No human evaluation of the rewritten annotations is reported. Section 3.2.
  • domain assumption Teacher-forced state-token supervision transfers to closed-loop inference, where predicted states drive the next window's resolution.
    At inference, the controller uses predicted s_t rather than ground truth. The end-to-end model is tested, but the transfer is not isolated. Sections 2.3 and 4.4.
  • domain assumption VLM clue localization and verification reliably identify the temporal intervals that contain the evidence needed for streaming answers.
    The OL617K pipeline assumes a VLM can localize and verify evidence intervals, then filters on whether the answer is inferable from the cropped interval. Sections 3.4.
invented entities (1)
  • Response-state tokens </Silence>, </Standby>, </Response> no independent evidence
    purpose: Special output vocabulary symbols that control streaming behavior, response timing, and the next window's visual resolution.
    These are internal trained tokens with no falsifiable handle outside the paper; they are teacher-forced during training and predicted at inference.

pith-pipeline@v1.3.0-alltime-deepseek · 34541 in / 13240 out tokens · 132632 ms · 2026-08-02T00:39:13.782404+00:00 · methodology

0 comments
read the original abstract

Recent advances in video understanding have spanned motion, long video, and streaming interaction, driving this field toward real-world applications. Despite this progress, current open-source models remain limited in several ways. They often struggle to generalize across diverse video types, making them effective only in specific domains. High computational demands further restrict their efficiency and scalability. Moreover, most models are only partially open, with key components such as training code, strategy, or datasets unavailable, which hinders reproducibility and slows community-driven development. To address these issues, we introduce VideoChat3, a fully open, efficient, and generalist video-centric MLLM. VideoChat3 advances video understanding through two complementary designs. For efficiency, we introduce Inflated 3D Vision Transformer (I3D-ViT) and Adaptive Frame Resolution for Streaming Video Perception, which enables efficient spatiotemporal representation and reduces the cost of processing video inputs during training and inference. For effectiveness, we develop a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets: VideoChat3-Academic2M, VideoChat3-LV116K, and VideoChat3-OL617K, covering general, long-form, and streaming video scenarios, improving the model's generalization across domains. By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts with only 4B parameters and higher efficiency.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

122 extracted references · 57 linked inside Pith

  1. [1]

    Qwen3-vl technical report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 , 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Internvideo2.5: Empowering video mllms with long and rich context modeling

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvideo2.5: Empowering video mllms with long and rich context modeling. arXiv preprint arXiv:2501.12386 , 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Videochat-flash: Hierarchical compression for long-context video modeling

    Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. Videochat-flash: Hierarchical compression for long-context video modeling. arXiv preprint arXiv:2501.00574 , 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Perceptionlm: Open-access data and models for detailed visual understanding

    Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi, Triantafyllos Afouras, Tushar Nagarajan, Muhammad Maaz, Yale Song, Tengyu Ma, Shuming Hu, Suyog Jain, et al. Perceptionlm: Open-access data and models for detailed visual understanding. arXiv preprint arXiv:2504.13180 , 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Molmo2: Open weights and data for vision-language models with video understanding and grounding

    Christopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park, Mohammadreza Salehi, Rohun Tripathi, Sangho Lee, Zhongzheng Ren, Chris Dongjoo Kim, Yinuo Yang, et al. Molmo2: Open weights and data for vision-language models with video understanding and grounding. arXiv preprint arXiv:2601.10611 , 2026

    Pith/arXiv arXiv 2026

  6. [6]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Ke-Yang Chen, Kexin Yang, Mei Li, Min Xue...

    Pith/arXiv arXiv 2024

  7. [7]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

    Pith/arXiv arXiv 2025

  8. [8]

    Videochat-r1

    Ziang Yan, Yinan He, Xinhao Li, Zhengrong Yue, Xiangyu Zeng, Yali Wang, Yu Qiao, Limin Wang, and Yi Wang. Videochat-r1. 5: Visual test-time scaling to reinforce multimodal reasoning by iterative perception. Advances in Neural Information Processing Systems , 38:119152–119184, 2026

    2026

  9. [9]

    Video-xl-2: Towards very long-video understanding through task-aware kv sparsification

    Minghao Qin, Xiangrui Liu, Zhengyang Liang, Yan Shu, Huaying Yuan, Juenjie Zhou, Shitao Xiao, Bo Zhao, and Zheng Liu. Video-xl-2: Towards very long-video understanding through task-aware kv sparsification. arXiv preprint arXiv:2506.19225 , 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Video-o3: Native interleaved clue seeking for long video multi-hop reasoning

    Xiangyu Zeng, Zhiqiu Zhang, Yuhan Zhu, Xinhao Li, Zikang Wang, Changlian Ma, Qingyu Zhang, Zizheng Huang, Kun Ouyang, Tianxiang Jiang, et al. Video-o3: Native interleaved clue seeking for long video multi-hop reasoning. arXiv preprint arXiv:2601.23224 , 2026

    Pith/arXiv arXiv 2026

  11. [11]

    Spacer: Reinforcing mllms in video spatial reasoning

    Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. Spacer: Reinforcing mllms in video spatial reasoning. arXiv preprint arXiv:2504.01805 , 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Timechat: A time-sensitive multimodal large language model for long video understanding

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14313–14323, 2024

    2024

  13. [13]

    Timesuite: Improving mllms for long video understanding via grounded tuning

    Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Yi Wang, et al. Timesuite: Improving mllms for long video understanding via grounded tuning. In International Conference on Learning Representations , volume 2025, pages 38057–38081, 2025

    2025

  14. [14]

    Timelens: Rethinking video temporal grounding with multimodal llms

    Jun Zhang, Teng Wang, Yuying Ge, Yixiao Ge, Xinhao Li, Ying Shan, and Limin Wang. Timelens: Rethinking video temporal grounding with multimodal llms. arXiv preprint arXiv:2512.14698 , 2025. 22

    arXiv 2025

  15. [15]

    Videollm-online: Online video large language model for streaming video

    Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online: Online video large language model for streaming video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18407–18418, 2024

    2024

  16. [16]

    Flash-vstream: Memory-based real-time understanding for long video streams

    Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-vstream: Memory-based real-time understanding for long video streams. arXiv preprint arXiv:2406.08085 , 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Online video understanding: Ovbench and videochat-online

    Zhenpeng Huang, Xinhao Li, Jiaqi Li, Jing Wang, Xiangyu Zeng, Cheng Liang, Tao Wu, Xi Chen, Liang Li, and Limin Wang. Online video understanding: Ovbench and videochat-online. In Proceedings of the Computer Vision and Pattern Recognition Conference , pages 3328–3338, 2025

    2025

  18. [18]

    Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reac- tion

    Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reac- tion. In Proceedings of the Computer Vision and Pattern Recognition Conference , pages 24045–24055, 2025

    2025

  19. [19]

    Streamforest: Efficient online video understanding with persistent event memory

    Xiangyu Zeng, Kefan Qiu, Qingyu Zhang, Xinhao Li, Jing Wang, Jiaxin Li, Ziang Yan, Kun Tian, Meng Tian, Xinhai Zhao, et al. Streamforest: Efficient online video understanding with persistent event memory. Advances in Neural Information Processing Systems , 38:75804–75835, 2026

    2026

  20. [20]

    Streaming video instruction tuning

    Jiaer Xia, Peixian Chen, Mengdan Zhang, Xing Sun, and Kaiyang Zhou. Streaming video instruction tuning. arXiv preprint arXiv:2512.21334 , 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Streambridge: Turning your offline video large language model into a proactive streaming assistant

    Haibo Wang, Bo Feng, Zhengfeng Lai, Mingze Xu, Shiyu Li, Weifeng Ge, Afshin Dehghan, Meng Cao, and Ping Huang. Streambridge: Turning your offline video large language model into a proactive streaming assistant. Advances in Neural Information Processing Systems , 38:132332–132359, 2026

    2026

  22. [22]

    Gemini 3: News and announcements

    Google. Gemini 3: News and announcements. https://blog.google/products-and-platforms/products/ gemini/gemini-3-collection/ , 2025

    2025

  23. [23]

    Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2. 5: Visual agentic intelligence. arXiv preprint arXiv:2602.02276 , 2026

    Pith/arXiv arXiv 2026

  24. [24]

    Bytedance Seed. Seed1. 8 model card: Towards generalized real-world agency. arXiv preprint arXiv:2603.20633 , 2026

    Pith/arXiv arXiv 2026

  25. [25]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, Ji Ma, Jiaqi Wang, Xiaoyi Dong, Hang Yan, Hewei Guo, Conghui He, Botian Shi, Zhenjiang Jin, Chao Xu, Bin Wang, Xingjian Wei, Wei Li, Wenjian Zhang, Bo Zhang, Pinlong Cai, Licheng Wen, Xiangchao Yan, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu...

  26. [26]

    Llava-video: Video instruction tuning with synthetic data

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Llava-video: Video instruction tuning with synthetic data. Transactions on Machine Learning Research , 2025

    2025

  27. [27]

    Spoken moments: Learning joint audio-visual representations from video descriptions

    Mathew Monfort, SouYoung Jin, Alexander Liu, David Harwath, Rogerio Feris, James Glass, and Aude Oliva. Spoken moments: Learning joint audio-visual representations from video descriptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14871–14881, 2021

    2021

  28. [28]

    Vript: A video is worth thousands of words, 2024

    Dongjie Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. Vript: A video is worth thousands of words, 2024. URL https://arxiv.org/abs/2406.06040

    Pith/arXiv arXiv 2024

  29. [29]

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang, Dehao Zhang, Dikang Du, Dongliang Wang, Enming Yuan, Enzhe Lu, Fang Li, Flood Sung, Guangda Wei, Guokun Lai, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haoning Wu, Haotian Yao, Haoyu Lu, Heng Wang, Hongcheng Gao, Huabi...

    Pith/arXiv arXiv 2025

  30. [30]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023

    2023

  31. [31]

    Caprl: Stimulating dense image caption capabilities via reinforcement learning

    Long Xing, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jianze Liang, Qidong Huang, Jiaqi Wang, Feng Wu, and Dahua Lin. Caprl: Stimulating dense image caption capabilities via reinforcement learning. arXiv preprint arXiv:2509.22647, 2025

    arXiv 2025

  32. [32]

    Sharegpt4v: Improving large multi-modal models with better captions, 2023

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions, 2023. URL https://arxiv.org/abs/2311.12793

    Pith/arXiv arXiv 2023

  33. [33]

    Densefusion-1m: Merging vision experts for comprehensive multimodal perception, 2024

    Xiaotong Li, Fan Zhang, Haiwen Diao, Yueze Wang, Xinlong Wang, and Ling-Yu Duan. Densefusion-1m: Merging vision experts for comprehensive multimodal perception, 2024. URL https://arxiv.org/abs/2407. 08303

    2024

  34. [34]

    Conceptual captions: A cleaned, hyper- nymed, image alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hyper- nymed, image alt-text dataset for automatic image captioning. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2556–2565, Melbourne, Australi...

    doi:10.18653/v1/p18-1238 2018

  35. [35]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In IEEE International Conference on Computer Vision , 2021

    2021

  36. [36]

    The kinetics human action video dataset, 2017

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, and Andrew Zisserman. The kinetics human action video dataset, 2017. URL https://arxiv.org/abs/1705.06950

    Pith/arXiv arXiv 2017

  37. [37]

    Perception encoder: The best visual embeddings are not at the output of the network, 2025

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Dollár, and Christoph Feichtenhofer. Perception encoder: The best visual embeddings are not at the output of the network, 2025. URL https...

    Pith/arXiv arXiv 2025

  38. [38]

    Tarsier2: Advancing large vision- language models from detailed video description to comprehensive video understanding, 2025

    Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advancing large vision- language models from detailed video description to comprehensive video understanding, 2025. URL https: //arxiv.org/abs/2501.07888

    Pith/arXiv arXiv 2025

  39. [39]

    Movie description, 2016

    Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. Movie description, 2016. URL https://arxiv.org/abs/1605.03705

    Pith/arXiv arXiv 2016

  40. [40]

    Vatex: A large-scale, high-quality multilingual dataset for video-and-language research, 2020

    Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. Vatex: A large-scale, high-quality multilingual dataset for video-and-language research, 2020. URL https://arxiv.org/abs/1904. 03493

    2020

  41. [41]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference on computer vision and pattern recognition, pages 961–970, 2015

    2015

  42. [42]

    Hollywood in homes: Crowdsourcing data collection for activity understanding

    Gunnar A Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding. In European conference on computer vision , pages 510–526. Springer, 2016

    2016

  43. [43]

    The” something something” video database for learning and evaluating visual common sense

    Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The” something something” video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision , pages 5842–5...

    2017

  44. [44]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 18995–19012, 2022

    2022

  45. [45]

    Sharegemini: Scaling up video caption data for multimodal large language models, June 2024

    Share. Sharegemini: Scaling up video caption data for multimodal large language models, June 2024. URL https://github.com/Share14/ShareGemini

    2024

  46. [46]

    Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o, 2024

    Erfei Cui, Yinan He, Zheng Ma, Zhe Chen, Hao Tian, Weiyun Wang, Kunchang Li, Yi Wang, Wenhai Wang, Xizhou Zhu, Lewei Lu, Tong Lu, Yali Wang, Limin Wang, Yu Qiao, and Jifeng Dai. Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o, 2024. URL https://sharegpt4o.github.io/. 24

    2024

  47. [47]

    Bee: A high-quality corpus and full-stack suite to unlock advanced fully open mllms, 2026

    Yi Zhang, Bolin Ni, Xin-Sheng Chen, Heng-Rui Zhang, Yongming Rao, Houwen Peng, Qinglin Lu, Han Hu, Meng-Hao Guo, and Shi-Min Hu. Bee: A high-quality corpus and full-stack suite to unlock advanced fully open mllms, 2026. URL https://arxiv.org/abs/2510.13795

    arXiv 2026

  48. [48]

    Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groen- eveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Sol- daini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, Pradeep Dasigi, Robert Berry, Saumya Malik, Saurabh Shah, Scott Geng, ...

  49. [49]

    Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L. Berg. Tvqa: Localized, compositional video question answering, 2019. URL https://arxiv.org/abs/1809.01696

    Pith/arXiv arXiv 2019

  50. [50]

    Finevideo

    Miquel Farré, Andi Marafioti, Lewis Tunstall, Leandro Von Werra, and Thomas Wolf. Finevideo. https: //huggingface.co/datasets/HuggingFaceFV/finevideo, 2024

    2024

  51. [51]

    Cinepile: A long video question answering dataset and benchmark

    Ruchit Rawal, Khalid Saifullah, Miquel Farré, Ronen Basri, David Jacobs, Gowthami Somepalli, and Tom Goldstein. Cinepile: A long video question answering dataset and benchmark. arXiv preprint arXiv:2405.08813 , 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Videogpt+: Integrating image and video encoders for enhanced video understanding, 2024

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Videogpt+: Integrating image and video encoders for enhanced video understanding, 2024. URL https://arxiv.org/abs/2406.09418

    Pith/arXiv arXiv 2024

  53. [53]

    Tgif-qa: Toward spatio-temporal reasoning in visual question answering, 2017

    Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio-temporal reasoning in visual question answering, 2017. URL https://arxiv.org/abs/1704.04497

    Pith/arXiv arXiv 2017

  54. [54]

    Tenenbaum

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. Clevrer: Collision events for video representation and reasoning, 2020. URL https://arxiv.org/abs/1910. 01442

    2020

  55. [55]

    Exo2ego: Exocentric knowledge guided mllm for egocentric video understanding, 2025

    Haoyu Zhang, Qiaohui Chu, Meng Liu, Haoxiang Shi, Yaowei Wang, and Liqiang Nie. Exo2ego: Exocentric knowledge guided mllm for egocentric video understanding, 2025. URL https://arxiv.org/abs/2503.09143

    arXiv 2025

  56. [56]

    Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models

    Wenyi Hong, Yean Cheng, Zhuoyi Yang, Weihan Wang, Lefan Wang, Xiaotao Gu, Shiyu Huang, Yuxiao Dong, and Jie Tang. Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8450–8460, 2025

    2025

  57. [57]

    Lasot: A high-quality benchmark for large-scale single object tracking

    Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5374–5383, 2019

    2019

  58. [58]

    Got-10k: A large high-diversity benchmark for generic object tracking in the wild

    Lianghua Huang, Xin Zhao, and Kaiqi Huang. Got-10k: A large high-diversity benchmark for generic object tracking in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43(5):1562–1577, May

  59. [59]

    Ross, Carl Vondrick, Caroline Pantofaru, Yeqing Li, Sudheendra Vijaya- narasimhan, George Toderici, Susanna Ricco, Rahul Sukthankar, Cordelia Schmid, and Jitendra Malik

    Chunhui Gu, Chen Sun, David A. Ross, Carl Vondrick, Caroline Pantofaru, Yeqing Li, Sudheendra Vijaya- narasimhan, George Toderici, Susanna Ricco, Rahul Sukthankar, Cordelia Schmid, and Jitendra Malik. A va: A video dataset of spatio-temporally localized atomic visual actions, 2018. URL https://arxiv.org/abs/1705. 08421

    2018

  60. [60]

    Egovqa-an egocentric video question answering benchmark dataset

    Chenyou Fan. Egovqa-an egocentric video question answering benchmark dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops , pages 0–0, 2019

    2019

  61. [61]

    Learning transferable temporal primitives for video reasoning via synthetic videos, 2026

    Songtao Jiang, Sibo Song, Chenyi Zhou, Yuan Wang, Ruizhe Chen, Tongkun Guan, Ruilin Luo, Yan Zhang, Zhihang Tang, Yuchong Sun, Hang Zhang, Zhibo Yang, Shuai Bai, Junyang Lin, and Zuozhu Liu. Learning transferable temporal primitives for video reasoning via synthetic videos, 2026. URL https://arxiv.org/abs/ 2603.17693. 25

    arXiv 2026

  62. [62]

    Tomato: Assessing visual temporal reasoning capabilities in multimodal foundation models

    Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, and Arman Cohan. Tomato: Assessing visual temporal reasoning capabilities in multimodal foundation models. In The Thirteenth International Conference on Learning Representations , 2025

    2025

  63. [63]

    Tvbench: Re- designing video-language evaluation

    Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees GM Snoek, and Yuki M Asano. Tvbench: Re- designing video-language evaluation. arXiv preprint arXiv:2410.07752 , 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Tempcompass: Do video llms really understand videos? In Findings of the Association for Computational Linguistics: ACL 2024 , pages 8731–8772

    Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video llms really understand videos? In Findings of the Association for Computational Linguistics: ACL 2024 , pages 8731–8772. Association for Computational Linguistics, 2024

    2024

  65. [65]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yun- hang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Caifeng Shan, Ran He, and Xing Sun. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. I...

    2025

  66. [66]

    Lvbench: An extreme long video understanding benchmark

    Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Lvbench: An extreme long video understanding benchmark. arXiv preprint arXiv:2406.08035 , 2024

    Pith/arXiv arXiv 2024

  67. [67]

    Videoeval-pro: Robust and realistic long video understanding evaluation

    Wentao Ma, Weiming Ren, Yiming Jia, Zhuofeng Li, Ping Nie, Ge Zhang, and Wenhu Chen. Videoeval-pro: Robust and realistic long video understanding evaluation. arXiv preprint arXiv:2505.14640 , 2025

    Pith/arXiv arXiv 2025

  68. [68]

    Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos

    Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. arXiv preprint arXiv:2501.13826 , 2025

    Pith/arXiv arXiv 2025

  69. [69]

    Mmvu: Measuring expert-level multi-discipline video understanding

    Yilun Zhao, Haowei Zhang, Lujing Xie, Tongyan Hu, Guo Gan, Yitao Long, Zhiyuan Hu, Weiyuan Chen, Chuhan Li, Zhijian Xu, et al. Mmvu: Measuring expert-level multi-discipline video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference , pages 8475–8489, 2025

    2025

  70. [70]

    Minerva: Evaluating complex video reasoning

    Arsha Nagrani, Sachit Menon, Ahmet Iscen, Shyamal Buch, Ramin Mehran, Nilpa Jha, Anja Hauth, Yukun Zhu, Carl Vondrick, Mikhail Sirotenko, et al. Minerva: Evaluating complex video reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 23968–23978, 2025

    2025

  71. [71]

    Video-mme-v2: Towards the next stage in benchmarks for comprehensive video understanding

    Chaoyou Fu, Haozhi Yuan, Yuhao Dong, Yi-Fan Zhang, Yunhang Shen, Xiaoxing Hu, Xueying Li, Jinsen Su, Chengwu Long, Xiaoyao Xie, et al. Video-mme-v2: Towards the next stage in benchmarks for comprehensive video understanding. arXiv preprint arXiv:2604.05015 , 2026

    Pith/arXiv arXiv 2026

  72. [72]

    Tall: Temporal activity localization via language query

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision , pages 5267–5275, 2017

    2017

  73. [73]

    Dense-captioning events in videos

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision , pages 706–715, 2017

    2017

  74. [74]

    Detecting moments and highlights in videos via natural language queries

    Jie Lei, Tamara L Berg, and Mohit Bansal. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems , 34:11846–11858, 2021

    2021

  75. [75]

    Vidi: Large multimodal models for video understanding and editing

    Vidi Team, Celong Liu, Chia-Wen Kuo, Dawei Du, Fan Chen, Guang Chen, Jiamin Yuan, Lingxi Zhang, Lu Guo, Lusha Li, et al. Vidi: Large multimodal models for video understanding and editing. arXiv preprint arXiv:2504.15681, 2025

    Pith/arXiv arXiv 2025

  76. [76]

    Vidi2: Large multimodal models for video understanding and creation

    Vidi Team, Celong Liu, Chia-Wen Kuo, Chuang Huang, Dawei Du, Fan Chen, Guang Chen, Haoji Zhang, Haojun Zhao, Lingxi Zhang, et al. Vidi2: Large multimodal models for video understanding and creation. arXiv preprint arXiv:2511.19529 , 2025

    arXiv 2025

  77. [77]

    Momentseeker: A task-oriented benchmark for long-video moment retrieval

    Huaying Yuan, Jian Ni, Zheng Liu, Yueze Wang, Junjie Zhou, Zhengyang Liang, Bo Zhao, Zhao Cao, Ji-Rong Wen, and Zhicheng Dou. Momentseeker: A task-oriented benchmark for long-video moment retrieval. Advances in Neural Information Processing Systems , 38, 2026

    2026

  78. [78]

    GPT-5 system card

    OpenAI. GPT-5 system card. https://openai.com/index/gpt-5-system-card/ , 2025

    2025

  79. [79]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 , 2025

    Pith/arXiv arXiv 2025

  80. [80]

    Introducing claude sonnet 4.5

    Anthropic. Introducing claude sonnet 4.5. https://www.anthropic.com/news/claude-sonnet-4-5 , 2025. 26

    2025

Showing first 80 references.