Pith. sign in

REVIEW 3 major objections 62 references

Treating video as events and matching embeddings to placeholders lets a model answer “when” questions with timestamps and fewer tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 12:09 UTC pith:XWRVNGED

load-bearing objection Solid, compositional SOTA for RTL with real token savings; the train/infer gap on Step-2 is real but not fatal, and the fixed-N event model is the bigger long-term limit. the 3 major comments →

arxiv 2607.04872 v1 pith:XWRVNGED submitted 2026-07-06 cs.CV cs.AI

EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization

Youngkil Song , Yoonjae Baek , Dongwon Kim , Inho Kim , Dongkeun Kim , Suha Kwak This is my paper
classification cs.CV cs.AI
keywords reasoning temporal localizationevent-centric tokenizationvideo chain-of-thoughtembedding matchingActivityNet-RTLReXTimegrounded video QAvisual token efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Reasoning temporal localization asks a video model to give a free-form answer that already includes the start and end times that support it, so reasoning and grounding cannot be separated. EventCoT does this by first turning the video into a small set of event tokens, selecting the events that matter for the question, and only then generating the answer while grounding start and end times by matching special placeholder tokens to frame embeddings. The paper shows this two-step chain of thought reaches the best reported scores on ActivityNet-RTL while using far fewer visual tokens than prior work, and transfers zero-shot to the grounded video QA benchmark ReXTime. A sympathetic reader cares because the same design both cuts token cost and disentangles temporal grounding from text generation, which the paper argues is why joint optimization fails in ordinary video LLMs.

Core claim

EventCoT claims that representing a video as compact event tokens, selecting question-relevant events as an intermediate reasoning step, and grounding the supporting interval by embedding matching of placeholder tokens yields state-of-the-art reasoning temporal localization with substantially fewer visual tokens than frame-uniform or agentic baselines.

What carries the argument

Event-centric chain-of-thought: DPC-KNN boundary detection plus event-context attention produces N event tokens; Step 1 matches a ⟨segment_key⟩ embedding to those tokens; Step 2 matches ⟨start⟩/⟨end⟩ embeddings to frame features of the selected events to insert timestamps into a single generated answer.

Load-bearing premise

That a fixed split of every video into sixteen events from one hundred uniformly sampled frames is a good enough semantic unit for the question-relevant selection step to work across the videos the method is tested on.

What would settle it

On ActivityNet-RTL, replace the learned event boundaries with uniform intervals of the same count and retrain; if mIoU and GPT-4 score collapse to LITA-level or below, the event-unit claim is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper introduces EventCoT, an event-centric chain-of-thought framework for reasoning temporal localization (RTL), where a model must jointly produce a free-form answer and the supporting time interval in one response. The method (i) segments a video into N non-overlapping events via a temporal-proximity-aware, power-scaled DPC-KNN boundary detector and an event-context attention tokenizer, (ii) selects question-relevant events by embedding matching of a ⟨segment_key⟩ placeholder against event tokens (Step 1), and (iii) generates the answer while grounding ⟨start⟩/⟨end⟩ by matching placeholder embeddings to frame features within selected (plus randomly sampled unselected) events (Step 2). On ActivityNet-RTL the Vicuna-7B model reports SOTA mIoU 58.5 and GPT-4 score 48.7 with far fewer visual tokens than LITA; zero-shot ReXTime results are also strong, especially with a Qwen2.5-7B backbone. Ablations support events over uniform intervals, embedding matching over discrete time tokens, and the main loss and component choices.

Significance. If the results hold under fair scrutiny, the work is a clear and useful contribution to efficient video-LLM temporal reasoning. RTL is harder than post-hoc grounded VQA because localization must be embedded in the generated answer; showing that event-level intermediate reasoning plus disentangled embedding matching can beat both dense uniform tokenization (LITA) and costly agentic retrieval, while cutting visual tokens by roughly an order of magnitude, is practically and scientifically valuable. Strengths include systematic ablations (events vs uniform, matching vs discrete tokens, losses, boundary detectors, components), explicit efficiency measurements (Fig. 1, Fig. S1), zero-shot transfer to ReXTime, and an honest Limitations section on fixed N/T. The design is reproducible in outline (hyperparameters fixed across datasets; training mixture and prompts documented).

major comments (3)
  1. Sec. 3.3.1 and Eq. (8): Step 2 is trained with S = events that overlap the ground-truth interval (oracle selection), plus a random 30% of unselected events R, while inference uses noisy Step-1 S with the same R sampling. This train–inference gap is load-bearing for the event-centric CoT claim. Table S6 shows that when Step-1 IoU is Low (26.3% of samples), Step-2 mIoU falls to 16.8, so the headline 58.5 mIoU mixes a strong high-IoU regime with weak recovery. The main text should report (i) how often predicted S recovers the GT-overlapping set (precision/recall of event selection under the inference threshold), (ii) Step-2 performance when trained and tested with predicted S only (no GT-overlap oracle), and (iii) an ablation that removes R at inference while keeping it at training (or vice versa). Without these, it remains unclear how much of the SOTA gain is genuine selection versus train
  2. Implementation Details and Limitations: N=16 events and T=100 frames are fixed for all videos, with hand-set DPC-KNN parameters (α=0.3, k=5, p=2). Table S3 shows N strongly affects localization (mIoU 39.5 at N=8 vs 53.0 at N=32), and the Limitations section correctly notes specialization to relatively short videos. The SOTA and efficiency claims on ActivityNet-RTL are therefore conditioned on a regime where a fixed coarse partition is adequate. Either restrict the claim explicitly to short/medium ActivityNet-scale videos, or provide at least one experiment with adaptive N (or longer videos / streaming) so that the event unit is not an untested free parameter of the central result.
  3. Table 1 comparison protocol: several strong baselines are zero-shot foundation models or agentic systems with different visual budgets and interaction loops, while EventCoT and LITA are fine-tuned on the same RTL-oriented mixture. Fine-tuned Qwen3.5-9B / TimeLens improve localization but hurt GPT-4 score, which supports the disentangling argument, yet the paper should state more carefully which comparisons are same-data fine-tunes versus zero-shot, and report EventCoT’s token count and wall-clock under a matched frame budget (T=100) against the fine-tuned open models only, so the efficiency–accuracy Pareto claim is not inflated by heterogeneous protocols.

Circularity Check

0 steps flagged

No circular derivation: EventCoT is an empirical ML method whose SOTA claims are measured on held-out benchmarks, not forced by construction from fitted inputs or self-citation.

full rationale

EventCoT’s load-bearing claims are empirical (ActivityNet-RTL mIoU/GPT-4 score; ReXTime zero-shot) obtained by supervised multi-turn training (L_lm + λ_sel L_sel + λ_ts L_ts) and external evaluation. Event boundary detection (DPC-KNN with fixed α, k, p), event tokens, embedding matching of ⟨segment_key⟩/⟨start⟩/⟨end⟩, and the S∪R frame set (Eq. 8) are design choices whose quality is measured against held-out labels, not quantities that algebraically equal their training targets. Training Step 2 on GT-overlapping events plus random R is a standard teacher-forcing / distribution-matching choice; it may create a train–inference gap (a validity concern), but it does not make reported mIoU equal a fitted constant by construction. The MATR (Song et al., 2024) citation only motivates DIoU, which is independently standard (ActionFormer); it is not a uniqueness theorem or load-bearing premise. Same training mixture as LITA is normal benchmarking. No self-definitional loop, no fitted-input-as-prediction, no uniqueness imported from the authors. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

11 free parameters · 5 axioms · 3 invented entities

Empirical systems paper: claims rest on standard VLM building blocks plus many hand-chosen hyperparameters for event segmentation and matching, and on the modeling choice that fixed-count semantic events are the right intermediate for RTL. No new physical entities; invented pieces are architectural modules.

free parameters (11)
  • Number of events N = 16
    Fixed to 16 for all videos; ablation shows N strongly affects mIoU (Table S3). Not learned per video.
  • Sampled frames T = 100
    Uniform frame budget fixed at 100 following LITA; affects boundary density and Step-2 resolution.
  • Temporal proximity weight α in d(i,j) = 0.3
    Hand-set weight mixing feature and temporal distance for boundary detection (Eq. 1).
  • k for DPC-KNN local density = 5
    Neighbor count for ρ_i; fixed without per-dataset search.
  • Power scaling p on δ = 2
    Exponent on normalized distance factor to sharpen peaks (Eq. 4).
  • Event context extension β = 3
    Extra frames attended outside each event interval.
  • Selection threshold factor γ = 0.5
    Events kept if r_n > γ·max r_n.
  • Matching temperature τ = 0.07
    Temperature in sigmoid/softmax embedding matching.
  • Loss weights λ_sel, λ_ts = 0.1 each
    Relative weights of selection and timestamp losses vs LM loss.
  • Unselected-event sampling ratio R = 30%
    Fraction of non-selected events mixed into Step-2 inputs to avoid boundary shortcuts.
  • Gaussian label σ for L_ce = 0.05
    Smoothing width for start/end frame targets.
axioms (5)
  • domain assumption Human-like event segmentation: continuous activity is usefully modeled as discrete non-overlapping semantic events for intermediate reasoning (Zacks & Swallow 2007).
    Motivates event-centric tokenization in Introduction and Sec. 3.1; not proven for RTL optimality.
  • domain assumption CLIP-L/14 frame features plus mean pooling and RoPE suffice as the visual substrate for boundary detection and matching.
    Sec. 3.1.1 freezes CLIP and projects; all later stages depend on this representation.
  • domain assumption Placeholder hidden states can be aligned to visual embeddings by cosine/dot-product matching to recover discrete events and timestamps (building on LISA/ET-Bench style designs).
    Core of Steps 1–2 (Eqs. 7–9); assumes LLM final-layer states remain discriminative for matching without projectors.
  • ad hoc to paper During training, providing GT-overlapping events plus random unselected events is a valid surrogate for noisy Step-1 inference.
    Sec. 3.3.1; bridges train/test gap but is a design choice, not a theorem.
  • domain assumption Standard next-token LM loss plus BCE selection and CE+DIoU localization jointly optimize RTL without destructive interference when stages are multi-turn.
    Sec. 3.4 overall objective; supported empirically but assumed for end-to-end training.
invented entities (3)
  • Event-centric tokenizer (event point + event context attention with extended mask) no independent evidence
    purpose: Compress each detected event into one discriminative token for Step-1 selection.
    Architectural module trained from scratch (Fig. 4, Eq. 6); no independent physical existence.
  • Placeholder tokens ⟨segment_key⟩, ⟨start⟩, ⟨end⟩ used for two-step embedding matching no independent evidence
    purpose: Disentangle event selection and timestamp grounding from free-form answer generation.
    Vocabulary extensions whose hidden states are matched to visual embeddings; roles are paper-specific.
  • Temporal proximity-aware + power-scaled DPC-KNN boundary score for video events no independent evidence
    purpose: Detect semantic event boundaries without shot-cut detectors.
    Adaptation of DPC-KNN with α-weighted distance and p-scaled δ; evaluated by boundary recall on ActivityNet-RTL.

pith-pipeline@v1.1.0-grok45 · 32428 in / 3885 out tokens · 34286 ms · 2026-07-11T12:09:56.967935+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization." pith.science (2026). https://pith.science/paper/XWRVNGED

@misc{pith2026260704872,
  author       = {Pith},
  title        = {Pith review of: EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XWRVNGED}},
  note         = {Machine review of arXiv:2607.04872}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, so high-level reasoning and precise temporal grounding must be produced jointly in a single response. To tackle this challenging task, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization of the input video to convert it into compact event tokens, enabling efficient identification of question-relevant events. It then reasons within the identified events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL for reasoning temporal localization while using substantially fewer visual tokens than previous work. To verify its general performance, we further evaluate EventCoT on the grounded video question answering benchmark ReXTime, where it attains strong zero-shot results.

Figures

Figures reproduced from arXiv: 2607.04872 by Dongkeun Kim, Dongwon Kim, Inho Kim, Suha Kwak, Yoonjae Baek, Youngkil Song.

Figure 1
Figure 1. Figure 1: Advantages of using events in RTL in terms of visual token usage (a), inference time (b), and perfor [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of EventCoT. EventCoT first tokenizes a video into compact event tokens via event-centric [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Event boundary detection results on ActivityNet-RTL (Huang et al., 2024b). Numerical val￾ues indicate boundary recall computed within a toler￾ance of ±5% of the video duration. candidates. However, relying solely on such dis￾tances leads to unreliable boundary detection, as frames in highly dynamic scenes appear distinct even within the same event, causing false alarms while missing subtle transitions in s… view at source ↗
Figure 4
Figure 4. Figure 4: Details of the event-centric tokenizer. By us [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative Results on ActivityNet-RTL ( [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Zero-shot qualitative results on ReXTime ( [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 14 linked inside Pith

  1. [1]

    https://api.semanticscholar.org/CorpusID:263218031 Gpt-4v(ision) system card

    2023. https://api.semanticscholar.org/CorpusID:263218031 Gpt-4v(ision) system card

    2023

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  3. [3]

    Anurag Arnab, Ahmet Iscen, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. 2025. Temporal chain of thought: Long-video understanding by thinking in frames. Proc. Advances in Neural Information Processing Systems (NeurIPS)

    2025

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    Pith/arXiv arXiv 2025

  5. [5]

    Brandon Castellano and contributors. 2025. PySceneDetect: Video Scene Cut Detection . https://www.scenedetect.com/. Version 0.6.6 (released March 9, 2025)

    2025

  6. [6]

    Keshigeyan Chandrasegaran, Agrim Gupta, Lea M Hadzic, Taran Kota, Jimming He, Crist \'o bal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, and Li Fei-Fei. 2024. Hourvideo: 1-hour video-language understanding. Proc. Advances in Neural Information Processing Systems (NeurIPS), 37:53168--53197

    2024

  7. [7]

    Jr-Jen Chen, Yu-Chien Liao, Hsi-Che Lin, Yu-Chu Yu, Yen-Chun Chen, and Frank Wang. 2024. Rextime: A benchmark suite for reasoning-across-time in videos. Proc. Advances in Neural Information Processing Systems (NeurIPS), 37:28662--28673

    2024

  8. [8]

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. 2024. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476

    Pith/arXiv arXiv 2024

  9. [9]

    Zixu Cheng, Da Li, Jian Hu, Ziquan Liu, Wei Li, and Shaogang Gong. 2026. Graphthinker: Reinforcing video reasoning with event graph thinking. arXiv preprint arXiv:2602.17555

    Pith/arXiv arXiv 2026

  10. [10]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    2023

  11. [11]

    Gheorghe Comanici et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261

    Pith/arXiv arXiv 2025

  12. [12]

    Cover and P

    T. Cover and P. Hart. 1967. https://doi.org/10.1109/TIT.1967.1053964 Nearest neighbor pattern classification . IEEE Transactions on Information Theory, 13(1):21--27

    doi:10.1109/tit.1967.1053964 1967

  13. [13]

    Mingjing Du, Shifei Ding, and Hongjie Jia. 2016. Study on density peaks clustering based on k-nearest neighbors and principal component analysis. Knowledge-Based Systems, 99:135--145

    2016

  14. [14]

    Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. 2024. Mmbench-video: A long-form multi-shot benchmark for holistic video understanding. Proc. Advances in Neural Information Processing Systems (NeurIPS), 37:89098--89124

    2024

  15. [15]

    Yongxin Guo, Jingyu Liu, Mingda Li, Qingbin Liu, Xi Chen, and Xiaoying Tang. 2025. Trace: Temporal grounding video llm via causal event modeling. Proc. International Conference on Learning Representations (ICLR)

    2025

  16. [16]

    Ayush Gupta, Anirban Roy, Rama Chellappa, Nathaniel D Bastian, Alvaro Velasquez, and Susmit Jha. 2025. Toga: Temporally grounded open-ended video qa with weak supervision. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 23593--23603

    2025

  17. [17]

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769

    Pith/arXiv arXiv 2024

  18. [18]

    Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. 2024 a . Vtimellm: Empower llm to grasp video moments. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 14271--14280

    2024

  19. [19]

    De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, and Jan Kautz. 2024 b . Lita: Language instructed temporal-localization assistant. In Proc. European Conference on Computer Vision (ECCV)

    2024

  20. [20]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    Pith/arXiv arXiv 2024

  21. [21]

    Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. 2024. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13700--13710

    2024

  22. [22]

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017. Dense-captioning events in videos. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 706--715

    2017

  23. [23]

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. 2024. Lisa: Reasoning segmentation via large language model. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9579--9589

    2024

  24. [24]

    Jie Lei, Tamara L Berg, and Mohit Bansal. 2021. Detecting moments and highlights in videos via natural language queries. Proc. Advances in Neural Information Processing Systems (NeurIPS), 34:11846--11858

    2021

  25. [25]

    Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. 2020. Tvqa+: Spatio-temporal grounding for video question answering. In Proc. Annual Meeting of the Association for Computational Linguistics (ACL)

    2020

  26. [26]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024 a . Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    Pith/arXiv arXiv 2024

  27. [27]

    Hongyu Li, Jinyu Chen, Ziyu Wei, Shaofei Huang, Tianrui Hui, Jialin Gao, Xiaoming Wei, and Si Liu. 2025. Llava-st: A multimodal large language model for fine-grained spatial-temporal understanding. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8592--8603

    2025

  28. [28]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355

    Pith/arXiv arXiv 2023

  29. [29]

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. 2024 b . Mvbench: A comprehensive multi-modal video understanding benchmark. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 22195--22206

    2024

  30. [30]

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. 2024 a . https://doi.org/10.18653/v1/2024.emnlp-main.342 Video- LL a VA : Learning united visual representation by alignment before projection . In Proc. Empirical Methods in Natural Language Processing (EMNLP), pages 5971--5984, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.emnlp-main.342 2024

  31. [31]

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. 2024 b . Vila: On pre-training for visual language models. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 26689--26699

    2024

  32. [32]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Proc. Advances in Neural Information Processing Systems (NeurIPS), 36:34892--34916

    2023

  33. [33]

    Ye Liu, Kevin Qinghong Lin, Chang Wen Chen, and Mike Zheng Shou. 2025. Videomind: A chain-of-lora agent for long video reasoning. arXiv preprint arXiv:2503.13444

    arXiv 2025

  34. [34]

    Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, and Chang W Chen. 2024. Et bench: Towards open-ended event-level video-language understanding. Proc. Advances in Neural Information Processing Systems (NeurIPS), 37:32076--32110

    2024

  35. [35]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. 2024. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585--12602

    2024

  36. [36]

    Sachit Menon, Ahmet Iscen, Arsha Nagrani, Tobias Weyand, Carl Vondrick, and Cordelia Schmid. 2025. Caviar: Critic-augmented video agentic reasoning. arXiv preprint arXiv:2509.07680

    Pith/arXiv arXiv 2025

  37. [37]

    OpenAI. 2025. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267

    Pith/arXiv arXiv 2025

  38. [38]

    Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, and Siliang Tang. 2024. Momentor: advancing video large language model with fine-grained temporal reasoning. In Proc. International Conference on Machine Learning (ICML), pages 41340--41356

    2024

  39. [39]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748--8763. PMLR

    2021

  40. [40]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. Technical report, OpenAI

    2019

  41. [41]

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. 2024. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 14313--14323

    2024

  42. [42]

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. 2024. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Proc. Advances in Neural Information Processing Systems (NeurIPS), 37:8612--8642

    2024

  43. [43]

    Youngkil Song, Dongkeun Kim, Minsu Cho, and Suha Kwak. 2024. Online temporal action localization with memory-augmented transformer. In Proc. European Conference on Computer Vision (ECCV), pages 74--91. Springer

    2024

  44. [44]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063

    2024

  45. [45]

    Qwen Team. 2026. Qwen3.5: Towards native multimodal agents. https://qwen.ai/blog?id=qwen3.5

    2026

  46. [46]

    Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Ming Ding, Xiaotao Gu, Shiyu Huang, Bin Xu, et al. 2025. Lvbench: An extreme long video understanding benchmark. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 22958--22967

    2025

  47. [47]

    Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. 2024. Videoagent: Long-form video understanding with large language model as agent. In Proc. European Conference on Computer Vision (ECCV), pages 58--76. Springer

    2024

  48. [48]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Proc. Advances in Neural Information Processing Systems (NeurIPS), 35:24824--24837

    2022

  49. [49]

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. 2021. Next-qa: Next phase of question-answering to explaining temporal actions. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9777--9786

    2021

  50. [50]

    Junbin Xiao, Angela Yao, Yicong Li, and Tat-Seng Chua. 2024. Can i trust your answer? visually grounded video question answering. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  51. [51]

    Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. 2023. Vid2seq: Large-scale pretraining of a visual language model for dense video captioning. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 10714--10726

    2023

  52. [52]

    Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2024. mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. arXiv preprint arXiv:2408.04840

    Pith/arXiv arXiv 2024

  53. [53]

    Jeffrey M Zacks and Khena M Swallow. 2007. Event segmentation. Current directions in psychological science, 16(2):80--84

    2007

  54. [54]

    Chen-Lin Zhang, Jianxin Wu, and Yin Li. 2022. Actionformer: Localizing moments of actions with transformers. In Proc. European Conference on Computer Vision (ECCV), pages 492--510. Springer

    2022

  55. [55]

    Hang Zhang, Xin Li, and Lidong Bing. 2023. Video-llama: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations, pages 543--553

    2023

  56. [56]

    Jun Zhang, Teng Wang, Yuying Ge, Yixiao Ge, Xinhao Li, Ying Shan, and Limin Wang. 2026. Timelens: Rethinking video temporal grounding with multimodal llms. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2026

  57. [57]

    Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. 2020. Distance-iou loss: Faster and better learning for bounding box regression. In Proc. AAAI Conference on Artificial Intelligence (AAAI), pages 12993--13000

    2020

  58. [58]

    Zhuo Zhi, Qiangqiang Wu, Wenbo Li, Yinchuan Li, Kun Shao, Kaiwen Zhou, et al. 2025. Videoagent2: Enhancing the llm-based agent system for long-form video understanding by uncertainty-aware cot. arXiv preprint arXiv:2504.04471

    Pith/arXiv arXiv 2025

  59. [59]

    Honglu Zhou, Xiangyu Peng, Shrikant Kendre, Michael S Ryoo, Silvio Savarese, Caiming Xiong, and Juan Carlos Niebles. 2025 a . Strefer: Empowering video llms with space-time referring and reasoning via synthetic instruction data. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 4289--4300

    2025

  60. [60]

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, et al. 2025 b . Mlvu: Benchmarking multi-task long video understanding. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13691--13701

    2025

  61. [61]

    Luowei Zhou, Chenliang Xu, and Jason Corso. 2018. Towards automatic learning of procedures from web instructional videos. In Proc. AAAI Conference on Artificial Intelligence (AAAI)

    2018

  62. [62]

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479

    Pith/arXiv arXiv 2025