Pith. sign in

REVIEW 5 major objections 6 minor 53 references

VideoForest: Person-Anchored Hierarchical Reasoning for Cross-Video Question Answering

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that cross-video question answering can be solved by using matched person identities as the bridges that link separate camera streams, rather than by analyzing each video in isolation.

desk verdict A clever modular system and a first cross-video QA benchmark, but the empirical claims are under-powered and under-disclosed; worth review, not yet trusted. read the letter →

arxiv 2508.03039 v1 pith:PRCBSJSX submitted 2025-08-05 cs.CV cs.MM

Yiran Meng , Junhong Ye , Wei Zhou , Guanghui Yue , Xudong Mao , Ruomei Wang , Baoquan Zhao This is my paper
classification cs.CVcs.MM
keywords cross-videoquestionansweringpersonre-identificationhierarchicalvideorepresentationmulti-agentreasoningmulti-camerasurveillancespatio-temporalperson-anchoredtree
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VideoForest claims that the hard part of cross-video question answering is establishing continuity between video streams, and that person identity is the right continuity signal. The paper builds a modular pipeline that re-identifies and tracks people across cameras, organizes each video into a multi-granularity tree of segments, and has four language-model agents retrieve and combine evidence from those trees to answer queries. On a new benchmark built from office-surveillance and human-action videos, the authors report 71.93% accuracy on person recognition, 83.75% on behavior analysis, and 51.67% on summarization and reasoning, outperforming all tested single-video models on every task. The appeal of the claim is that this works without end-to-end training on multi-video data, so a camera network can be indexed and queried with existing components.

What carries the argument

The load-bearing object is a person-anchored hierarchical forest: each video is organized as a tree $\mathcal{T}=(V,E)$, whose nodes are tuples $v=(t_{\mathrm{start}},t_{\mathrm{end}},R_v,C_v,\Gamma_v)$ holding temporal bounds, person re-identification descriptors $R_v$, a semantic content vector $C_v$, and child nodes $\Gamma_v$. Because every node carries $R_v$, the same physical person is addressable across different trees, so search can follow identity instead of matching frames exhaustively. Around the forest sit an adaptive segmenter (local visual change, global deviation, and person-set change) that fixes node boundaries, and a four-agent system that filters videos, reads a confidence-weighted knowledge base, traverses trees top-down, and integrates the retrieved evidence into an answer.

What would settle it

Run VideoForest on CrossVideoQA with ground-truth person identities substituted for the tracker's predicted identities; if average accuracy stays at the reported 69.12%, person anchoring is not the active ingredient, and if it rises sharply, the current results are capped by identity errors.

Watch

Extended reading notes

Core claim

The central discovery is that a person's trajectory, once matched across camera views, acts as a natural bridge that lets a query hop from one video to another. Each node in VideoForest's tree carries person re-identification descriptors alongside visual content and timestamps, so a question like 'who visited all three buildings between 14:00 and 16:00' becomes a retrieval problem over identity-labeled segments rather than a free-form video-reasoning problem. The authors claim this person-anchored organization is what closes the gap to single-video baselines: identity continuity supplies the cross-video correspondence that monolithic video-language models lack, and the whole system runs without end-to-end training.

Load-bearing premise

The load-bearing premise is that the person identities returned by the re-identification and tracking module are correct and stable across cameras and time; if identities are merged or dropped, the tree connects the wrong people and every downstream answer inherits the error.

Editorial extensions

If this is right

  • If the claim holds, cross-video question answering can be assembled from reusable parts—a tracker, a re-identification model, a captioner, and a language-model agent loop—with no multi-video training run.
  • Person identity becomes the primary index for multi-camera queries, so who, where, and when questions reduce to trajectory retrieval over identity-labeled nodes.
  • The confidence-weighted knowledge base implies that repeated queries become faster and self-correcting, because conflicting entries are demoted or replaced rather than re-answered from scratch.
  • The benchmark results indicate that cross-spatiotemporal queries remain the hardest configuration, so future gains are likely to come from better spatial-temporal fusion rather than from retrieval alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorially, the same person-anchored tree could be applied to other persistent objects—vehicles, packages, or animals—making it a template for cross-camera indexing beyond surveillance.
  • Editorially, the cleanest test of the person-anchoring claim is to feed ground-truth identities into the tree; the paper's ablations remove re-identification from search but do not measure how identity errors themselves cap performance.
  • Editorially, a natural extension is to evaluate the pipeline on a multi-camera corpus with known identity ground truth, so re-identification accuracy and question-answering accuracy can be reported together.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces VideoForest, a modular pipeline for cross-video question answering: it extracts frame-level ViCLIP embeddings and person detections, segments videos with a three-criterion adaptive boundary detector, builds per-video hierarchical trees whose nodes carry person re-identification trajectories, and answers queries using a CrewAI-based multi-agent system with a confidence-weighted knowledge base. The authors construct a new benchmark, CrossVideoQA, from Edinburgh Office Surveillance and HACS, and report that VideoForest outperforms several video-LLM baselines, with 71.93% person recognition, 83.75% behavior analysis, and 51.67% summarization/reasoning accuracy, claiming statistically significant advantages. The central claim is that person-anchored hierarchical reasoning is an effective alternative to end-to-end trained single-stream video models for cross-video queries.

Significance. If the empirical claims held, VideoForest would be a valuable contribution: the person-anchored, training-free tree representation is a natural fit for multi-camera surveillance, and CrossVideoQA addresses a genuine gap in benchmark coverage. The paper also provides a clear formalization of the cross-video QA problem (Eq. 1) and a modular architecture that is easy to understand. However, the current evidence does not yet establish the claimed superiority: the benchmark size and scoring are undocumented, the significance claim is unsupported by any statistical test, and identity-level accuracy, which is the load-bearing premise of the whole approach, is not measured.

major comments (5)
  1. [§4.4.1 and Table 2] The claim of "statistically significant performance advantages" is not supported by any significance test, confidence interval, or error bar. The percentages in Table 2 (72.00%, 69.23%, 65.38%, 61.54%) are consistent with denominators of only 25–26 questions per modality; at n≈26, the 95% confidence interval spans roughly ±18 percentage points, so most of the reported 8–17 point gaps over the best baselines are within sampling noise. Please report exact question counts per category and modality, a scoring protocol, paired tests across models (or bootstrap intervals), and error bars for VideoForest and the baselines.
  2. [§3.2–§3.4, Eqs. (3) and (9)] The entire tree structure and cross-video retrieval are anchored on the person identities produced by the ReID/tracking model, but the paper never reports identity-level accuracy (e.g., precision/recall, id-switch rate) on CrossVideoQA or on the EOSD portion. Without this measurement, the reader cannot tell whether the final QA gains come from correct person bridging or from a tracker that silently merges or drops identities. Please add a ReID-specific evaluation and describe the tracker/ReID model versions and failure modes.
  3. [§4.1.1] The CrossVideoQA benchmark is a union of EOSD (18 videos) and HACS (50,000 independent action clips), but HACS, as released, has no cross-video person identities or spatial/temporal correspondences; the cross-spatial and cross-temporal modalities therefore appear to rest almost entirely on the 18 Edinburgh videos. The paper gives no per-source question counts, no number of distinct identities, no per-modality statistics, and no answer distributions. Please clarify exactly how queries are constructed for each modality and how many questions come from each source; otherwise the benchmark's validity and difficulty cannot be assessed.
  4. [§4.2 and §4.3] The implementation details and baseline evaluation protocol are underspecified. The paper does not state which LLM powers the multi-agent reasoning system, which visual encoder/ReID/tracker models are used, the prompt templates or decoding settings, or how many frames/clips are fed to each baseline under the "sequential processing protocol." This makes the reported numbers not reproducible and leaves open the possibility that the comparisons are confounded by unequal information access.
  5. [Tables 1–3] The reported VideoForest overall accuracy is internally inconsistent. Table 1 gives Overall Acc = 69.12%, but the four modality percentages in Table 2 average to 67.04% (and Table 3 reports a Full Model average of 67.54%). Please clarify the weighting scheme or correct the inconsistency, as the current numbers cannot both be right.
minor comments (6)
  1. [Eq. (15)] There is a typo in the third case of the update function: "if," should read "if".
  2. [Notation] The symbol S is used both for the segmentation boundary function (Eq. 4) and for the search function (Eq. 16); please rename one of them to avoid confusion.
  3. [Tables 3 and 4] Tables 3 and 4 are typeset as very long single-row paragraphs; reformatting them into proper tables would greatly improve readability.
  4. [§4.3] Several baseline names contain typos or spacing errors, e.g., "LLaV AOneVision", "LLaV A-NeXTVideo", and "VideoLLaMA3".
  5. [§4.3] The sequential processing protocol for baselines is only described in one sentence; specify the exact prompt, the number of frames or clips supplied to each model, and how answers were extracted.
  6. [§3.2] The threshold values (epsilon_1, epsilon_2, Delta_P, tau_rel, tau_conf, tau_sim) are never reported, nor is the held-out set used for cross-validation; please list the final values and describe the held-out split.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported accuracies are empirical outcomes on an independently labeled benchmark, not consequences of the method's own definitions.

full rationale

VideoForest's load-bearing equations—segmentation (Eqs. 4–7), node construction (Eq. 9), and tree search (Eq. 16)—take ReID/tracking outputs as inputs and organize them hierarchically; the claimed accuracies (71.93%, 83.75%, 51.67%) are measured against CrossVideoQA, whose questions and answers were produced by human specialists and expert review (§4.1.3) rather than by the model's own ReID or tree. Thus the person-recognition task tests the same capability the ReID module provides, but that is task alignment, not a self-definitional reduction: the ground-truth labels are not constructed from p(f) or R_v. Threshold tuning (ε1, ε2, ΔP) by cross-validation is standard parameter selection and does not rename a fitted value as a prediction. The only self-citation ([23]) is a background statement in related work and is not load-bearing. The main weaknesses are evidential, not circular: §4.4.1 asserts statistically significant without a reported test, Table 2's modality-level percentages imply small denominators, and CrossVideoQA is author-constructed without an external cross-video benchmark; these affect statistical power and generalizability but do not make any result true by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The framework introduces software components: person-anchored tree nodes, four agent types, and a global knowledge base. These are designed architectural elements implemented with existing tools such as CrewAI, not new physical entities, forces, or conserved quantities. The main unverified premises are identity reliability, benchmark answerability, and the capability of the unnamed LLM agent stack.

free parameters (6)
  • epsilon_1 (local transition threshold) = not reported
    Used in Eq. 5 for adaptive segmentation; determined by cross-validation on a held-out dataset (Section 3.2), affects all downstream tree construction and retrieval.
  • epsilon_2 (global deviation threshold) = not reported
    Used in Eq. 6 for adaptive segmentation; tuned by cross-validation, with value not reported.
  • Delta_P (person-set change threshold) = not reported
    Used in Eq. 7 to detect boundaries when the set of visible person identities changes; tuned by cross-validation, value not reported.
  • tau_rel (relevance threshold) = not reported
    Used in Eq. 16 to decide whether a tree node is relevant enough to return content; described as configurable but not specified.
  • tau_conf (knowledge base confidence threshold) = not reported
    Used in Section 3.5.2 to prioritize knowledge base entries; value not reported.
  • tau_sim (semantic conflict similarity threshold) = not reported
    Used in Eq. 15 to decide when a new knowledge base entry conflicts with an existing one; value not reported.
assumptions (5)
  • domain assumption Person identities from ReID and tracking are correct and stable across cameras and time.
    Central design premise in Sections 3.1 and 3.4; if identity labels are wrong, person-anchored tree nodes correlate the wrong people across videos.
  • domain assumption CrossVideoQA questions are answerable from the provided videos and have correct gold answers.
    The benchmark is built by LLM augmentation plus expert review (Section 4.1.3), but no inter-annotator agreement, answerability statistics, or error analysis on the question set are reported.
  • domain assumption The undisclosed LLM behind the CrewAI agents can perform reliable relevance scoring, retrieval, and synthesis.
    The reasoning pipeline in Section 3.5 delegates core decisions to LLM agents; without naming the model, the contribution of each component cannot be isolated.
  • domain assumption Adaptive segmentation thresholds tuned on a held-out split generalize to test queries.
    Section 3.2 states cross-validation on a held-out dataset, but the split, threshold values, and sensitivity are not disclosed.
  • domain assumption ViCLIP and ShareGPT4Video features are sufficient semantic representations for the query types in CrossVideoQA.
    The method reuses pretrained encoders (Sections 3.2 and 4.3) without verifying their failure modes on low-resolution surveillance footage or action clips.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VideoForest: Person-Anchored Hierarchical Reasoning for Cross-Video Question Answering." pith.science (2026). https://pith.science/paper/PRCBSJSX

@misc{pith2026250803039,
  author       = {Pith},
  title        = {Pith review of: VideoForest: Person-Anchored Hierarchical Reasoning for Cross-Video Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PRCBSJSX}},
  note         = {Machine review of arXiv:2508.03039}
}
read the original abstract

Cross-video question answering presents significant challenges beyond traditional single-video understanding, particularly in establishing meaningful connections across video streams and managing the complexity of multi-source information retrieval. We introduce VideoForest, a novel framework that addresses these challenges through person-anchored hierarchical reasoning. Our approach leverages person-level features as natural bridge points between videos, enabling effective cross-video understanding without requiring end-to-end training. VideoForest integrates three key innovations: 1) a human-anchored feature extraction mechanism that employs ReID and tracking algorithms to establish robust spatiotemporal relationships across multiple video sources; 2) a multi-granularity spanning tree structure that hierarchically organizes visual content around person-level trajectories; and 3) a multi-agent reasoning framework that efficiently traverses this hierarchical structure to answer complex cross-video queries. To evaluate our approach, we develop CrossVideoQA, a comprehensive benchmark dataset specifically designed for person-centric cross-video analysis. Experimental results demonstrate VideoForest's superior performance in cross-video reasoning tasks, achieving 71.93% accuracy in person recognition, 83.75% in behavior analysis, and 51.67% in summarization and reasoning, significantly outperforming existing methods. Our work establishes a new paradigm for cross-video understanding by unifying multiple video streams through person-level features, enabling sophisticated reasoning across distributed visual information while maintaining computational efficiency.

Figures

Figures reproduced from arXiv: 2508.03039 by the authors.

Figure 1
Figure 1. Comparison of single-video vs. cross-video question [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. VideoForest architecture for cross-video question answering. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of our distributed multi-agent framework for cross-video reasoning. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Exemplars from CrossVideoQA illustrating VideoForest’s multi-modal reasoning architecture. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 16 canonical work pages

  1. [1]

    Yizhak Ben-Shabat, Xin Yu, Fatemeh Sadat Saleh, Dylan Campbell, Cristian Rodriguez-Opazo, Hongdong Li, and Stephen Gould. 2023. The IKEA ASM VideoForest: Person-Anchored Hierarchical Reasoning for Cross-Video Question Answering MM ’25, October 27–31, 2025, Dublin, Ireland Dataset: Understanding People Assembling Furniture through Actions, Objects and Pose...

    work page Pith review arXiv 2023

  2. [2]

    Zesen Cheng-Zhiqiang Hu Yuqian Yuan Guanzheng Chen Sicong Leng Yuming Jiang Hang Zhang Xin Li Peng Jin Wenqi Zhang Fan Wang Lidong Bing Deli Zhao Boqiang Zhang, Kehan Li. 2025. VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding.arXiv preprint arXiv:2501.13106 (2025). https://arxiv.org/abs/2501.13106

    arXiv 2025

  3. [3]

    Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zisserman. 2022. A Short Note on the Kinetics-700 Human Action Dataset. arXiv:1907.06987 [cs.CV] https://arxiv.org/abs/1907.06987

    arXiv 2022

  4. [4]

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, Li Yuan, Yu Qiao, Dahua Lin, Feng Zhao, and Jiaqi Wang. 2024. ShareGPT4Video: Improving Video Understanding and Generation with Better Captions.arXiv preprint arXiv:2406.04325(2024)

    arXiv 2024

  5. [5]

    Dingxin Cheng, Mingda Li, Jingyu Liu, Yongxin Guo, Bin Jiang, Qingbin Liu, Xi Chen, and Bo Zhao. 2024. Enhancing Long Video Understanding via Hierarchical Event-Based Memory. arXiv:2409.06299 [cs.CV] https://arxiv.org/abs/2409.06299

    arXiv 2024

  6. [6]

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. 2024. VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs.arXiv preprint arXiv:2406.07476(2024). https://arxiv.org/abs/ 2406.07476

    arXiv 2024

  7. [7]

    Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. 2025. Videoagent: A memory-augmented multimodal agent for video understanding. InEuropean Conference on Computer Vision. Springer, 75–92

    work page 2025

  8. [8]

    Hao Fei, Shengqiong Wu, Meishan Zhang, Min Zhang, Tat-Seng Chua, and Shuicheng Yan. 2024. Enhancing Video-Language Representations With Struc- tural Spatio-Temporal Alignment.IEEE Transactions on Pattern Analysis and Machine Intelligence46, 12 (2024), 7701–7719. doi:10.1109/TPAMI.2024.3393452

    work page arXiv 2024

Show all 53 references
  1. [9]

    Jiajun Fei, Dian Li, Zhidong Deng, Zekun Wang, Gang Liu, and Hui Wang. 2024. Video-CCAM: Enhancing Video-Language Understanding with Causal Cross- Attention Masks for Short and Long Videos. arXiv:2408.14023 [cs.CV] https: //arxiv.org/abs/2408.14023

    2024 arXiv

  2. [10]

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2024. Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis.arXiv preprint arXiv:2405.21075(2024)

    2024 arXiv

  3. [11]

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles

  4. [12]

    Md Mohaiminul Islam, Ngan Ho, Xitong Yang, Tushar Nagarajan, Lorenzo Torre- sani, and Gedas Bertasius. 2024. Video ReCap: Recursive Captioning of Hour-Long Videos.arXiv preprint arXiv:2402.13250(2024)

    2024 arXiv

  5. [13]

    Md Mohaiminul Islam, Tushar Nagarajan, Huiyu Wang, Gedas Bertasius, and Lorenzo Torresani. 2025. BIMBA: Selective-Scan Compression for Long-Range Video Question Answering.arXiv preprint arXiv:2503.09590(2025)

    2025 arXiv

  6. [14]

    Cognition Labs. 2024. CrewAI: Framework for Building Multi-Agent Systems. https://github.com/joaomdmoura/crewai. Accessed: 2025-04

    2024

  7. [15]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-OneVision: Easy Visual Task Transfer.arXiv preprint arXiv:2408.03326(2024)

    2024 arXiv

  8. [16]

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. 2024. Mvbench: A comprehensive multi-modal video understanding benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 22195–22206

    2024

  9. [17]

    Ruizhe Li, Jiahao Guo, Mingxi Li, Zhengqian Wu, and Chao Liang. 2023. A Hier- archical Deep Video Understanding Method with Shot-Based Instance Search and Large Language Model. InProceedings of the 31st ACM International Confer- ence on Multimedia(Ottawa ON, Canada)(MM ’23). A...

    2023

  10. [18]

    Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al . 2024. VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling.arXiv preprint arXiv:2501.00574(2024)

    2024 arXiv

  11. [19]

    Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, and Min Zhang

  12. [20]

    Huabin Liu, Filip Ilievski, and Cees G. M. Snoek. 2025. Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoning. arXiv:2501.05069 [cs.CV] https://arxiv.org/abs/2501.05069

    2025 arXiv

  13. [21]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. LLaVA-NeXT: Improved reasoning, OCR, and world knowl- edge. https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  14. [22]

    Huabin Liu, Xiao Ma, Cheng Zhong, Yang Zhang, and Weiyao Lin. 2024. Time- Craft: Navigate Weakly-Supervised Temporal Grounded Video Question Answer- ing via Bi-directional Reasoning. InComputer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4,...

    2024 doi

  15. [23]

    Mingyang Liu, Fan Zhou, Ruomei Wang, Baoquan Zhao, and Fuwei Zhang. 2025. Semantic Distance-Aware Cross-Modal Attention Mechanism for Video Question Answering.IEEE Transactions on Instrumentation and Measurement(2025)

    2025

  16. [24]

    Yi Liu, Limin Wang, Yali Wang, Xiao Ma, and Yu Qiao. 2022. Fine- Action: A Fine-Grained Video Dataset for Temporal Action Localization. arXiv:2105.11107 [cs.CV] https://arxiv.org/abs/2105.11107

    2022 arXiv

  17. [25]

    Ziyu Ma, Chenhui Gou, Hengcan Shi, Bin Sun, Shutao Li, Hamid Rezatofighi, and Jianfei Cai. 2024. DrVideo: Document Retrieval Based Long Video Understanding. arXiv:2406.12846 [cs.CV] https://arxiv.org/abs/2406.12846

    2024 arXiv

  18. [26]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan

  19. [27]

    Rawat, and Thomas B

    Neelu Madan, Andreas Moegelmose, Rajat Modi, Yogesh S. Rawat, and Thomas B. Moeslund. 2024. Foundation Models for Video Understanding: A Survey. arXiv:2405.03770 [cs.CV] https://arxiv.org/abs/2405.03770

    2024 arXiv

  20. [28]

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. 2023. Egoschema: A diagnostic benchmark for very long-form video language un- derstanding.Advances in Neural Information Processing Systems36 (2023), 46212– 46244

    2023

  21. [29]

    InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    2024

  22. [30]

    Jong Sung Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryoo, Donghyun Kim, and Michael S. Ryoo. 2024. Too Many Frames, not all Useful: Efficient Strategies for Long-Form Video QA.ArXivabs/2406.09396 (2024). https: //api.semanticscholar.org/CorpusID:270440923

    2024

  23. [31]

    Qasim, R

    T. Qasim, R. B. Fisher, and N. Bhatti. 2021. Ground-truthing Large Human Behavior Monitoring Datasets. InProceedings of the 2020 International Conference on Pattern Recognition. Online

    2021

  24. [32]

    Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, and Li Yuan. 2023. Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models.arXiv preprint arXiv:2311.16103 (2023)

    2023 arXiv

  25. [33]

    Kate Sanders, Nathaniel Weir, and Benjamin Van Durme. 2024. TV- TREES: Multimodal Entailment Trees for Neuro-Symbolic Video Reasoning. arXiv:2402.19467 [cs.CL] https://arxiv.org/abs/2402.19467

    2024 arXiv

  26. [34]

    Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta

    Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. 2016. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. arXiv:1604.01753 [cs.CV] https://arxiv.org/abs/1604. 01753

    2016 arXiv

  27. [35]

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. 2024. TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 14313–14323. doi:10.1109/CVPR52733.2024.01357

    2024

  28. [36]

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. 2023. InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation. InThe Twelfth International Conference on Learning Representations

    2023

  29. [37]

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, Sen Xing, Guo Chen, Junting Pan, Jiashuo Yu, Yali Wang, Limin Wang, and Yu Qiao. 2022. InternVideo: General Video Foundation Models via Generative and Discriminativ...

    2022 arXiv

  30. [38]

    Junke Wang, Dongdong Chen, Chong Luo, Xiyang Dai, Lu Yuan, Zuxuan Wu, and Yu-Gang Jiang. 2023. Chatvideo: A tracklet-centric multimodal and versatile video understanding system.arXiv preprint arXiv:2304.14407(2023)

    2023 arXiv

  31. [39]

    Yingquan Wang, Pingping Zhang, Shang Gao, Xia Geng, Hu Lu, and Dong Wang. 2021. Pyramid Spatial-Temporal Aggregation for Video-Based Person Re-Identification. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 12026–12035

    2021

  32. [40]

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. 2024. LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. arXiv:2407.15754 [cs.CV] https://arxiv.org/abs/2407.15754

    2024 arXiv

  33. [41]

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kai Chen, Wenhai Wang, Yu Qiao, Yali Wang, and Limin Wang. 2025. InternVideo2.5: Empow- ering Video MLLMs with Long and Rich Context Modeling.arXi...

    2025 arXiv

  34. [42]

    Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. 2024. LLaVA-Critic: Learning to Evaluate Multi- modal Models. (2024). arXiv:2410.02712 [cs.CV] https://arxiv.org/abs/2410.02712

    2024 arXiv

  35. [43]

    Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2024. mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models. arXiv:2408.04840 [cs.CV] MM ’25, October 27–31, 2025, Dublin, Ireland Yiran M...

    2024 arXiv

  36. [44]

    Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, and Tat-Seng Chua. 2022. Video as conditional graph hierarchy for multi-granular question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 2804–2812

    2022

  37. [45]

    Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. 2024. A Simple LLM Framework for Long-Range Video Question-Answering. arXiv:2312.17235 [cs.CV] https://arxiv.org/abs/2312. 17235

    2024 arXiv

  38. [46]

    Hang Zhang, Xin Li, and Lidong Bing. 2023. Video-LLaMA: An Instruction- tuned Audio-Visual Language Model for Video Understanding.arXiv preprint arXiv:2306.02858(2023). https://arxiv.org/abs/2306.02858

    2023 arXiv

  39. [47]

    Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. 2023. Self- Chained Image-Language Model for Video Localization and Question Answering. arXiv:2305.06988 [cs.CV] https://arxiv.org/abs/2305.06988

    2023 arXiv

  40. [48]

    Hang Zhao, Zhicheng Yan, Lorenzo Torresani, and Antonio Torralba. 2019. HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Local- ization.arXiv preprint arXiv:1712.09374(2019)

    2019 arXiv

  41. [49]

    Qianyu Zhou, Xiangtai Li, Lu He, Yibo Yang, Guangliang Cheng, Yunhai Tong, Lizhuang Ma, and Dacheng Tao. 2023. TransVOD: End-to-End Video Object Detection With Spatial-Temporal Transformers.IEEE Transactions on Pattern Analysis and Machine Intelligence45, 6 (June 2023), 7853–7...

    2023

  42. [50]

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-NeXT: A Strong Zero-shot Video Understanding Model. https://llava-vl.github.io/blog/2024-04-30-llava-next- video/

    2024

  43. [53]

    Elias Stengel-Eskin Jaehong Yoon Feng Cheng Gedas Bertasius Mohit Bansal Ziyang Wang, Shoubin Yu. 2024. VideoTree: Adaptive Tree-based Video Repre- sentation for LLM Reasoning on Long Videos.arxiv(2024)

    2024

  44. [2015]

    https://api.semanticscholar.org/CorpusID:1710722

    ActivityNet: A large-scale video benchmark for human activity understand- ing.2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 961–970. https://api.semanticscholar.org/CorpusID:1710722

    2015

  45. [2024]

    https://api.semanticscholar.org/CorpusID: 270559556

    VideoVista: A Versatile Benchmark for Video Understanding and Reason- ing.ArXivabs/2406.11303 (2024). https://api.semanticscholar.org/CorpusID: 270559556

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.