Pith.

Computer Vision and Pattern Recognition · arXiv:2609.21386 v1 · submitted 18 Sep 2026 · 69 references

AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents

Seoyeon An, Hyeonseo Jang, Minsu Kim, Chanho Lee, Younghan Park, Kangwook Lee

The paper says

“Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world.”

more of the abstract

While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-level queries or global summaries that require only single-step inference. Real-world video understanding involves more challenging tasks that require multi-hop multimodal reasoning, and there is a critical absence of video benchmarks equipped to rigorously evaluate these agentic capabilities. To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. Beyond standard question-answer pairs, AgentVidBench provides step-by-step solution traces to support trajectory evaluation that assesses whether agents explicitly acquire the evidence needed to justify their answers. Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores. We further present a simple yet effective agentic strategy that serves as a competitive baseline on AgentVidBench, establishing our benchmark as a holistic testbed for future research on agentic video understanding. Code and datasets are available at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 21 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

A solid multi-hop video QA benchmark with a novel trajectory evaluation; the milestone-circularity concern is real but not fatal, and the paper handles it more responsibly than most. This line is wrong

the referee pass did not run

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 69 references say.

What would settle it

Recruit three independent annotators not involved in the original curation. For each of the 100 questions, provide only the video, the question, and the answer options (no solution note). Ask each annotator to independently derive a set of evidence milestones following the same tool-agnostic guidelines from Appendix C.3. Compute pairwise Fleiss' κ on milestone existence per question, and measure the overlap between the original author-derived milestones and the independent annotator consensus. If milestone-level agreement κ < 0.7 on a substantial fraction of questions (e.g., >20%), the reference milestones reflect path-specific rather than evidence-required information, weakening the trajectory evaluation.

from the skeptic's reading

How the review went

3 readings by deepseek-v4-flash · 21 Sep 2026

  1. Reader holds up confidence high
  2. Skeptic agrees: says it holds up only with conditions 38 s · 42,877 tokens read, 3,807 written Milestones were extracted by the same annotators who authored the questions (Appendix C.2–C.3), creating a potential circularity: the reference evidence set may reflect one specific human solution path rather than a complete and impartial evidence checklist needed to evaluate alternative valid…
  3. Editor wrote the verdict above 15 s · 43,746 tokens read, 1,222 written

Not run on this paper: the referee, the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper does what a good benchmark paper should do: it identifies a real gap—multi-hop video questions that require active, tool-mediated reasoning—and builds a careful evaluation around it. The construction is thorough: three-annotator consensus (Fleiss' κ 0.826), 25 distractors per question, and a validated LLM judge for trajectory scoring. The judge validation is the strongest part: they test separation of correct/incorrect runs, sensitivity to different judge models and prompts, adversarial synthetic trajectories, and human–judge alignment (r=0.77). That is more validation than most process-level evaluation papers provide.

The stress-test concern—milestones derived by the same authors who wrote the questions—is legitimate, and I'm glad you flagged it. But the paper does take it seriously. Appendix C.3 shows that replacing the original human reasoning notes with abstracted, tool-agnostic milestones actually reduces path-specific cues (99% of milestones are tool-agnostic vs. 15% of original explanations). The judge uses milestones, not the raw human solution. The adversarial tests also show that removing evidence for one milestone drops the score sharply, suggesting the judge is responding to specific evidence requirements rather than narrative fluency. Could an alternative valid path that reaches the correct answer via different evidence be penalized? Possibly. But the benchmark is small (100 questions), the authors acknowledge this as a limitation, and the validation provides reasonable evidence that the judge tracks evidence coverage rather than path matching.

The softest spot is the scale: 100 questions, 71 videos. The authors are honest about this and provide bootstrap CIs and McNemar tests to show statistical reliability. That helps, but community adoption will ultimately depend on whether the benchmark can scale. The other soft spot is that their own Video-ReAct agent uses Gemini 2.5 Pro as the perception tool, which is the same backend they use for judging, raising a potential coupling question (though they ablate this in Appendix H.2 by aligning tools across frameworks).

Bottom line: this is a serious, well-executed contribution. The theoretical motivation (fixed-depth transformer limitation) is borrowed from prior work but applied sensibly. The evaluation methodology is transparent. I would send this to peer review; it will likely get a fair hearing and a revise-and-resubmit that asks for either scaling data or a clearer plan for it. I'd cite it if I were working on agentic video understanding.

The strongest objection

The trajectory evaluation is the central methodological novelty of AgentVidBench, but it rests entirely on the completeness and impartiality of the tool-agnostic evidence milestones (§3.4, Appendix C.3). A risk arises because the same five authors performed both question creation (including drafting the ground-truth reasoning note) and milestone extraction from those notes. There was no independent annotation pass that derived milestones from the video and question alone, blinded to the original solution path. While the paper validates the LLM judge via human–judge correlation (r=0.77, Appendix D.4), the human raters in that study were also authors and used the same milestones — they could share the same blind spots. Without independent milestone derivation, it is unclear whether the benchmark penalizes trajectories that reach the correct answer via a different but equally valid set of evidence steps. The paper's adversarial tests (Appendix D.4) confirm that removing evidence for one milestone drops the score, but they do not test whether an alternative valid reasoning path that omits a given milestone would be unfairly penalized. If milestones encode one specific human path rather than the requisite evidence in full, the trajectory scores may systematically undervalue agent behaviors that are sound but differ in evidence-gathering strategy.

The reader highlighted 'the validity of the human-curated evidence milestones as accurate and sufficient proxies for the reasoning steps' as the weakest assumption. I concur, and I identify a specific mechanism — the lack of independence between question authors and milestone extractors — that makes this assumption load-bearing. The paper's validation (judge sensitivity, adversarial tests, human alignment) is thorough but does not break the circularity: the human raters in Appendix D.4 were also authors who saw the same milestones. The validation confirms that the judge applies the milestones consistently, but does not confirm that the milestones themselves are complete and unbiased. Given that the trajectory evaluation is a core contribution claimed in the paper (contribution 3), this concern warrants a CONDITIONAL verdict: the benchmark should be accepted contingent on an independent milestone validation study. If that validation holds, ACCEPT is justified.

Pith's reading of the paper (a machine's summary, not the authors')

The paper introduces AgentVidBench, a benchmark of 100 multi-hop video question-answer pairs that require spatial, temporal, and causal reasoning across multiple video segments. Each question is paired with human-curated evidence milestones, enabling trajectory evaluation that checks whether the agent acquired the necessary evidence, not just the final answer. Experiments with 12 state-of-the-art MLLMs show that even the best single-pass model (Gemini 2.5 Pro) reaches only 51% accuracy, while integrating the model into agentic workflows that iteratively inspect the video raises accuracy to 58% and substantially improves reasoning trajectory quality. The benchmark is designed specifically to evaluate a gentic video understanding, a capability that existing benchmarks almost entirely neglect because they test only single-step inference.

What carries the argument. The central machinery of AgentVidBench is its combination of multi-hop questions that demand spatial, temporal, and causal reasoning, along with tool-agnostic evidence milestones that enable fair trajectory evaluation across diverse a gentic framworks. Each question requires iterative video inspection—zooming into specific frames, adjusting playback speed, and seeking across time—rather than processing the video in a single pass. The evaluation pipeline scores agent trajectories on five dimensions: task understanding, evidence coverage, evidence grounding, evidence completeness, and reasoning faithfulness.

Core claim. A gentVidBench is a multi-hop video question answering benchmark that requires active, iterative video exploration rather than single-pass inference. Its central discovery is that conventional single-turn MLMs are fundamenally limited on compositional video tasks, with the best model achieving 51% accuracy, while a gentic workflows like Video-ReAct improve accuracy to 58% and produce evidence-grounded reasoning. The benchmark further introduces a milestone-based trajectory evaluation that measures whether a gents explicitly acquire the evidence needed to justify their answers, going beyond final-answer accuracy alone.

The premise it rests on. The load-bearing premise of the benchmark is that the human-curated evidence milestones accurately and sufficiently capture the reasoning steps needed to answer each question; if milestones are incomplete, ambiguous, or contain path-specific cues despite the tool-agnostic design, the trajectory evaluation could reward or penalize agents for the wrong reasons.

If the paper is right (Pith's inferences, not the authors')

Future video QA benchmarks should incorporate multi-hop reasoning requirements to better capture real-world understanding needs.

The design of a gentic framework matters: the paper shows that flexible, task-adaptive planning (e.g., Video-ReAct) outperform more rigid pipelines like STAR and DVD.

The milestone-based trajectory evaluation provides a template for assessing reasoning quality in multimodal tasks beyond simple accuracy.

Even with improved accuracy (58%), significant residual errors remain in counting, verification, and exclusion checking, pointing to concrete bottlenecks for future research.

The benchmark demonstrates that a gentic workflows can substantially reduce token usage compared to naive single-pass processing by focusing computation on relevant video segments.

The 100-question size, while small, is statistically justified with bootstrap confidence intervals and McNear's tests; scaling the benchmark via automated data collection could further strengthen its conclusions.

The trajectory evaluation's reliance on an LLM judge and human-curated milestones means that any systematic bias in the milestones could distort scores, but the paper's adversarial validation partially mitigates this.

The Video-ReAct baseline uses Gemini 2.5 Pro as a visual tool, which suggests that much of the gain may come from the underlying vision capability rather than the planning alone, though the ablation study controls for some of this.

Extending AgentVidBench to open-ended questions (not multiple-choice) would test whether models can generate free-form reasoning, not just select among options.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-level queries or global summaries that require only single-step inference. Real-world video understanding involves more challenging tasks that require multi-hop multimodal reasoning, and there is a critical absence of video benchmarks equipped to rigorously evaluate these agentic capabilities. To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. Beyond standard question-answer pairs, AgentVidBench provides step-by-step solution traces to support trajectory evaluation that assesses whether agents explicitly acquire the evidence needed to justify their answers. Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores. We further present a simple yet effective agentic strategy that serves as a competitive baseline on AgentVidBench, establishing our benchmark as a holistic testbed for future research on agentic video understanding. Code and datasets are available at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 0.81

“Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.20 · states limitation 0.39 · human participants 0.05 · controlled comparison 0.65 · randomized assignment 0.01 · states quantitative result 0.80 · compares with prior results 0.87 · mentions code or data availability 0.99

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world. background 1.00 0.04 0.02 asserted 1.00
While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-level queries or global summaries that require only single-step inference. gap 0.59 0.07 0.02 asserted 1.00
Real-world video understanding involves more challenging tasks that require multi-hop multimodal reasoning, and there is a critical absence of video benchmarks equipped to rigorously evaluate these agentic capabilities. gap 1.00 0.07 0.05 asserted 1.00
To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. contribution 1.00 0.90 0.08 asserted 0.99
Beyond standard question-answer pairs, AgentVidBench provides step-by-step solution traces to support trajectory evaluation that assesses whether agents explicitly acquire the evidence needed to justify their answers. contribution 0.68 0.90 0.05 asserted 1.00
Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores. result 1.00 0.96 0.42 asserted 1.00
We further present a simple yet effective agentic strategy that serves as a competitive baseline on AgentVidBench, establishing our benchmark as a holistic testbed for future research on agentic video understanding. contribution 0.90 0.95 0.27 asserted 1.00
Code and datasets are available at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench. contribution 0.76 0.54 0.02 asserted 0.98

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,440 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (8)
Figure 1 of the paper
Figure 1. An illustrative example from AgentVidBench, requiring spatial, temporal, and causal
Figure 2 of the paper
Figure 2. Overview of AgentVidBench. (a) Benchmark construction (Section
Figure 3 of the paper
Figure 3. AgentVidBench at a glance. Panels (a) and (b) summarize video-level statistics, including
Figure 4 of the paper
Figure 4. Trajectory score profiles of vanilla MLLMs and agentic frameworks. Each radar plot shows
Figure 5 of the paper
Figure 5. Qualitative example of Video-ReAct (Ours). Round 1 spends a dense budget on a narrow window (2:05–2:27, 5 FPS, medium) to read the seven station names off the route map; Round 2 inverts the setting (2:27–end, 1 FPS, low) to survey every station visit at once, grounding water-supply artifacts at three stations and clearing the other four. 28
Figure 6 of the paper
Figure 6. Trajectory score profiles of proprietary MLLMs under Single-Turn and agentic frameworks
Figure 7 of the paper
Figure 7. Trajectory score profiles of Qwen-family open-source MLLMs under Single-Turn and
Figure 8 of the paper
Figure 8. Trajectory score profiles of Gemma-family open-source MLLMs under Single-Turn and

Figures are the authors', reproduced from the paper.

References (69)
  1. [1] Anthropic. Introducing claude opus 4.7. https://www.anthropic.com/news/ claude-opus-4-7, 2026 on Pith
  2. [2] Anthropic. Introducing claude opus 4.8. https://www.anthropic.com/news/ claude-opus-4-8, 2026 on Pith
  3. [3] J. Carreira and A. Zisserman. Quo vadis, action recognition? A new model and the kinetics dataset. InCVPR, pages 4724–4733. IEEE Computer Society, 2017 on Pith
  4. [4] B. Chen, Z. Yue, S. Chen, Z. Wang, Y . Liu, P. Li, and Y . Wang. Lvagent: Long video understanding by multi-round dynamical collaboration of MLLM agents. InICCV, pages 20237–20246. IEEE, 2025 on Pith
  5. [5] Y . Chen, Y . Ge, Y . Ge, M. Ding, B. Li, R. Wang, R. Xu, Y . Shan, and X. Liu. Egoplan-bench: Benchmarking multimodal large language models for human-level planning.Int. J. Comput. Vis., 134(3):118, 2026 on Pith
  6. [6] J. Dang, H. Song, J. Xiao, B. Wang, H. Peng, H. Li, X. Yang, M. Wang, and T. Chua. MUPA: towards multi-path agentic reasoning for grounded video question answering.arXiv preprint arXiv:2506.18071, 2025 on PithPith's reviewarXiv
  7. [7] N. Dziri, X. Lu, M. Sclar, X. L. Li, L. Jiang, B. Y . Lin, S. Welleck, P. West, C. Bhagavatula, R. L. Bras, J. D. Hwang, S. Sanyal, X. Ren, A. Ettinger, Z. Harchaoui, and Y . Choi. Faith and fate: Limits of transformers on compositionality. InNeurIPS, 2023 on Pith
  8. [8] S. Fan, J. Cui, M.-H. Guo, and S. Yang. Tool-augmented spatiotemporal reasoning for stream- lining video question answering task. InNeurIPS, 2025 on Pith
  9. [9] S. Fan, M. Guo, and S. Yang. Agentic keyframe search for video question answering.arXiv preprint arXiv:2503.16032, 2025 on PithPith's reviewarXiv
  10. [10] Y . Fan, X. Ma, R. Wu, Y . Du, J. Li, Z. Gao, and Q. Li. Videoagent: A memory-augmented multimodal agent for video understanding. InECCV (22), Lecture Notes in Computer Science, pages 75–92. Springer, 2024 on Pith
  11. [11] C. Fu, Y . Dai, Y . Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y . Shen, M. Zhang, P. Chen, Y . Li, S. Lin, S. Zhao, K. Li, T. Xu, X. Zheng, E. Chen, C. Shan, R. He, and X. Sun. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video anal... on Pith
  12. [12] C. Fu, H. Yuan, Y . Dong, Y . Zhang, Y . Shen, X. Hu, X. Li, J. Su, C. Long, X. Xie, Y . Xie, X. Zheng, X. Yang, H. Cao, Y . Wu, Z. Liu, X. Sun, C. Shan, and R. He. Video-mme-v2: Towards the next stage in benchmarks for comprehensive video understanding.arXiv preprint arXiv:26... on PithPith's reviewarXiv
  13. [13] Gemini Team. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025 on PithPith's reviewarXiv
  14. [14] R. Girdhar, A. El-Nouby, Z. Liu, M. Singh, K. V . Alwala, A. Joulin, and I. Misra. Imagebind one embedding space to bind them all. InCVPR, pages 15180–15190. IEEE, 2023 on Pith
  15. [15] Google DeepMind. Gemini 3.1 pro: A smarter model for your most com- plex tasks. https://blog.google/innovation-and-ai/models-and-research/ gemini-models/gemini-3-1-pro/, 2026 on Pith
  16. [16] Google DeepMind. Gemma 4: Byte for byte, the most capable open models. https://blog. google/innovation-and-ai/technology/developers-tools/gemma-4/, 2026 on Pith
  17. [17] D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, X. Zhang, X. Yu, Y . Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F. Lin, F. Dai, F. L... on Pith
  18. [18] Y . Jang, Y . Song, Y . Yu, Y . Kim, and G. Kim. TGIF-QA: toward spatio-temporal reasoning in visual question answering. InCVPR, pages 1359–1367. IEEE Computer Society, 2017 on Pith
  19. [19] K. Khurana and U. Deshpande. Video question-answering techniques, benchmark datasets and evaluation metrics leveraging video captioning: A comprehensive survey.IEEE Access, 9:43799–43823, 2021 on Pith
  20. [20] H. Kim, Y . Lee, G. Lee, C. Finn, and K. Lee. Whale: A simple recipe for joint harness-weight optimization.arXiv preprint arXiv:2609.00196, 2026 on PithPith's reviewarXiv
show 49 more
  1. [21] J. Y . Koh, R. Lo, L. Jang, V . Duvvur, M. C. Lim, P. Huang, G. Neubig, S. Zhou, R. Salakhutdinov, and D. Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. In ACL (1), pages 881–905. Association for Computational Linguistics, 2024 on Pith
  2. [22] N. Kugo, X. Li, Z. Li, A. Gupta, A. Khatua, N. Jain, C. Patel, Y . Kyuragi, Y . Ishii, M. Tanabiki, K. Kozuka, and E. Adeli. Videomultiagents: A multi-agent framework for video question answering.arXiv preprint arXiv:2504.20091, 2025 on PithPith's reviewarXiv
  3. [23] Y . Lee, R. Nair, Q. Zhang, K. Lee, O. Khattab, and C. Finn. Meta-harness: End-to-end optimization of model harnesses.arXiv preprint arXiv:2603.28052, 2026 on PithPith's reviewarXiv
  4. [24] J. Lei, L. Yu, M. Bansal, and T. L. Berg. TVQA: localized, compositional video question answering. InEMNLP, pages 1369–1379. Association for Computational Linguistics, 2018 on Pith
  5. [25] K. Li, Y . Wang, Y . He, Y . Li, Y . Wang, Y . Liu, Z. Wang, J. Xu, G. Chen, P. Lou, L. Wang, and Y . Qiao. Mvbench: A comprehensive multi-modal video understanding benchmark. InCVPR, pages 22195–22206. IEEE, 2024 on Pith
  6. [26] K. Mangalam, R. Akshulakov, and J. Malik. Egoschema: A diagnostic benchmark for very long-form video language understanding. InNeurIPS, 2023. 11 on Pith
  7. [27] T. Montes and F. Lozano. Viqagent: Zero-shot video question answering via agent with open-vocabulary grounding validation. InVISAPP (2), pages 79–90. SCITEPRESS, 2026 on Pith
  8. [28] J. Mun, P. H. Seo, I. Jung, and B. Han. Marioqa: Answering questions by watching gameplay videos. InICCV, pages 2886–2894. IEEE Computer Society, 2017 on Pith
  9. [29] A. Nagrani, S. Menon, A. Iscen, S. Buch, R. Mehran, N. Jha, A. Hauth, Y . Zhu, C. V ondrick, M. Sirotenko, C. Schmid, and T. Weyand. Minerva: Evaluating complex video reasoning. In ICCV, pages 23968–23978. IEEE, 2025 on Pith
  10. [30] NVIDIA. World simulation with video foundation models for physical AI.arXiv preprint arXiv:2511.00062, 2025 on PithPith's reviewarXiv
  11. [31] OpenAI. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024 on PithPith's reviewarXiv
  12. [32] OpenAI. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024 on PithPith's reviewarXiv
  13. [33] OpenAI. Openai GPT-5 system card.arXiv preprint arXiv:2601.03267, 2026 on PithPith's reviewarXiv
  14. [34] Qwen Team. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025 on PithPith's reviewarXiv
  15. [35] Qwen Team. Qwen3.5: Towards native multimodal agents, 2026 on Pith
  16. [36] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. InICML, volume 202, pages 28492–28518. PMLR, 2023 on Pith
  17. [37] M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. P. Lillicrap, J. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, I. Antonoglou, R. Anil, S. Borgeaud, A. M. Dai, K. Millican, E. Dyer, M. Glaese, T. Sottiaux, B. Lee, F. Viola, M. Reynolds, Y . Xu, J. Molloy, J... on PithPith's reviewarXiv
  18. [38] Z. Ren, Y . Wei, X. Guo, Y . Zhao, B. Kang, J. Feng, and X. Jin. Videoworld: Exploring knowledge learning from unlabeled videos. InCVPR, pages 29029–29039. Computer Vision Foundation / IEEE, 2025 on Pith
  19. [39] X. Shang, Y . Li, J. Xiao, W. Ji, and T. Chua. Video visual relation detection via iterative inference. InACM Multimedia, pages 3654–3663. ACM, 2021 on Pith
  20. [40] X. Shen, W. Zhang, J. Chen, and M. Elhoseiny. Vgent: Graph-based retrieval-reasoning- augmented generation for long video understanding. InNeurIPS, 2025 on Pith
  21. [41] Y . Shen, K. Song, X. Tan, D. Li, W. Lu, and Y . Zhuang. Hugginggpt: Solving AI tasks with chatgpt and its friends in hugging face. InNeurIPS, 2023 on Pith
  22. [42] G. Sun, L. Liang, T. Li, B. Yu, M. Wu, and B. Zhang. Video question answering: a survey of models and datasets.Mob. Networks Appl., 26(5):1904–1937, 2021 on Pith
  23. [43] Y . Tang, J. Bi, S. Xu, L. Song, S. Liang, T. Wang, D. Zhang, J. An, J. Lin, R. Zhu, A. V osoughi, C. Huang, Z. Zhang, P. Liu, M. Feng, F. Zheng, J. Zhang, P. Luo, J. Luo, and C. Xu. Video understanding with large language models: A survey.IEEE Trans. Circuits Syst. Video Tech... on Pith
  24. [44] M. Tapaswi, Y . Zhu, R. Stiefelhagen, A. Torralba, R. Urtasun, and S. Fidler. Movieqa: Un- derstanding stories in movies through question-answering. InCVPR, pages 4631–4640. IEEE Computer Society, 2016 on Pith
  25. [45] X. Wang, R. B. Girshick, A. Gupta, and K. He. Non-local neural networks. InCVPR, pages 7794–7803. Computer Vision Foundation / IEEE Computer Society, 2018. 12 on Pith
  26. [46] X. Wang, Y . Zhang, O. Zohar, and S. Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. InECCV (80), Lecture Notes in Computer Science, pages 58–76. Springer, 2024 on Pith
  27. [47] Z. Wang, S. Yu, E. Stengel-Eskin, J. Yoon, F. Cheng, G. Bertasius, and M. Bansal. Videotree: Adaptive tree-based video representation for LLM reasoning on long videos. InCVPR, pages 3272–3283. Computer Vision Foundation / IEEE, 2025 on Pith
  28. [48] Z. Wang, H. Zhou, S. Wang, J. Li, C. Xiong, S. Savarese, M. Bansal, M. S. Ryoo, and J. C. Niebles. Active video perception: Iterative evidence seeking for agentic long video understanding.arXiv preprint arXiv:2512.05774, 2025 on PithPith's reviewarXiv
  29. [49] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models. InNeurIPS, 2022 on Pith
  30. [50] H. Wu, D. Li, B. Chen, and J. Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. InNeurIPS, 2024 on Pith
  31. [51] J. Xiao, N. Huang, H. Qin, D. Li, Y . Li, F. Zhu, Z. Tao, J. Yu, L. Lin, T. Chua, and A. Yao. Videoqa in the era of llms: An empirical study.Int. J. Comput. Vis., 133(7):3970–3993, 2025 on Pith
  32. [52] J. Xiao, X. Shang, A. Yao, and T. Chua. Next-qa: Next phase of question-answering to explaining temporal actions. InCVPR, pages 9777–9786. Computer Vision Foundation / IEEE, 2021 on Pith
  33. [53] J. Xiao, A. Yao, Z. Liu, Y . Li, W. Ji, and T. Chua. Video as conditional graph hierarchy for multi-granular question answering. InAAAI, pages 2804–2812. AAAI Press, 2022 on Pith
  34. [54] T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y . Liu, Y . Xu, S. Zhou, S. Savarese, C. Xiong, V . Zhong, and T. Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. InNeurIPS, 2024 on Pith
  35. [55] D. Xu, Z. Zhao, J. Xiao, F. Wu, H. Zhang, X. He, and Y . Zhuang. Video question answering via gradually refined attention over appearance and motion. InACM Multimedia, pages 1645–1653. ACM, 2017 on Pith
  36. [56] X. Yang, F. Feng, W. Ji, M. Wang, and T. Chua. Deconfounded video moment retrieval with causal intervention. InSIGIR, pages 1–10. ACM, 2021 on Pith
  37. [57] Y . Yang, Z. Zhang, X. Zhang, Y . Zeng, H. Li, and W. Zuo. Physworld: From real videos to world models of deformable objects via physics-aware demonstration synthesis.arXiv preprint arXiv:2510.21447, 2025 on PitharXiv
  38. [58] Z. Yang, G. Chen, X. Li, W. Wang, and Y . Yang. Doraemongpt: Toward understanding dynamic scenes with large language models (exemplified as A video agent). InICML, Proceedings of Machine Learning Research, pages 55976–55997. PMLR / OpenReview.net, 2024 on Pith
  39. [59] Z. Yang, L. Li, J. Wang, K. Lin, E. Azarnasab, F. Ahmed, Z. Liu, C. Liu, M. Zeng, and L. Wang. MM-REACT: prompting chatgpt for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023 on PithPith's reviewarXiv
  40. [60] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y . Cao. React: Synergizing reasoning and acting in language models. InICLR. OpenReview.net, 2023 on Pith
  41. [61] Y . Yao, Y . Du, D. Zhu, M. Hahn, and A. Koller. Language models can learn implicit multi-hop reasoning, but only if they have lots of training data. InEMNLP, pages 9684–9702. Association for Computational Linguistics, 2025 on Pith
  42. [62] G. Yehudai, N. Amsel, and J. Bruna. Compositional reasoning with transformers, rnns, and chain of thought. InNeurIPS, 2025 on Pith
  43. [63] Z. Yu, D. Xu, J. Yu, T. Yu, Z. Zhao, Y . Zhuang, and D. Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. InAAAI, pages 9127–9134. AAAI Press, 2019. 13 on Pith
  44. [64] C. Zhang, Y . Lin, Z. Wang, M. Bansal, and G. Bertasius. Silvr: A simple language-based video reasoning framework.Trans. Mach. Learn. Res., 2026 on Pith
  45. [65] X. Zhang, Z. Jia, Z. Guo, J. Li, B. Li, H. Li, and Y . Lu. Deep video discovery: Agentic search with tool use for long-form video understanding. InNeurIPS, 2025 on Pith
  46. [66] Y . Zhong, W. Ji, J. Xiao, Y . Li, W. Deng, and T. Chua. Video question answering: Datasets, algorithms and challenges. InEMNLP, pages 6439–6455. Association for Computational Linguistics, 2022 on Pith
  47. [67] P. Zhou, L. Wang, Z. Liu, Y . Hao, P. Hui, S. Tarkoma, and J. Kangasharju. A survey on generative AI and LLM for video generation, understanding, and streaming.arXiv preprint arXiv:2404.16038, 2024 on PithPith's reviewarXiv
  48. [68] S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y . Bisk, D. Fried, U. Alon, and G. Neubig. Webarena: A realistic web environment for building autonomous agents. InICLR. OpenReview.net, 2024 on Pith
  49. [69] J. Zuo, Y . Deng, L. Kong, J. Yang, R. Jin, Y . Zhang, N. Sang, L. Pan, Z. Liu, and C. Gao. Videolucy: Deep memory backtracking for long video understanding. InNeurIPS, 2025. 14 A Appendix – Comparison of AgentVidBench with Existing Benchmarks We note that AgentVidBench unique... on Pith
Cited by 1 Pith paper

Reviewed papers on Pith that cite this one. The full citation record

  1. Reading, Writing, Sharing, and Interacting with the World through Video cs.CV · Oct 2026

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment