Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Running ground-first, answer-first, and joint paths, then fusing the verified evidence, sets a new state of the art in grounded video QA.

desk verdict A useful multi-path recipe with a solid NExT-GQA result, but the DeVE-QA 'zero-shot' SOTA and the 2B-beats-7B claim are contradicted by the paper's own tables. read the letter →

arxiv 2506.18071 v2 pith:PHNZ4YTI submitted 2025-06-22 cs.CV cs.AI

Jisheng Dang , Huilin Song , Junbin Xiao , Bimei Wang , Han Peng , Haoxuan Li , Xun Yang , Meng Wang
show 1 more author
Tat-Seng Chua
This is my paper · ORCID
classification cs.CVcs.AI
keywords groundedvideoquestionansweringmulti-agentreasoningtemporalgroundingreflectionagentproductofexpertsmixturemomentretrievalvideo-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MUPA sets out to prove that a video question-answering system can ground its answers in visible evidence by refusing to rely on a single reasoning path. It runs three paths in parallel—localize-then-answer, answer-then-localize, and jointly localize-and-answer—then subjects each answer–evidence pair to a verifier and fuses the surviving candidates into one answer with temporal boundaries. The paper reports that this design raises temporal grounding fidelity (IoP) without lowering answer accuracy, with MUPA-7B reaching 30.3% grounded accuracy on NExT-GQA and 47.4% on DeVE-QA, and MUPA-2B surpassing 7B-scale competitors on NExT-GQA. If true, the contribution is a parameter-shared recipe for trustworthy video question answering that adds no trainable parameters at the fusion stage.

What carries the argument

Three reasoning paths supply diversity: Path-1 grounds the question to moments, then answers; Path-2 answers first, then retrieves moments by combining answer and question; Path-3 decodes answer and evidence jointly. The reflection agent carries the argument: a Verifier trained with LoRA judges whether each zoomed-in, boundary-marked span contains the queried event, and the product-of-experts rule $p_{ik}=c_{ik}v_{ik}$ re-ranks spans so a span survives only if the Grounder and Verifier both believe it. Multi-path fusion then converts spans to (start, end) points, clusters them with confidence-weighted $k$-means, and refines cluster centers by weighted least squares, so the final top-$K$ evidence spans are a closed-form compromise across paths.

What would settle it

Train the Verifier on human-annotated ground-truth spans from NExT-GQA or DeVE-QA instead of Grounder-produced pseudo-spans while holding everything else fixed; if MUPA's Acc@GQA and IoP scores do not change, the reflection stage is not the source of the gains, whereas a drop would show the current Verifier mostly imitates the Grounder's biases rather than auditing them.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that answer–evidence consistency, not extra capacity, is the bottleneck for grounded video question answering. The system uses four agents: a Grounder that localizes candidate moments, an Answerer that proposes answers, a GQA agent that does both at once, and a Reflective Agent that scores each answer–evidence pair, suppresses inconsistent pairs with a product-of-experts re-scoring rule, and fuses the rest by confidence-weighted clustering. The paper argues this makes answers driven primarily by video content rather than textual shortcuts. The headline empirical claim is state-of-the-art zero-shot grounded QA accuracy—30.3% on NExT-GQA and 47.4% on DeVE-QA at 7B scale—with grounding metrics improving across the board and answer accuracy staying on par with single-path models.

Load-bearing premise

The load-bearing premise is that the Verifier acts as an independent auditor of the Grounder's evidence, yet it is trained on pseudo-ground-truth spans produced by that same Grounder; if the Verifier has internalized the Grounder's mistakes, the claimed suppression of hallucinated evidence is not independently established.

Editorial extensions

If this is right

  • If the central claim holds, a 2B-parameter grounded-QA model can beat larger systems by spending compute on multiple reasoning paths and verification rather than on raw scale.
  • The same frozen backbone with swappable LoRA adapters covers moment retrieval, video QA, and grounded video QA, so one deployment can serve all three tasks.
  • Because the fusion step (product-of-experts re-scoring plus weighted $k$-means) is parameter-free, the reflection stage's only extra trainable component is the Verifier itself.
  • Each final answer comes with a confidence-weighted consensus time interval, making errors more inspectable by a human or downstream model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Verifier is trained on spans produced by the same Grounder (Appendix C.1), so the reflection agent may be auditing the Grounder's own biases rather than an external notion of correctness; the paper does not isolate how much of the reported gain is independent verification.
  • The answer-first path exploits the assumption that answer prediction is more reliable than temporal localization; if a deployment's localization model is stronger than its QA head, the fixed three-path weighting would likely be suboptimal.
  • Inference cost scales with the number of paths, and the paper notes there is no planner to prune redundant trajectories; a learned gating module that skips low-value paths is a natural follow-up and can be tested on the same benchmarks.
  • A scaling curve across backbone sizes would clarify whether multi-path diversity remains beneficial as single-path models grow stronger.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes MUPA, a multi-path agentic framework for Grounded VideoQA. It combines a Grounder, an Answerer, a joint GQA agent, and a Reflective Agent along three reasoning paths (ground-first, answer-first, joint), then applies a verifier with product-of-experts rescoring for single-path consistency and mixture-of-experts clustering for multi-path fusion. The authors report state-of-the-art Acc@GQA results on NExT-GQA (30.3% for MUPA-7B) and DeVE-QA (47.4% for MUPA-7B), and claim that the 2B model outperforms all 7B-scale competitors. The paper includes ablations of the three paths and the reflection mechanism, qualitative examples, and additional moment-retrieval experiments on TACoS.

Significance. If the NExT-GQA result holds, MUPA provides a useful demonstration that orchestrating multiple lightweight LoRA-tuned agents with a post-hoc verifier and fusion module can improve grounded VideoQA over a single-path VideoMind baseline. The multi-path decomposition and the PoE/MoE pipeline are simple, modular, and the code is released. The ablation on NExT-GQA shows consistent, if modest, gains from multi-path reasoning and from the reflection mechanism. However, the DeVE-QA half of the central SOTA claim, the zero-shot description of those results, and the blanket '2B outperforms all 7B' claim are contradicted by the paper's own tables and training-data appendix; the verifier is trained on the Grounder's own pseudo-labels, so the claimed impartial auditing is not independently established.

major comments (3)
  1. [§4.1, Table 2, Appendix C.1] The DeVE-QA results are not zero-shot. Table 5 lists DeVE-QA (78K samples) in the supervised fine-tuning data of the GQA agent, and Section 3.2.1 states that the GQA agent is shared by all three reasoning paths. Every path therefore sees DeVE-QA question-answer pairs and ground-truth moments during training. Consequently, the 47.4% Acc@GQA on DeVE-QA is an in-domain fine-tuned number, not a zero-shot result, and the 'Zero-shot' label in Table 2 and the phrase 'zero-shot results on the challenging DEVE-QA benchmark' in Section 4.1 are inaccurate. This also contradicts the Section 4.1 claim that MUPA relies on 'no task-specific fine-tuning.' The NExT-GQA result is not affected, but the DeVE-QA part of the SOTA claim must be reframed as a fine-tuned comparison.
  2. [Abstract and Table 2] The claim that 'Despite using only 2B parameters, our method outperforms all 7B-scale competitors' is false on DeVE-QA: MUPA-2B achieves 43.9 Acc@GQA while VideoMind-7B achieves 44.2. The 2B claim does hold on NExT-GQA (28.7 vs. 28.2 for VideoMind-7B), but the abstract's blanket statement and Section 4.1's 'outperforming all existing 7B-scale competitors' must be restricted to NExT-GQA or otherwise qualified.
  3. [§3.3.1 and Appendix C.1] The Verifier is trained on pseudo-labels generated by the pretrained Grounder, and at inference the PoE fusion multiplies the Grounder's confidence by the Verifier score. The paper presents the Reflective Agent as 'an impartial auditor,' but the Verifier's training signal is the Grounder's own output, so the verification may simply imitate the Grounder's biases rather than independently audit its evidence. The claimed suppression of hallucinated evidence is therefore not independently established. The authors should either train the Verifier with human-annotated or otherwise independent spans, or provide an analysis (e.g., per-span error breakdown or calibration against ground-truth moments) showing that the Verifier does not merely confirm the Grounder's mistakes.
minor comments (4)
  1. [§4.1 and Table 1] The text states that MUPA-2B achieves 29.0% Acc@GQA and a mIoP of 39.7%, but Table 1 reports 28.7 and 39.1; these numbers should be reconciled.
  2. [Table 3] The column header 'w w/o R1@0.5 mIoU R1@0.5 mIoP' is ambiguous as typeset, and the table does not clearly show a multi-path-with-reflection versus multi-path-without-reflection comparison. Section 4.3's claim that the Reflection Agent improves the full MUPA system would be better supported by an explicit row pair for the multi-path condition.
  3. [§3.3.2 and Algorithm 2] Equation (3) defines answer consolidation using sum over i and k of p_ik, while Algorithm 2 uses sum over i of p_i; the paper should clarify which path-level or span-level confidence is intended in the final voting.
  4. [Appendix C.1] The reference to 'HiREST [? ]' is unresolved, and the appendix title 'Discussion on Entropy-based Measurement' does not match the actual content of the appendix.

Circularity Check

1 steps flagged · score 6.0 of 10

Circularity score 6: the reflection Verifier is trained on the Grounder's own pseudo-labeled spans and then multiplied into the Grounder's confidence, making 'verification' partly self-confirming; external benchmarks keep the central SOTA claim partially independent.

  1. fitted input called prediction [Appendix C.1 (Verifier dataset construction) and Section 3.3.1 (PoE Re-scoring, pik = cik * vik)]
    "Verifier datasets are constructed by running the pretrained Grounder on each source benchmark and collecting its predicted spans as pseudo-ground-truth. ... After interpreting cik and vik as independent posterior probabilities, we combine them using the PoE, to obtain the fused confidence score, pik = cik vik."

    The Verifier's training signal is the Grounder's own predicted spans, and at inference the same Verifier scores the Grounder's spans. The Verifier score vik is multiplied directly into the Grounder's confidence cik to re-rank evidence. A Verifier trained on the Grounder's pseudo-labels will tend to approve the very spans the Grounder favors, so it cannot independently detect systematic hallucinated evidence produced by the Grounder. The claimed 'impartial auditor' and the claim that reflection 'eliminates hallucinated evidence' are therefore partly self-confirming by construction. The final benchmark numbers are still measured against external ground truth, so the circularity affects the reflection mechanism's causal justification rather than the raw reported scores.

full rationale

The main evaluation chain is not circular: MUPA's final Acc@GQA numbers are measured against external NExT-GQA and DeVE-QA ground truth, and the multi-path aggregation is governed by closed-form PoE/MoE operations that are not fitted to the test labels. The one load-bearing circular step is the reflection Verifier: its training labels are the Grounder's own predicted spans (pseudo-ground-truth), and its inference score is multiplied into the Grounder's confidence to re-rank evidence. Hence the 'verification' is partly a self-agreement check rather than an independent audit, and the claimed grounding-fidelity improvement from reflection is not fully established by the internal machinery. I do not count the DeVE-QA 'zero-shot' contradiction as circularity: Section 4.1 calls Table 2 'zero-shot results on the challenging DEVE-QA benchmark,' but Table 5 lists DeVE-QA (78K) inside the GQA supervised fine-tuning set, so MUPA's 47.4% Acc@GQA is an in-domain fine-tuned number and the comparison with VideoMind-7B's zero-shot 44.2 is unfair. That is a contamination/labeling concern for the SOTA and efficiency claims, not a derivation that reduces to its own inputs. No load-bearing self-citation chain is present: VideoMind [20] is an external baseline whose Chain-of-LoRA and Grounder/Verifier designs are reused, but MUPA is also benchmarked against VideoMind on the same external test sets, so the central empirical claim retains independent content.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a set of design choices (loss weights, span counts, cluster count, IoU thresholds) and on the validity of the PoE and MoE fusion assumptions. The largest unexamined burden is the Verifier being trained on Grounder pseudo-labels, which threatens the independence of the reflection mechanism. No new physical or mathematical entities are introduced.

free parameters (5)
  • Loss weights lambda_ans and lambda_loc in Eq. (1)
    Balancing weights for answer and localization losses; values are not reported in the paper, but the grounded QA accuracy depends on this balance.
  • Number of candidate spans N = 5
    The Verifier loop evaluates the top-N=5 Grounder spans per path; changing N changes which evidence is considered.
  • Number of final evidence segments K = 3 (default 5)
    The MoE fusion clusters spans into K segments; the implementation default is 5 but the authors override it to 3.
  • NMS IoU threshold = 0.75
    Redundant spans from the Grounder are eliminated with non-maximum suppression at IoU 0.75 during inference.
  • Verifier positive IoU threshold = 0.5
    During Verifier training, spans with IoU over 0.5 are treated as positive examples, defining what counts as a correct temporal match.
assumptions (5)
  • domain assumption Grounder confidence c_ik and Verifier score v_ik are independent posterior probabilities for the same event.
    Section 3.3.1: the PoE product p_ik = c_ik * v_ik assumes independence. The Verifier is trained on the Grounder's own outputs, so this independence is questionable.
  • standard math Weighted k-means on span endpoints is equivalent to maximizing the likelihood of an isotropic Gaussian mixture under the given weights.
    Section 3.3.2 asserts this equivalence without derivation; it is a standard but nontrivial result, and the weighting scheme is ad hoc.
  • standard math The weighted average of cluster points in Eq. (5) is the Gauss-Markov minimum-variance unbiased estimate of the true boundary.
    Section 3.3.2: Gauss-Markov optimality requires weights proportional to inverse variances; the paper uses confidence weights, so the optimality claim is not justified.
  • domain assumption Acc@GQA with IoP >= 0.5 is a valid measure of answer grounding and trustworthiness.
    The paper adopts the benchmark metric from NExT-GQA and DeVE-QA; IoP rewards short predicted spans inside the ground truth more leniently than IoU.
  • ad hoc to paper Verifier pseudo-labels generated by the pretrained Grounder provide a realistic training signal.
    Appendix C.1: Verifier training data are the Grounder's own predicted spans, which may encode systematic Grounder errors rather than independent judgments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering." pith.science (2026). https://pith.science/paper/PHNZ4YTI

@misc{pith2026250618071,
  author       = {Pith},
  title        = {Pith review of: MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PHNZ4YTI}},
  note         = {Machine review of arXiv:2506.18071}
}
read the original abstract

Grounded Video Question Answering (Grounded VideoQA) requires aligning textual answers with explicit visual evidence. However, modern multimodal models often rely on linguistic priors and spurious correlations, resulting in poorly grounded predictions. In this work, we propose MUPA, a cooperative MUlti-Path Agentic approach that unifies video grounding, question answering, answer reflection and aggregation to tackle Grounded VideoQA. MUPA features three distinct reasoning paths on the interplay of grounding and QA agents in different chronological orders, along with a dedicated reflection agent to judge and aggregate the multi-path results to accomplish consistent QA and grounding. This design markedly improves grounding fidelity without sacrificing answer accuracy. Despite using only 2B parameters, our method outperforms all 7B-scale competitors. When scaled to 7B parameters, MUPA establishes new state-of-the-art results, with Acc@GQA of 30.3% and 47.4% on NExT-GQA and DeVE-QA respectively, demonstrating MUPA' effectiveness towards trustworthy video-language understanding. Our code is available in https://github.com/longmalongma/MUPA.

Figures

Figures reproduced from arXiv: 2506.18071 by the authors.

Figure 1
Figure 1. Comparison of end-to-end approach and our multi-agent & multi-path approach. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview. The controller first senses the incoming task and steers the query to the reasoning routes with multi-path reasoning. Each single-path output is then critiqued by the reflection agent, which employs a verifier followed by a PoE aggregation to refine evidence–answer hypotheses. For GQA, the reflections from Paths 1–3 undergo further reconciliation through a MoE fusion mechanism, yielding final answer along … view at source ↗
Figure 3
Figure 3. Detailed architecture of multi-path reasoning for Grounded VideoQA. MUPA employs [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of the MUPA workflow. Each of the three reasoning paths produces candidate [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the Reflection agent’s two core operations: [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding

    cs.CV 2026-08 conditional novelty 5.0 of 10

    A training-free multi-agent explore-verify system with four specialized agents and a shared evidence registry outperforms zero-shot and RL-finetuned baselines on video anomaly understanding benchmarks.

  2. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Reference graph

Works this paper leans on

66 extracted references · 37 canonical work pages · cited by 2 Pith papers

  1. [1]

    Revealing single frame bias for video-and-language learning,

    J. Lei, T. L. Berg, and M. Bansal, “Revealing single frame bias for video-and-language learning,” arXiv preprint arXiv:2206.03428, 2022

    arXiv 2022

  2. [2]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al., “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,” arXiv preprint arXiv:2409.12191, 2024

    arXiv 2024

  3. [3]

    Rethinking Multi-Modal Alignment in Video Question Answering from Feature and Sample Perspectives

    S. Xiao, L. Chen, K. Gao, Z. Wang, Y . Yang, Z. Zhang, and J. Xiao, “Rethinking multi-modal alignment in video question answering from feature and sample perspectives,”arXiv preprint arXiv:2204.11544, 2022

    work page Pith review arXiv 2022

  4. [4]

    Can I trust your answer? Visually grounded video question answering,

    J. Xiao, A. Yao, Y . Li, and T.-S. Chua, “Can I trust your answer? Visually grounded video question answering,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 13204–13214

    work page 2024

  5. [5]

    Tvqa: Localized, compositional video question answering,

    Lei, Jie, Yu, Licheng, Bansal, Mohit, et al., “Tvqa: Localized, compositional video question answering,” arXiv preprint arXiv:1809.01696, 2018

    arXiv 2018

  6. [6]

    Zero-shot video question answering via frozen bidirectional language models,

    Yang, Antoine, Miech, Antoine, Sivic, Josef,et al., “Zero-shot video question answering via frozen bidirectional language models,” Advances in Neural Information Processing Systems, 2022, pp. 124–141

    work page 2022

  7. [7]

    Demonstrating and reducing shortcuts in vision-language representation learning,

    Bleeker, Maurits, Hendriksen, Mariya, Yates, Andrew, et al., “Demonstrating and reducing shortcuts in vision-language representation learning,” arXiv preprint arXiv:2402.17510, 2024

    arXiv 2024

  8. [8]

    Question-Answering Dense Video Events,

    Qin, Hangyu, Xiao, Junbin, Yao, Angela, “Question-Answering Dense Video Events,”arXiv preprint arXiv:2409.04388, 2024

    arXiv 2024

Show all 66 references
  1. [9]

    Grounding action descrip- tions in videos,

    Regneri, Michaela, Rohrbach, Marcus, Wetzel, Dominikus, et al., “Grounding action descrip- tions in videos,” Transactions of the Association for Computational Linguistics , vol. 1, pp. 25–36, 2013

    2013

  2. [10]

    Qvhighlights: Detecting moments and highlights in videos via natural language queries.(2021),

    Lei, J, Berg, TL, Bansal, M, “Qvhighlights: Detecting moments and highlights in videos via natural language queries.(2021),” URL https://arxiv.org/abs/2107.09609, 2021

    2021 arXiv

  3. [11]

    Videoagent: Long-form video understanding with large language model as agent,

    Wang, Xiaohan, Zhang, Yuhui, Zohar, Orr,et al., “Videoagent: Long-form video understanding with large language model as agent,” in European Conference on Computer Vision, 2024, pp. 58–76

    2024

  4. [12]

    Morevqa: Exploring modular reason- ing models for video question answering,

    Min, Juhong, Buch, Shyamal, Nagrani, Arsha, et al., “Morevqa: Exploring modular reason- ing models for video question answering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 13235–13245

    2024

  5. [13]

    Videoagent: A memory-augmented multimodal agent for video understanding,

    Fan, Yue, Ma, Xiaojian, Wu, Rujie, et al., “Videoagent: A memory-augmented multimodal agent for video understanding,” in European Conference on Computer Vision, 2024, pp. 75–92. 10

    2024

  6. [14]

    Retrieval-based video language model for efficient long video question answering,

    Xu, Jiaqi, Lan, Cuiling, Xie, Wenxuan, et al., “Retrieval-based video language model for efficient long video question answering,” arXiv preprint arXiv:2312.04931, 2023

    2023

  7. [15]

    Hierarchical video-moment retrieval and step-captioning,

    Zala, Abhay, Cho, Jaemin, Kottur, Satwik, et al., “Hierarchical video-moment retrieval and step-captioning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23056–23065

    2023

  8. [16]

    Tgif-qa: Toward spatio-temporal reasoning in visual question answering,

    Jang, Yunseok, Song, Yale, Yu, Youngjae, Kim, Youngjin, and Kim, Gunhee, “Tgif-qa: Toward spatio-temporal reasoning in visual question answering,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2758–2766

    2017

  9. [17]

    Video Question Answering via Gradually Refined Attention over Appearance and Motion,

    Xu, Dejing, Zhao, Zhou, Xiao, Jun, et al., “Video Question Answering via Gradually Refined Attention over Appearance and Motion,” in ACM Multimedia, 2017

    2017

  10. [18]

    Activitynet-qa: A dataset for understanding complex web videos via question answering,

    Yu, Zhou, Xu, Dejing, Yu, Jun,et al., “Activitynet-qa: A dataset for understanding complex web videos via question answering,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 9127–9134, 2019

    2019

  11. [19]

    Next-qa: Next phase of question- answering to explaining temporal actions,

    Xiao, Junbin, Shang, Xindi, Yao, Angela, and Chua, Tat-Seng, “Next-qa: Next phase of question- answering to explaining temporal actions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9777–9786

    2021

  12. [20]

    VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning,

    Liu, Ye, Lin, Kevin Qinghong, Chen, Chang Wen, and Shou, Mike Zheng, “VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning,” in arXiv preprint arXiv:2503.13444, 2025

    2025

  13. [21]

    Videoqa in the era of llms: An empirical study,

    Xiao, Junbin, Huang, Nanxin, Qin, Hangyu, et al., “Videoqa in the era of llms: An empirical study,”International Journal of Computer Vision, pp. 1–24, 2025

    2025

  14. [22]

    Grounded-videollm: Sharpening fine-grained temporal grounding in video large language models,

    Wang, Haibo, Xu, Zhiyang, Cheng, Yu,et al., “Grounded-videollm: Sharpening fine-grained temporal grounding in video large language models,” arXiv preprint arXiv:2410.03290, 2024

    2024 arXiv

  15. [23]

    Cinepile: A long video question answering dataset and benchmark,

    Rawal, Ruchit, Saifullah, Khalid, Farré, Miquel, et al., “Cinepile: A long video question answering dataset and benchmark,” arXiv preprint arXiv:2405.08813, 2024

    2024 arXiv

  16. [24]

    The surprising effectiveness of multimodal large language models for video moment retrieval,

    Meinardus, Boris, Batra, Anil, Rohrbach, Anna, and Rohrbach, Marcus, “The surprising effectiveness of multimodal large language models for video moment retrieval,”arXiv preprint arXiv:2406.18113, 2024

    2024

  17. [25]

    Neptune: The Long Orbit to Bench- marking Long Video Understanding,

    Nagrani, Arsha, Zhang, Mingda, Mehran, Ramin, et al., “Neptune: The Long Orbit to Bench- marking Long Video Understanding,”arXiv preprint arXiv:2412.09582, 2024

    2024 arXiv

  18. [26]

    Self-consistency improves chain of thought reasoning in language models,

    Wang, Xuezhi, Wei, Jason, Schuurmans, Dale, Le, Quoc, Chi, Ed, Narang, Sharan, Chowdh- ery, Aakanksha, and Zhou, Denny, “Self-consistency improves chain of thought reasoning in language models,” arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  19. [27]

    Chain-of-thought prompting elicits reasoning in large language models,

    Wei, Jason, Wang, Xuezhi, Schuurmans, Dale, et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems, vol. 35, pp. 24824–24837, 2022

    2022

  20. [28]

    Tree of thoughts: Deliberate problem solving with large language models,

    Yao, Shunyu, Yu, Dian, Zhao, Jeffrey,et al., “Tree of thoughts: Deliberate problem solving with large language models,” Advances in Neural Information Processing Systems, vol. 36, pp. 11809–11822, 2023

    2023

  21. [29]

    Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents,

    He, Chengbo, Zou, Bochao, Li, Xin, et al., “Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents,”arXiv preprint arXiv:2501.00430, 2024

    2024 arXiv

  22. [30]

    React: Synergizing reasoning and acting in language models,

    Yao, Shunyu, Zhao, Jeffrey, Yu, Dian, et al., “React: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR), 2023

    2023

  23. [31]

    Reflexion: Language agents with verbal reinforcement learning,

    Shinn, Noah, Cassano, Federico, Gopinath, Ashwin, et al., “Reflexion: Language agents with verbal reinforcement learning,” Advances in Neural Information Processing Systems, vol. 36, pp. 8634–8652, 2023

    2023

  24. [32]

    Improving multi-agent debate with sparse communication topology,

    Li, Yunxuan, Du, Yibing, Zhang, Jiageng, et al., “Improving multi-agent debate with sparse communication topology,”arXiv preprint arXiv:2406.11776, 2024

    2024 arXiv

  25. [33]

    An empirical study of end-to-end video-language transformers with masked visual modeling,

    Fu, Tsu-Jui, Li, Linjie, Gan, Zhe, et al., “An empirical study of end-to-end video-language transformers with masked visual modeling,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22898–22909

    2023

  26. [34]

    Lan- guage repository for long video understanding,

    Kahatapitiya, Kumara, Ranasinghe, Kanchana, Park, Jongwoo, and Ryoo, Michael S., “Lan- guage repository for long video understanding,” arXiv preprint arXiv:2403.14622, 2024. 11

    2024 arXiv

  27. [35]

    Streaming long video understanding with large language models,

    Qian, Rui, Dong, Xiaoyi, Zhang, Pan, et al., “Streaming long video understanding with large language models,” Advances in Neural Information Processing Systems, vol. 37, pp. 119336– 119360, 2024

    2024

  28. [36]

    A simple LLM framework for long-range video question-answering,

    Zhang, Ce, Lu, Taixi, Islam, Md Mohaiminul, et al., “A simple LLM framework for long-range video question-answering,” arXiv preprint arXiv:2312.17235, 2023

    2023 arXiv

  29. [37]

    Hawkeye: Training video-text LLMs for grounding text in videos,

    Wang, Yueqian, Meng, Xiaojun, Liang, Jianxin,et al., “Hawkeye: Training video-text LLMs for grounding text in videos,” arXiv preprint arXiv:2403.10228, 2024

    2024 arXiv

  30. [38]

    Task preference optimization: Improving multimodal large language models with vision task alignment,

    Yan, Ziang, Li, Zhilin, He, Yinan,et al., “Task preference optimization: Improving multimodal large language models with vision task alignment,” arXiv preprint arXiv:2412.19326, 2024

    2024 arXiv

  31. [39]

    TVR: A large-scale dataset for video-subtitle moment retrieval,

    Lei, Jie, Yu, Licheng, Berg, Tamara L., and Bansal, Mohit, “TVR: A large-scale dataset for video-subtitle moment retrieval,” in ECCV, 2020, pp. 447–463

    2020

  32. [40]

    UMT: Unified multi-modal transformers for joint video moment retrieval and highlight detection,

    Liu, Ye, Li, Siyuan, Wu, Yang,et al., “UMT: Unified multi-modal transformers for joint video moment retrieval and highlight detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3042–3051

    2022

  33. [41]

    MomentDiff: Generative video moment retrieval from random to real,

    Li, Pandeng, Xie, Chen-Wei, Xie, Hongtao, et al., “MomentDiff: Generative video moment retrieval from random to real,” Advances in Neural Information Processing Systems, vol. 36, pp. 65948–65966, 2023

    2023

  34. [42]

    Query-dependent video representation for moment retrieval and highlight detection,

    Moon, WonJun, Hyun, Sangeek, Park, SangUk, et al., “Query-dependent video representation for moment retrieval and highlight detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23023–23033

    2023

  35. [43]

    UnivTG: Towards unified video- language temporal grounding,

    Lin, Kevin Qinghong, Zhang, Pengchuan, Chen, Joya, et al., “UnivTG: Towards unified video- language temporal grounding,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 2794–2804

    2023

  36. [44]

    R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding,

    Liu, Ye, He, Jixuan, Li, Wanhua,et al., “R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding,” inECCV, 2024, pp. 421–438

    2024

  37. [45]

    Learning 2D temporal adjacent networks for moment localization with natural language,

    Zhang, Songyang, Peng, Houwen, Fu, Jianlong, and Luo, Jiebo, “Learning 2D temporal adjacent networks for moment localization with natural language,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12870–12877, 2020

    2020

  38. [46]

    Span-based localizing network for natural language video localization,

    Zhang, Hao, Sun, Aixin, Jing, Wei, and Zhou, Joey Tianyi, “Span-based localizing network for natural language video localization,” arXiv preprint arXiv:2004.13931, 2020

    2004 arXiv

  39. [47]

    Dense- captioning events in videos,

    Krishna, Ranjay, Hata, Kenji, Ren, Frederic, Fei-Fei, Li, and Niebles, Juan Carlos, “Dense- captioning events in videos,” inProceedings of the IEEE International Conference on Computer Vision, 2017, pp. 706–715

    2017

  40. [48]

    Negative sample matters: A renaissance of metric learning for temporal grounding,

    Wang, Zhenzhi, Wang, Limin, Wu, Tao, Li, Tianhao, and Wu, Gangshan, “Negative sample matters: A renaissance of metric learning for temporal grounding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, pp. 2613–2623, 2022

    2022

  41. [49]

    Towards generalisable video moment retrieval: Visual-dynamic injection to image-text pre-training,

    Luo, Dezhao, Huang, Jiabo, Gong, Shaogang, Jin, Hailin, and Liu, Yang, “Towards generalisable video moment retrieval: Visual-dynamic injection to image-text pre-training,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23045– 23055

    2023

  42. [50]

    VideoChat: Chat-centric video understanding,

    Li, KunChang, He, Yinan, Wang, Yi, et al., “VideoChat: Chat-centric video understanding,” arXiv preprint arXiv:2305.06355, 2023

    2023 arXiv

  43. [51]

    Video-LLaMA: An instruction-tuned audio-visual language model for video understanding,

    Zhang, Hang, Li, Xin, and Bing, Lidong, “Video-LLaMA: An instruction-tuned audio-visual language model for video understanding,” arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

  44. [52]

    Video- ChatGPT: Towards detailed video understanding via large vision and language models,

    Maaz, Muhammad, Rasheed, Hanoona, Khan, Salman, and Khan, Fahad Shahbaz, “Video- ChatGPT: Towards detailed video understanding via large vision and language models,”arXiv preprint arXiv:2306.05424, 2023

    2023 arXiv

  45. [53]

    V ALLEY: Video assistant with large language model enhanced ability,

    Luo, Ruipu, Zhao, Ziwang, Yang, Min, et al., “V ALLEY: Video assistant with large language model enhanced ability,”arXiv preprint arXiv:2306.07207, 2023

    2023 arXiv

  46. [54]

    ChatVTG: Video temporal grounding via chat with video dialogue large language models,

    Qu, Mengxue, Chen, Xiaodong, Liu, Wu, Li, Alicia, and Zhao, Yao, “ChatVTG: Video temporal grounding via chat with video dialogue large language models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1847–1856. 12

    2024

  47. [55]

    Momentor: Advancing video large language model with fine-grained temporal reasoning,

    Qian, Long, Li, Juncheng, Wu, Yu,et al., “Momentor: Advancing video large language model with fine-grained temporal reasoning,” arXiv preprint arXiv:2402.11435, 2024

    2024 arXiv

  48. [56]

    ET Bench: Towards open-ended event-level video-language understanding,

    Liu, Ye, Ma, Zongyang, Qi, Zhongang, Wu, Yang, Shan, Ying, and Chen, Chang W, “ET Bench: Towards open-ended event-level video-language understanding,”Advances in Neural Information Processing Systems, vol. 37, pp. 32076–32110, 2024

    2024

  49. [57]

    LITA: Language instructed temporal-localization assistant,

    Huang, De-An, Liao, Shijia, Radhakrishnan, Subhashree, et al., “LITA: Language instructed temporal-localization assistant,” in European Conference on Computer Vision, 2024, pp. 202– 218

    2024

  50. [58]

    RexTime: A benchmark suite for reasoning- across-time in videos,

    Chen, Jr-Jen, Liao, Yu-Chien, Lin, Hsi-Che,et al., “RexTime: A benchmark suite for reasoning- across-time in videos,” Advances in Neural Information Processing Systems , vol. 37, pp. 28662–28673, 2024

    2024

  51. [59]

    VTimeLLM: Empower LLM to grasp video moments,

    Huang, Bin, Wang, Xin, Chen, Hong, Song, Zihan, and Zhu, Wenwu, “VTimeLLM: Empower LLM to grasp video moments,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14271–14280

    2024

  52. [60]

    TimeChat: A time-sensitive mul- timodal large language model for long video understanding,

    Ren, Shuhuai, Yao, Linli, Li, Shicheng, Sun, Xu, and Hou, Lu, “TimeChat: A time-sensitive mul- timodal large language model for long video understanding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14313–14323

    2024

  53. [61]

    LoRA: Low-rank adaptation of large language models,

    Hu, Edward J, Shen, Yelong, Wallis, Phillip, et al., “LoRA: Low-rank adaptation of large language models,” International Conference on Learning Representations (ICLR), vol. 1, no. 2, pp. 3, 2022

    2022

  54. [62]

    Self-chained image-language model for video localization and question answering,

    Yu, Shoubin, Cho, Jaemin, Yadav, Prateek, and Bansal, Mohit, “Self-chained image-language model for video localization and question answering,” Advances in Neural Information Process- ing Systems, vol. 36, pp. 76749–76771, 2023

    2023

  55. [63]

    GPT-4o System Card,

    Hurst, Aaron, Lerer, Adam, Goucher, Adam P,et al., “GPT-4o System Card,”arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  56. [64]

    Cosmo: Contrastive streamlined multimodal model with interleaved pre-training,

    Wang, Alex Jinpeng, Li, Linjie, Lin, Kevin Qinghong,et al., “Cosmo: Contrastive streamlined multimodal model with interleaved pre-training,” arXiv preprint arXiv:2401.00849, 2024

    2024 arXiv

  57. [65]

    Reinforcing video reasoning with focused thinking,

    Dang, Jisheng, Wu, Jingze, Wang, Teng, et al., “Reinforcing video reasoning with focused thinking,” arXiv preprint arXiv:2505.24718, 2025

    2025 arXiv

  58. [66]

    SynPO: Synergizing descriptiveness and preference optimization for video detailed captioning,

    Dang, Jisheng, Zhang, Yizhou, Ye, Hao et al. , “SynPO: Synergizing descriptiveness and preference optimization for video detailed captioning,” arXiv preprint arXiv:2506.00835, 2025. 13 A Discussion on Entropy-based Measurement In this document, we provide more descriptions of ...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.