Pith. sign in

REVIEW 5 major objections 5 minor 46 references

POVQA shows that compressing each second of video into one pooled image, interleaved with subtitles and trained on rationale+answer labels, lifts long-video QA F1 from 0.212 to about 0.55 with only 239 examples

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

POVQA reports large F1 gains on a new 239-example video QA dataset after rationale-based fine-tuning, but its own keyframe-only ablation matches the full pooling pipeline, and fine-tuning hurts zero-shot TVQA accuracy.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection The paper's own keyframe ablation shows that a single frame matches the full pooled pipeline, so the central 'temporal pooling drives gains' claim is unsupported. the 5 major comments →

arxiv 2510.01009 v3 pith:OT53LJ3Z submitted 2025-10-01 cs.CV cs.MM

POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiency

Ashim Dahal , Ankit Ghimire , Saydul Akbar Murad , Nick Rahimi This is my paper
classification cs.CV cs.MM
keywords video question answeringtemporal poolingrationale supervisiondata efficiencypreference optimizationzero-shot transferlong-video understandingvision-language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that for long-video question answering, condensing every second of footage into a single temporally pooled image (1 fps) preserves enough evidence to answer questions, provided the model is fine-tuned on rationale-plus-answer supervision. On the authors' ReasonVQA dataset (239 human-annotated QA with rationales across 12 movies), supervised fine-tuning lifts F1 from 0.212 to about 0.55, with pooled frames plus rationale supervision driving most of the gain. The same pipeline transfers zero-shot to TVQA at 64.7% after SFT+DPO, though pooling alone reaches 69.7%. If this holds, it suggests that data-efficient, token-budget-friendly video QA is feasible with a small labeled rationales corpus.

Core claim

The central claim is that 1 Hz temporal pooling—one motion-summarizing image per second—combined with lightweight supervision on rationale+answer targets lets a 7B vision-language model answer long-video questions under a fixed token budget, and that the temporal evidence from pooling is what carries the task. The authors report that SFT improves the best pooled-only baseline from 0.212 to 0.550 F1, and that the gains persist across pooling schemes, with training on one pooling operator transferring to others at test time. They also report zero-shot TVQA accuracy of 64.7% after SFT+DPO, exceeding prior zero-shot baselines. They caution that these results are preliminary and that pooling can

What carries the argument

The central object is the temporally pooled frame: four pooling operators (blend-blur with last frame, weighted average, exponential weighted average, ramp weighted average) each compress the 24–60 frames of one second into a single image that is interleaved with subtitle text and fed to the model. That compression reduces a 5-minute clip to ≤60 pooled visual tokens while keeping every second represented, shifting the burden from raw-frame counting to spatial attention over pooled tokens plus temporal attention across adjacent pooled seconds.

Load-bearing premise

The claim rests on the assumption that the 60 pooled frames contribute temporal evidence beyond what the single key-frame at the question moment already supplies; Table 6 suggests that for token-level metrics the key-frame alone may be sufficient.

What would settle it

Compare the fine-tuned model on ReasonVQA with all 60 pooled frames versus with only the key-frame. If the pooled model does not beat the key-frame model on F1, BLEU-4, and ROUGE-L by a margin beyond noise—and only embedding cosine similarity improves—then the paper's load-bearing temporal-pooling claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Video QA can be done under a fixed token budget (about 16k tokens for 5 minutes) without sacrificing dense temporal coverage.
  • Rationale supervision on a very small dataset (239 examples) can yield large gains over pooled-only baselines.
  • Pooling operators generalize across train/eval: training on one pooling scheme transfers to others seen at test time.
  • Zero-shot transfer to an external benchmark (TVQA) is competitive with or better than prior zero-shot systems, suggesting pooling alone is a strong driver of accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editor's inference: The KeyFrame ablation implies the temporal-pooling claim may be overstated for lexical metrics, since key-frame-only models match pooled F1 after fine-tuning; the real contribution of pooling appears to be semantic grounding (embedding similarity) rather than exact answer tokens.
  • Editor's inference: The TVQA discrepancy (pooling-only 69.7% vs SFT+DPO 64.7%) hints that fine-tuning on ReasonVQA may not transfer positively to TVQA, and that a domain-matched preference dataset might be needed to turn rationale training into accuracy gains.
  • Editor's inference: A testable extension is to evaluate pooled inputs at higher frame rates (2–4 fps) versus key-frame-only to find the point where temporal evidence becomes lexically essential rather than merely helpful for embedding similarity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes POVQA, a 1-Hz temporal pooling pipeline for long-video question answering. Each second of video is compressed into one pooled frame via four operators — weighted average, exponentially weighted average, ramp-weighted average, and blend-blur with last frame — and these frames are interleaved with subtitles in chronological order. The authors then fine-tune Qwen2.5-VL-7B with QLoRA SFT on rationale-plus-answer targets and optionally apply DPO. They introduce ReasonVQA, a dataset of 239 human-annotated QA plus rationale triples from 12 movies, and report SFT raising F1 from 0.212 to 0.550, with SFT+DPO reaching 0.543. They also report 64.7% zero-shot accuracy on TVQA after SFT+DPO. The central claim is that 'pooled evidence plus rationale supervision provides the main performance gains in this setting.'

Significance. If the central claim were supported, the paper would demonstrate that dense 1-Hz temporal pooling, combined with a very small rationale-supervised tuning set, yields large gains over pooled-only baselines and transfers zero-shot. The paper provides explicit pooling formulas with few free parameters, a new dataset, and an extensive cross-evaluation sweep (50 experimental runs), and the external TVQA check is an independent benchmark. However, the main empirical claim is contradicted by the paper's own ablation: KeyFrame-only fine-tuned models achieve F1 0.553–0.563, matching or exceeding the best pooled 60-frame results (0.543–0.550) on the headline F1 metric. The TVQA transfer result is also internally inconsistent because fine-tuning lowers accuracy relative to the paper's own pooling-only baseline. As a result, the paper's claimed contribution — temporal pooling as the driver of gains — is not established by the reported evidence.

major comments (5)
  1. [§4.3, Table 6 vs. Tables 2–3; Abstract] The abstract's central claim that 'pooled evidence plus rationale supervision provides the main performance gains' is contradicted by Table 6. KeyFrame-only ablation (max over SFT/DPO) reaches F1 0.553–0.563 across BBLF/WA/WAE/WAR, while the best pooled configurations in Tables 2–3 achieve F1 0.550 (SFT, WAE-trained on WA evaluation) or 0.543 (SFT+DPO). Thus the 0.212→0.550 gain is explainable by SFT on rationale+answer targets alone; it does not require the 60 pooled frames. The only positive pooled deltas are Embed Cosine (+0.013) and Embed Cos-R (+0.033), which are not the headline F1 metric and are not validated against human judgments. The §4.3 defense that pooled frames buy 'semantic grounding and reasoning consistency' is an assertion, not a demonstrated effect, and the paper's own admission that 'lexical deltas are near-zero or slightly negative' undermines the stated conclusion.
  2. [§4.2.2, Table 5 vs. Table 7; Conclusion] The TVQA results are internally inconsistent. Table 7 shows pooling-only (no fine-tuning) achieves 69.7% zero-shot accuracy, while Table 5 reports 64.7% after SFT+DPO. Thus fine-tuning on ReasonVQA reduces zero-shot TVQA accuracy by 5 percentage points. The conclusion states 'SFT+DPO sharpen reasoning faithfulness and conciseness' and the abstract highlights 64.7%, but the paper's own baseline is higher. This contradicts the claim of positive zero-shot transfer and should be reconciled or the claim removed.
  3. [Abstract vs. §5 Conclusion vs. Tables 2–3] The main numerical result is reported inconsistently: the abstract says 'SFT improves the best pooled-only baseline from 0.212 to 0.550 F1,' the conclusion says 'improves F1 from 0.212 to 0.543,' Table 2's best SFT F1 is 0.550, and Table 3's best SFT+DPO F1 is 0.543. The reader cannot tell which number is intended as the headline result. This is a load-bearing inconsistency because the paper's thesis rests on the magnitude of this improvement.
  4. [§4.2.1, Table 4] The claimed benefit of DPO is unsupported by Table 4. Across all evaluation poolers, DPO deltas are mostly negative or negligible: F1 −0.002 to −0.009, BLEU-1 −0.001 to −0.020, and Embed Cos-R gains of at most +0.007. The positive BLEU-4 delta of +0.026 under WAR is isolated. No statistical significance tests are reported, and with only 239 annotated examples, differences of 0.01–0.02 F1 are likely noise. The sentence 'DPO consistently strengthens rationale quality' is therefore an overstatement.
  5. [§4.1, §4.2.1] ReasonVQA has only 239 examples and no confidence intervals or significance tests are reported for the F1/BLEU/ROUGE comparisons. The claims that 'the diagonal is not always optimal' and that cross-pooler robustness holds are based on tiny differences (e.g., 0.506 vs. 0.533 F1) without uncertainty quantification. This is especially concerning because Table 6 shows KeyFrame-only models produce nearly the same scores as pooled models, suggesting the metric cannot discriminate the temporal-pooling hypothesis with the current dataset size.
minor comments (5)
  1. [§3.6, Eq. (17)] The token budget calculation assumes m=256 visual tokens per image, but Qwen2.5-VL typically uses dynamic resolution and the actual number may vary. Please state whether m is fixed or averaged.
  2. [§4.1.3] The training details mention 'uniformly sample 16 frames per step.' Since the pipeline uses 60 pooled images per sample, it is unclear what the 16-frame sampling refers to — is it a batch-level subsample or a per-sample frame budget? Please clarify.
  3. [Table 1 vs. Table 6] Table 1 reports 'Key Frame only' F1 of 0.070, while Table 6 reports KeyFrame-only fine-tuned F1 of 0.553–0.563. Please clarify whether the Table 1 baseline includes subtitle text and the system prompt key-frame, or only the raw frame, since the comparison to pooled baselines depends on this.
  4. [§1, Figure 1] Figure 1 is described as an 'Abstract concept' but the figure itself appears to be a block diagram; consider a more descriptive caption and consistent terminology for 'pooling-only' vs. '1 fps pooled images'.
  5. [§2] The related-work section cites [8], [20], and [37] in the TVQA table but does not discuss them in the text; please either integrate or remove these citations from the table.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central mechanism has no fitted 'theory' parameters, but the pooling weights and blur parameters are hand-chosen and unreported. The main unstated premises are that pooled 1fps images preserve needed evidence and that a keyframe is not sufficient — the latter is contradicted by the paper's own Table 6.

free parameters (3)
  • λ (WAE recency exponent)
    Defined in Eq. 4; no value reported; chosen by hand to shape recency weighting and affects the pooled image content.
  • α (BBLF blend weight)
    Defined in Eq. 6; no value reported; controls the balance between the last frame and the blurred average.
  • σ (BBLF Gaussian blur radius)
    Defined in Eq. 6; no value reported; controls the spatial smoothing applied to the pooled image.
axioms (4)
  • domain assumption 1 fps pooled images retain enough intra-second motion/appearance information for the QA task
    Invoked in §3.1 and §3.6; undercut by the paper's own KeyFrame-only results (Table 6) that show a single frame matches the full pipeline.
  • domain assumption Subtitles are available and temporally aligned; questions are answered from video+subtitle interleaving
    Used in §3.2–3.3; many deployment settings do not have aligned subtitles, limiting generality.
  • domain assumption A key-frame at the exact video pause second is available for the prompt
    Used in §3.3 and in the KeyFrame-only ablation; availability of this exact frame is not guaranteed in real-world QA.
  • domain assumption Qwen2.5-VL's image encoder treats pooled/blurred images as equivalently informative to raw frames
    Assumed in §3.6 token-budget discussion; not directly verified, and the KeyFrame ablation suggests the encoder may not need the pooled temporal context.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiency." pith.science (2026). https://pith.science/paper/OT53LJ3Z

@misc{pith2026251001009,
  author       = {Pith},
  title        = {Pith review of: POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OT53LJ3Z}},
  note         = {Machine review of arXiv:2510.01009}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Long-video multimodal question answering requires structured reasoning over visual evidence and dialogue, but Large Vision-Language Models (LVLMs) are constrained by context-window and compute limits. We propose POVQA, which compresses each second into a temporally pooled image (1 fps pooled images) to maintain dense temporal coverage under a fixed token budget. We then train Qwen2.5-VL-7B with supervised fine-tuning (SFT) on rationale+answer targets, and optionally apply Direct Preference Optimization (DPO) for preference alignment. We introduce ReasonVQA as a pilot diagnostic dataset with 12 movies and 239 human-annotated QA+rationale triplets for controlled analysis of long-context multimodal reasoning under compression. On ReasonVQA, SFT improves the best pooled-only baseline from 0.212 to 0.550 F1, showing that pooled evidence plus rationale supervision provides the main performance gains in this setting. In zero-shot transfer, POVQA also reaches 64.7\% on TVQA after SFT+DPO. These results are preliminary: ReasonVQA is small, pooling can lose fine-grained temporal order, and DPO effects are not uniformly positive across settings. Code, dataset, and additional qualitative evaluations are available at \href{https://povqa.github.io}{https://povqa.github.io}.

Figures

Figures reproduced from arXiv: 2510.01009 by Ankit Ghimire, Ashim Dahal, Nick Rahimi, Saydul Akbar Murad.

Figure 1
Figure 1. Figure 1: Abstract concept of inference on POVQA pipeline on evaluation set of ReasonVQA [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the training process of POVQA on ReasonVQA dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative analysis on a random sample of TVQA. Frames sub-sampled to fit on single page. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative analysis on a random sample on ReasonVQA. Frames sub-sampled to fit on single page. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 14 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 7

    Pith/arXiv arXiv 2023

  2. [2]

    Audio visual scene-aware dialog (avsd) challenge at dstc7.arXiv preprint arXiv:1806.00525, 2018

    Huda Alamri, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Jue Wang, Irfan Essa, Dhruv Batra, Devi Parikh, Anoop Cherian, Tim K Marks, et al. Audio visual scene-aware dialog (avsd) challenge at dstc7.arXiv preprint arXiv:1806.00525, 2018. 2

    Pith/arXiv arXiv 2018

  3. [3]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

  4. [4]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Luˇci´c, and Cordelia Schmid. Vivit: A video vision transformer. InProceedings of the IEEE/CVF international conference on computer vision, pages 6836–6846, 2021. 2

    2021

  5. [5]

    Goldfish: Vision- language understanding of arbitrarily long videos

    Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Mingchen Zhuge, Jian Ding, Deyao Zhu, Jürgen Schmidhuber, and Mohamed Elhoseiny. Goldfish: Vision- language understanding of arbitrarily long videos. InEu- ropean Conference on Computer Vision, pages 251–267. Springer, 2024. 7

    2024

  6. [6]

    Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Machine Learning, pages 813–824

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Machine Learning, pages 813–824. PMLR, 2021. 2

    2021

  7. [7]

    Pali-x: On scaling up a multilingual vision and language model.arXiv preprint arXiv:2305.18565, 2023

    Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebas- tian Goodman, Xiao Wang, Yi Tay, et al. Pali-x: On scaling up a multilingual vision and language model.arXiv preprint arXiv:2305.18565, 2023. 1

    Pith/arXiv arXiv 2023

  8. [8]

    Vindlu: A recipe for ef- fective video-and-language pretraining

    Feng Cheng, Xizi Wang, Jie Lei, David Crandall, Mohit Bansal, and Gedas Bertasius. Vindlu: A recipe for ef- fective video-and-language pretraining. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10739–10750, 2023. 7

    2023

  9. [9]

    Transformer-xl: At- tentive language models beyond a fixed-length context.arXiv preprint arXiv:1901.02860, 2019

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer-xl: At- tentive language models beyond a fixed-length context.arXiv preprint arXiv:1901.02860, 2019. 2

    Pith/arXiv arXiv 1901

  10. [10]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088– 10115, 2023. 2

    2023

  11. [11]

    Knowit vqa: Answering knowledge-based questions about videos

    Noa Garcia, Mayu Otani, Chenhui Chu, and Yuta Nakashima. Knowit vqa: Answering knowledge-based questions about videos. InProceedings of the AAAI conference on artificial intelligence, pages 10826–10834, 2020. 2

    2020

  12. [12]

    Agqa 2.0: An updated benchmark for compositional spatio-temporal reasoning.arXiv preprint arXiv:2204.06105, 2022

    Madeleine Grunde-McLaughlin, Ranjay Krishna, and Ma- neesh Agrawala. Agqa 2.0: An updated benchmark for compositional spatio-temporal reasoning.arXiv preprint arXiv:2204.06105, 2022. 2

    Pith/arXiv arXiv 2022

  13. [13]

    Reveal: Retrieval-augmented visual-language pre-training with multi-source multimodal knowledge mem- ory

    Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A Ross, and Alireza Fathi. Reveal: Retrieval-augmented visual-language pre-training with multi-source multimodal knowledge mem- ory. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23369–23379, 2023. 2

    2023

  14. [14]

    Ku, Qian Liu, and Wenhu Chen

    Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max W.F. Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi- image instruction tuning.Trans. Mach. Learn. Res., 2024,

    2024

  15. [15]

    Visual question answer- ing: Datasets, algorithms, and future challenges.Computer Vision and Image Understanding, 163:3–20, 2017

    Kushal Kafle and Christopher Kanan. Visual question answer- ing: Datasets, algorithms, and future challenges.Computer Vision and Image Understanding, 163:3–20, 2017. 2

    2017

  16. [16]

    An image grid can be worth a video: Zero-shot video question answering using a vlm.IEEE Access, 2024

    Wonkyun Kim, Changin Choi, Wonseok Lee, and Wonjong Rhee. An image grid can be worth a video: Zero-shot video question answering using a vlm.IEEE Access, 2024. 7

    2024

  17. [17]

    Tvqa: Localized, compositional video question answering.arXiv preprint arXiv:1809.01696, 2018

    Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. Tvqa: Localized, compositional video question answering.arXiv preprint arXiv:1809.01696, 2018. 1, 2, 5

    Pith/arXiv arXiv 2018

  18. [18]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models.arXiv preprint arXiv:2407.07895, 2024

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models.arXiv preprint arXiv:2407.07895, 2024. 2

    Pith/arXiv arXiv 2024

  19. [19]

    Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR,

  20. [20]

    Mvbench: A comprehensive multi-modal video understand- ing benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understand- ing benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195– 22206, 2024. 7

    2024

  21. [21]

    HERO: Hierarchical encoder for Video+Language omni-representation pre-training

    Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, and Jingjing Liu. HERO: Hierarchical encoder for Video+Language omni-representation pre-training. InPro- ceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2046–2065, Online, 2020. Association for Computational Linguistics. 7

    2020

  22. [22]

    Video-llava: Learning united visual representation by alignment before projection.EMNLP, 2023

    Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection.EMNLP, 2023. 1

    2023

  23. [23]

    Common- sense video question answering through video-grounded en- tailment tree reasoning

    Huabin Liu, Filip Ilievski, and Cees GM Snoek. Common- sense video question answering through video-grounded en- tailment tree reasoning. InProceedings of the Computer Vi- sion and Pattern Recognition Conference, pages 3262–3271,

  24. [24]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fa- had Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. InPro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 2024. 1

    2024

  25. [25]

    Question-instructed visual descriptions for zero-shot video answering

    David Mogrovejo and Thamar Solorio. Question-instructed visual descriptions for zero-shot video answering. InFindings of the Association for Computational Linguistics ACL 2024, pages 9329–9339, 2024. 7 9

    2024

  26. [26]

    Streaming long video un- derstanding with large language models.Advances in Neural Information Processing Systems, 37:119336–119360, 2024

    Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video un- derstanding with large language models.Advances in Neural Information Processing Systems, 37:119336–119360, 2024. 2

    2024

  27. [27]

    Direct prefer- ence optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741, 2023

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct prefer- ence optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741, 2023. 2

    2023

  28. [28]

    Timechat: A time-sensitive multimodal large lan- guage model for long video understanding

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large lan- guage model for long video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14313–14323, 2024. 2

    2024

  29. [29]

    Action recognition using visual attention.arXiv preprint arXiv:1511.04119, 2015

    Shikhar Sharma, Ryan Kiros, and Ruslan Salakhutdinov. Action recognition using visual attention.arXiv preprint arXiv:1511.04119, 2015. 2

    Pith/arXiv arXiv 2015

  30. [30]

    Moviechat: From dense token to sparse memory for long video understanding

    Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18221–18232, 2024. 2

    2024

  31. [31]

    Modularized self-reflected video reasoner for multimodal llm with application to video ques- tion answering

    Zihan Song, Xin Wang, Zi Qian, Hong Chen, Longtao Huang, Hui Xue, and Wenwu Zhu. Modularized self-reflected video reasoner for multimodal llm with application to video ques- tion answering. InForty-second International Conference on Machine Learning. 2

  32. [32]

    Movieqa: Understanding stories in movies through question-answering

    Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Anto- nio Torralba, Raquel Urtasun, and Sanja Fidler. Movieqa: Understanding stories in movies through question-answering. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4631–4640, 2016. 1

    2016

  33. [33]

    Mm- interleaved: Interleaved image-text generative modeling via multi-modal feature synchronizer.ArXiv, abs/2401.10208,

    Changyao Tian, Xizhou Zhu, Yuwen Xiong, Weiyun Wang, Zhe Chen, Wenhai Wang, Yuntao Chen, Lewei Lu, Tong Lu, Jie Zhou, Hongsheng Li, Yu Qiao, and Jifeng Dai. Mm- interleaved: Interleaved image-text generative modeling via multi-modal feature synchronizer.ArXiv, abs/2401.10208,

    Pith/arXiv arXiv

  34. [34]

    Video- mae: Masked autoencoders are data-efficient learners for self-supervised video pre-training.Advances in neural infor- mation processing systems, 35:10078–10093, 2022

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Video- mae: Masked autoencoders are data-efficient learners for self-supervised video pre-training.Advances in neural infor- mation processing systems, 35:10078–10093, 2022. 2

    2022

  35. [35]

    Fastvlm: Efficient vision encoding for vision language models

    Pavan Kumar Anasosalu Vasu, Fartash Faghri, Chun-Liang Li, Cem Koc, Nate True, Albert Antony, Gokula Santhanam, James Gabriel, Peter Grasch, Oncel Tuzel, et al. Fastvlm: Efficient vision encoding for vision language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19769–19780, 2025. 2

    2025

  36. [36]

    Temporal segment networks: Towards good practices for deep action recognition.CoRR, abs/1608.00859, 2016

    Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks: Towards good practices for deep action recognition.CoRR, abs/1608.00859, 2016. 2

    Pith/arXiv arXiv 2016

  37. [37]

    Vila: Efficient video- language alignment for video question answering

    Xijun Wang, Junbang Liang, Chun-Kai Wang, Kenan Deng, Yu Lou, Ming C Lin, and Shan Yang. Vila: Efficient video- language alignment for video question answering. InEu- ropean Conference on Computer Vision, pages 186–204. Springer, 2024. 7

    2024

  38. [38]

    Internvideo: General video foundation models via generative and discriminative learning.arXiv preprint arXiv:2212.03191,

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning.arXiv preprint arXiv:2212.03191,

    Pith/arXiv arXiv

  39. [39]

    Longvlm: Efficient long video understand- ing via large language models

    Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. Longvlm: Efficient long video understand- ing via large language models. InEuropean Conference on Computer Vision, pages 453–470. Springer, 2024. 2

    2024

  40. [40]

    Star: A benchmark for situated reasoning in real-world videos.arXiv preprint arXiv:2405.09711, 2024

    Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos.arXiv preprint arXiv:2405.09711, 2024. 2

    Pith/arXiv arXiv 2024

  41. [41]

    Adaframe: Adaptive frame selection for fast video recognition

    Zuxuan Wu, Caiming Xiong, Chih-Yao Ma, Richard Socher, and Larry S Davis. Adaframe: Adaptive frame selection for fast video recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1278–1287, 2019. 2

    2019

  42. [42]

    Next-qa: Next phase of question-answering to explaining tem- poral actions

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question-answering to explaining tem- poral actions. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9777–9786,

  43. [43]

    Zero-shot video question answering via frozen bidirectional language models.Advances in Neural Information Processing Systems, 35:124–141, 2022

    Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. Zero-shot video question answering via frozen bidirectional language models.Advances in Neural Information Processing Systems, 35:124–141, 2022. 7

    2022

  44. [44]

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, et al. Qwen2. 5-1m technical report. arXiv preprint arXiv:2501.15383, 2025. 1

    Pith/arXiv arXiv 2025

  45. [45]

    Enhancing long video ques- tion answering with scene-localized frame grouping.arXiv preprint arXiv:2508.03009, 2025

    Xuyi Yang, Wenhao Zhang, Hongbo Jin, Lin Liu, Hongbo Xu, Yongwei Nie, Fei Yu, and Fei Ma. Enhancing long video ques- tion answering with scene-localized frame grouping.arXiv preprint arXiv:2508.03009, 2025. 2

    Pith/arXiv arXiv 2025

  46. [46]

    Self-chained image-language model for video localization and question answering.Advances in Neural Information Processing Systems, 36:76749–76771, 2023

    Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image-language model for video localization and question answering.Advances in Neural Information Processing Systems, 36:76749–76771, 2023. 7 10

    2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.