Pith. sign in

REVIEW 6 major objections 8 minor 46 references

TextVidBench: A Benchmark for Long Video Scene Text Understanding

T0 review · 6 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TextVidBench, the first benchmark for scene-text question answering on videos longer than three minutes, finds that current multimodal models answer at best 16.0% of its text-retrieval questions.

desk verdict A valuable long-video scene-text benchmark whose central claim is conditional on an unverified annotation pipeline; deserves peer review and careful revision. read the letter →

arxiv 2506.04983 v1 pith:EI5OIKLU submitted 2025-06-05 cs.CV

Yangyang Zhong , Ji Qi , Yuan Yao , Pengxin Luo , Yunfeng Yan , Donglian Qi , Zhiyuan Liu , Tat-Seng Chua This is my paper
classification cs.CV
keywords long-videoscenetextunderstandingvideovisualquestionansweringneedle-in-haystacktemporalgroundingdynamicscaptioningrotarypositionembeddinginterpolationmultimodallargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that existing video text question-answering benchmarks, with clips averaging under ten seconds, reward models that can sample a few frames and therefore miss the long-horizon reasoning real applications need. To close the gap it builds TextVidBench, a benchmark of about 30 hours of video across nine domains with an average length of 2,306 seconds, and over 5,000 question-answer pairs organized into three tasks: retrieving a text-bearing frame, localizing that frame in time, and describing how on-screen text evolves. The central claim is that current multimodal large language models largely fail these tasks—the best average accuracy on Text Needle-in-Haystack is 16.0%, temporal grounding stays below 31% even at 120 seconds of tolerance, and Text Dynamics Captioning scores stay below 5 out of 10. The paper also claims that lightweight fine-tuning with inflatable temporal rotary position embeddings, non-uniform position interpolation, and explicit time prompts improves a strong base model on all three tasks.

What carries the argument

The load-bearing mechanism is the three-stage evaluation framework paired with three model-side additions. IT-RoPE (Inflated Temporal Rotary Position Embedding) modifies the standard 1D rotary position embedding so that all tokens belonging to the same video frame share one temporal position index while retaining an intra-frame flattened index; attention scores then depend on both which frame a token came from and where it sits inside that frame, and the scheme degenerates to ordinary RoPE for text tokens. NUPI (Non-uniform Position Interpolation) extends IT-RoPE by searching for non-uniform frequency scaling coefficients so the model can handle longer visual token sequences. The time prompt injects the total video duration, the number of sampled frames, and each frame's timestamp into the model input, which the ablation shows is almost a prerequisite for temporal grounding.

What would settle it

Re-annotate a random sample of TextVidBench's question-answer pairs with several independent human annotators and measure agreement; if agreement is low, or if errors cluster in the GPT-4o-generated questions, the reported 16.0% top accuracy and the relative model rankings would need to be revised.

Watch

Extended reading notes

Core claim

TextVidBench is the first benchmark designed specifically for long-video scene-text question answering, with videos averaging 2,306 seconds—roughly 100 times longer than earlier ViteVQA datasets. It covers nine domains (driving, egocentric, entertainment, game, knowledge, life record, sports, talk shows, and video news) and its three-stage evaluation separates finding a text-bearing frame (Text Needle-in-Haystack), predicting the frame's timestamp (Text Temporal Grounding), and summarizing how text changes over time (Text Dynamics Captioning). Across these tasks the paper reports that existing multimodal large language models perform poorly: the best model, the authors' improved version of MiniCPM-V 2.6, reaches 16.0% average accuracy on Text Needle-in-Haystack, 15.05% average accuracy on temporal grounding at 120-second tolerance, and 4.59 out of 10 on Text Dynamics Captioning. The authors' method-level claim is that inflating 1D rotary position embeddings with a temporal dimension (IT-RoPE), extending the context with non-uniform position interpolation, and injecting frame timestamps through a time prompt each contribute measurable gains, with the time prompt being essential for temporal localization.

Load-bearing premise

The benchmark's ground truth is produced by an automatic pipeline—a model selects text-rich frames, GPT-4o writes the questions, and humans correct only about 20% of them—so the reported scores inherit any systematic error in those generated pairs.

Editorial extensions

If this is right

  • Short-video text benchmarks overstate model readiness: performance on long-video text retrieval declines steadily as video length grows, so a model that looks strong on M4-ViteVQA can still be near chance on 20-minute videos.
  • Explicit timestamp information is the difference between zero and nonzero temporal grounding: without the time prompt, localization accuracy collapses to near zero, even for the improved model.
  • Tracking dynamic text such as game scores remains an open problem: every tested 7B-scale model, including the proposed one, scores below 5 out of 10 on describing how scores and in-game currency change.
  • A base model can be upgraded for long-video text tasks without a full architecture replacement: the paper's lightweight fine-tuning with IT-RoPE, NUPI, and time prompting improves MiniCPM-V 2.6 on all three benchmark tasks while preserving short-video performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark's difficulty profile is representative, the bottleneck for long-video text understanding may be temporal indexing rather than text recognition: a model that reads text well but cannot timestamp frames will fail grounding, so future training should include explicit timestamp supervision.
  • The annotation pipeline could be validated by measuring inter-annotator agreement on a held-out subset, since only about 20% of pairs were human-revised; low agreement or systematic GPT-4o errors would change the reported scores.
  • The three-task design could transfer to other sparse-text long-video domains, such as surveillance or lecture archives, offering a cheap way to test whether temporal-text reasoning learned on sports and news generalizes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 8 minor

Summary. This paper introduces TextVidBench, a benchmark for long-video scene text understanding. The benchmark contains over 5,000 QA pairs from 9 video categories totaling about 23 hours, with an average video length of 2,306 seconds. It defines three tasks: Text Needle-in-Haystack (frame-level text retrieval and QA), Text Temporal Grounding (predicting timestamps within 30/60/120s tolerance), and Text Dynamics Captioning (describing changes in text, scored by GPT-4 from 0 to 10). The authors also propose an improved MLLM built on MiniCPM-V 2.6, adding IT-RoPE temporal position encoding, NUPI non-uniform position interpolation, a Time Prompt, and fine-tuning on 50k video-instruction pairs. Experiments show the benchmark is challenging: the best average accuracy on Text Needle-in-Haystack is 16.0%, temporal grounding accuracies are low (e.g., 15.05% at 120s for the proposed model), and all models score below 5 on Text Dynamics Captioning. The proposed model outperforms its base model on all three tasks and achieves gains on the public M4-ViteVQA benchmark.

Significance. If the annotations and evaluation protocols are sound, TextVidBench is a valuable contribution: it is the first benchmark targeting long-video scene text, and its results show a clear capability gap in current MLLMs. The proposed IT-RoPE and Time Prompt are simple and potentially generalizable, and the paper includes a broad baseline evaluation across several strong models. However, the reliability of the benchmark's ground truth is not yet established, a random baseline is missing for the temporal grounding task, and the GPT-4 scoring of captions is unvalidated. These issues must be addressed before the benchmark's scores can be interpreted as reliable measures of model performance.

major comments (6)
  1. [Section 3.1] The benchmark ground truth is generated semi-automatically: MiniCPM-V filters text-rich frames (step 2), GPT-4o writes questions and answers (step 4), and human reviewers correct approximately 20% of pairs (step 5). No inter-annotator agreement, annotation guidelines, or independent audit are reported. Because all scores in Tables 2-5 are computed against these labels, systematic GPT-4o errors (e.g., misreading scene text) would bias every model comparison in the same direction, so the claim that Ours outperforms MiniCPM-V 2.6 could be an artifact of label noise. Please report the number of annotators, their agreement statistics, and a category-wise audit of the 20% corrections, and make the annotation protocol available.
  2. [Section 3.2, Table 4] The Text Temporal Grounding task lacks a random baseline. For an average video length of 2,306 seconds, a uniform random timestamp is within ±30/60/120s of the ground truth with probability about 2.6/5.2/10.4%. Many reported values are at or below this chance level (e.g., LongVA 120s accuracies of 4.53%, 2.71%, and 0.0%; LongVILA 120s accuracies of 4.27%, 1.45%, and 0.0%). The proposed model's 120s average of 15.05% is modestly above chance. Please add random and constant-prediction baselines to Table 4 and discuss what the task measures in their light.
  3. [Section 3.2, Text Dynamics Captioning] The GPT-4 evaluation protocol for captioning is not validated. The paper states that GPT-4 assigns a score from 0 to 10 by comparing model output with standard answers, but gives no scoring prompt, no rubric, and no evidence that GPT-4 scores agree with human judgments. Without this, the comparisons in Table 5 (e.g., Ours 4.59 vs. MiniCPM-V 3.44) and the claim that 'none achieve an average score above 5' are not interpretable. Please provide the exact scoring prompt and the results of a human-GPT-4 correlation study on a sample of outputs.
  4. [Appendix F.1 (EliteSet)] The proposed model is fine-tuned on EliteSet, which is described as being created 'based on the video data collected in VideoTextCap' and using 'the same data production process as the benchmark.' VideoTextCap itself is built from YouTube videos in the same nine categories used by TextVidBench. If any benchmark videos or near-duplicates appear in the training data, the reported improvements (Tables 2-5) would be inflated. Please clarify whether the benchmark videos were excluded from VideoTextCap/EliteSet and, ideally, evaluate the model on a fully disjoint set of long videos to show the gains are not due to distributional overlap.
  5. [Section 4.4, Appendix E] The NUPI scaling coefficients λ_i are found by evolutionary search, but the paper does not state which data were used for the search. If the search was performed on TextVidBench or on a validation subset derived from it, then the NUPI gains in Table 7 are not a fair comparison with baselines, because they reflect test-set adaptation. Please state the optimization objective and data used to select λ_i, and report experiments with coefficients chosen on a completely separate validation set.
  6. [Section 5.2.1, Appendix F.1] The validation on M4-ViteVQA uses the proposed model fine-tuned on VideoTextQA, which includes 'public datasets on video text comprehension, including M4-ViteVQA, RoadTextVQA, and NewsVideoQA.' The paper does not state whether the official training split or the full datasets (including the test split) were used. If the M4-ViteVQA test set was included in training, the reported accuracy/ANLS gains over MiniCPM-V 2.6 are invalid. Please specify the exact train/test protocol used for these public datasets.
minor comments (8)
  1. [Throughout] The text contains several typos and formatting errors: 'enrichs' should be 'enriches', 'n recent years' should be 'In recent years', and 'LenthY' in Eq. (9) should be a product symbol.
  2. [Section 3.2] The accuracy metric (ACC) for Text Needle-in-Haystack is not defined; please state the normalization and matching rule used to convert model outputs to binary correct/incorrect.
  3. [Table 4] The rows for Ours are ambiguous because 'Ours' and 'Ours(acc)' are not clearly separated into the 30s, 60s, and 120s blocks; please label each row explicitly.
  4. [Appendix D] In the IT-RoPE description, the rotation angle notation is inconsistent: the text says 'for even values of i, the corresponding rotation angle is given by kθi', but k is not defined; please clarify the indexing.
  5. [Section 3.3 / Abstract] The abstract says the benchmark contains over 5,000 QA pairs, while Appendix C reports 2,190 pairs for Text Needle and 600 for Text Change; the total count should be reconciled and stated transparently.
  6. [Section 2.3 / Section 3.3] The claim of being the 'first' long-video text benchmark would be stronger with a systematic comparison of existing video-text datasets' duration ranges and task designs, rather than only qualitative statements.
  7. [Table 1] The caption lists 'Random' and 'Human' baselines but does not explain how they were constructed; please provide the reference or a footnote describing these baselines.
  8. [Limitations] The Limitations section focuses only on computational resources and does not mention the potential annotation errors or the distributional overlap between training and evaluation data; please expand it to discuss these limitations.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the benchmark is independently anchored by M4-ViteVQA, and remaining overlaps are annotation-quality and domain-match risks, not equations reducing to inputs.

full rationale

The paper's derivation chain is not circular. The proposed method is validated on M4-ViteVQA, an external benchmark with 3,183 test question-answer pairs, and its three task-specific improvements (IT-RoPE, NUPI, time prompting) are then evaluated on TextVidBench, whose labels are fixed independently of the model's predictions. The closest overlaps are: (a) MiniCPM-V, a base model co-authored by one of the present authors, is used both as the initialization for the proposed model and as the frame-text filter in benchmark construction; (b) EliteSet training data is produced with the same GPT-4o pipeline as the benchmark; and (c) NUPI's scaling coefficients come from an evolutionary search whose objective is not disclosed. These are real threats to external validity, including label noise, selection bias, distributional overlap, and hyperparameter reporting transparency. However, none exhibits the required reduction: no equation equates the reported benchmark scores to the training or filtering inputs, no fitted parameter is renamed as a prediction, and no self-citation is used to forbid alternatives or establish a uniqueness result. The benchmark's GPT-4o-generated questions with only approximately 20% human correction and no reported inter-annotator agreement are an annotation audit gap rather than a circularity. The score reflects one minor self-citation (MiniCPM-V) that is not load-bearing to the central comparative claim.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The benchmark claim rests on annotation quality and evaluation protocol assumptions rather than on new physical entities. The only fitted parameters on the method side are the NUPI scaling coefficients, whose search procedure is not fully disclosed.

free parameters (1)
  • NUPI scaling coefficients lambda_i = not reported
    Found through evolutionary search over IT-RoPE rotation frequencies in Appendix E. The paper does not disclose which data or objective guided the search, so the gains in Table 7 may partially reflect selection on evaluation-like criteria rather than a parameter-free prediction.
assumptions (3)
  • domain assumption GPT-4o-generated question-answer pairs are accurate after human correction of approximately 20% of the data.
    Section 3.1 steps 4 and 5. No inter-annotator agreement, annotation error rate, or independent audit is reported, so the benchmark ground truth quality is assumed.
  • domain assumption Uniformly sampling 64 frames preserves enough temporal information for temporal grounding in 2306-second videos.
    Appendix H.2 uses 64 frames for all temporal grounding experiments. No sensitivity analysis is provided, and the sampling density may affect the difficulty of the task.
  • domain assumption GPT-4 as an automated judge gives a meaningful score for Text Dynamics Captioning.
    Section 3.2 assigns scores from 0 to 10 via GPT-4 without citing a prior validation of this protocol or reporting correlation with human judgments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TextVidBench: A Benchmark for Long Video Scene Text Understanding." pith.science (2026). https://pith.science/paper/EI5OIKLU

@misc{pith2026250604983,
  author       = {Pith},
  title        = {Pith review of: TextVidBench: A Benchmark for Long Video Scene Text Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EI5OIKLU}},
  note         = {Machine review of arXiv:2506.04983}
}
read the original abstract

Despite recent progress on the short-video Text-Visual Question Answering (ViteVQA) task - largely driven by benchmarks such as M4-ViteVQA - existing datasets still suffer from limited video duration and narrow evaluation scopes, making it difficult to adequately assess the growing capabilities of powerful multimodal large language models (MLLMs). To address these limitations, we introduce TextVidBench, the first benchmark specifically designed for long-video text question answering (>3 minutes). TextVidBench makes three key contributions: 1) Cross-domain long-video coverage: Spanning 9 categories (e.g., news, sports, gaming), with an average video length of 2306 seconds, enabling more realistic evaluation of long-video understanding. 2) A three-stage evaluation framework: "Text Needle-in-Haystack -> Temporal Grounding -> Text Dynamics Captioning". 3) High-quality fine-grained annotations: Containing over 5,000 question-answer pairs with detailed semantic labeling. Furthermore, we propose an efficient paradigm for improving large models through: (i) introducing the IT-Rope mechanism and temporal prompt engineering to enhance temporal perception, (ii) adopting non-uniform positional encoding to better handle long video sequences, and (iii) applying lightweight fine-tuning on video-text data. Extensive experiments on multiple public datasets as well as TextVidBench demonstrate that our new benchmark presents significant challenges to existing models, while our proposed method offers valuable insights into improving long-video scene text understanding capabilities.

Figures

Figures reproduced from arXiv: 2506.04983 by the authors.

Figure 1
Figure 1. The figure illustrates three distinct task set [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Semi-automatic Annotation Process [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. The overall architecture of our model. B Model Architecture The model comprises three fundamental components: a visual encoder, a visual projector (MLP), and a large language model. The overall architecture is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Statistical data on benchmark. • Visual Encoder: We employ the pre-trained visual encoder from MiniCPM-V2.6 (Yao et al., 2024), which is based on the SigLIP SoViT-400m/14 (Zhai et al., 2023) architecture. • Visual Projection Layer (MLP): This layer utilizes the pre-tra…
Figure 5
Figure 5. Figure 5: Visualization of attention scores across different transformer layers for various channel groups, where [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Visualization of model output [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Visualization of model output [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Visualization of model output [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Visualization of model output [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 13 canonical work pages

  1. [1]

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pages 2425--2433

    2015

  2. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    arXiv 2025

  4. [4]

    Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Mar c al Rusinol, Ernest Valveny, CV Jawahar, and Dimosthenis Karatzas. 2019. Scene text visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4291--4301

    2019

  5. [5]

    Houlun Chen, Xin Wang, Hong Chen, Zeyang Zhang, Wei Feng, Bin Huang, Jia Jia, and Wenwu Zhu. 2024. Verified: A video corpus moment retrieval benchmark for fine-grained video understanding. arXiv preprint arXiv:2410.08593

    arXiv 2024

  6. [6]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171--4186

    2019

  7. [7]

    Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fan Yang, and Mao Yang. 2024. Longrope: Extending llm context window beyond 2 million tokens. arXiv preprint arXiv:2402.13753

    arXiv 2024

  8. [8]

    Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440--1448

    work page 2015

Show all 46 references
  1. [9]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904--6913

    2017

  2. [10]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun. 2020. Single path one-shot neural architecture search with uniform sampling. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVI ...

    2020

  4. [12]

    Pranay Gupta and Manish Gupta. 2022. Newskvqa: Knowledge-aware news video question answering. In Pacific-asia conference on knowledge discovery and data mining, pages 3--15. Springer

    2022

  5. [13]

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3...

    2018

  6. [14]

    Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. 2018. Localizing moments in video with temporal language. In Empirical Methods in Natural Language Processing (EMNLP)

    2018

  7. [15]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . Preprint, arXiv:2106.09685

    2021 arXiv

  8. [16]

    Ronghang Hu, Amanpreet Singh, Trevor Darrell, and Marcus Rohrbach. 2020. Iterative answer prediction with pointer-augmented multimodal transformers for textvqa. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9992--10002

    2020

  9. [17]

    Soumya Jahagirdar, Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2022. Watching the news: Towards videoqa models that can read

    2022

  10. [18]

    Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. 2017. Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2758--2766

    2017

  11. [19]

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, and 1 others. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal...

    2017

  12. [20]

    Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. 2018. Tvqa: Localized, compositional video question answering. arXiv preprint arXiv:1809.01696

    2018 arXiv

  13. [21]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  14. [22]

    Ilya Loshchilov and Frank Hutter. 2017. https://arxiv.org/abs/1608.03983 Sgdr: Stochastic gradient descent with warm restarts . Preprint, arXiv:1608.03983

    2017 arXiv

  15. [23]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195--3204

    2019

  16. [24]

    Jonghwan Mun, Paul Hongsuck Seo, Ilchae Jung, and Bohyung Han. 2017. Marioqa: Answering questions by watching gameplay videos. In Proceedings of the IEEE International Conference on Computer Vision, pages 2867--2875

    2017

  17. [25]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, , and 1 others. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  18. [26]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063

    2024

  19. [27]

    Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2016. Movieqa: Understanding stories in movies through question-answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631--4640

    2016

  20. [28]

    George Tom, Minesh Mathew, Sergi Garcia, Dimosthenis Karatzas, and C. V. Jawahar. 2023. https://arxiv.org/abs/2307.03948 Reading between the lanes: Text videoqa on the road . Preprint, arXiv:2307.03948

    2023 arXiv

  21. [29]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  22. [30]

    Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. 2016. Coco-text: Dataset and benchmark for text detection and recognition in natural images. arXiv preprint arXiv:1601.07140

    2016 arXiv

  23. [31]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, and 1 others. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  24. [32]

    Qi Wu, Damien Teney, Peng Wang, Chunhua Shen, Anthony Dick, and Anton Van Den Hengel. 2017. Visual question answering: A survey of methods and datasets. Computer Vision and Image Understanding, 163:21--40

    2017

  25. [33]

    Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, and Yueting Zhuang. 2017. Video question answering via gradually refined attention over appearance and motion. In Proceedings of the 25th ACM international conference on Multimedia, pages 1645--1653

    2017

  26. [34]

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. 2016. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5288--5296

    2016

  27. [35]

    Li Xu, He Huang, and Jun Liu. 2021. Sutd-trafficqa: A question answering benchmark and an efficient network for video reasoning over traffic events. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9878--9888

    2021

  28. [36]

    Fuzhao Xue, Yukang Chen, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, and 1 others. 2024. Longvila: Scaling long-context visual language models for long videos. arXiv preprint arXiv:2408.10188

    2024 arXiv

  29. [37]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, and 1 others. 2024. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800

    2024 arXiv

  30. [38]

    Yunan Ye, Zhou Zhao, Yimeng Li, Long Chen, Jun Xiao, and Yueting Zhuang. 2017. Video question answering via attribute-augmented attention network learning. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pages...

    2017

  31. [39]

    Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. 2019. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 9127--9134

    2019

  32. [40]

    Kuo-Hao Zeng, Tseng-Hung Chen, Ching-Yao Chuang, Yuan-Hong Liao, Juan Carlos Niebles, and Min Sun. 2017. Leveraging video descriptions to learn video question answering. In Proceedings of the AAAI conference on artificial intelligence, volume 31

    2017

  33. [41]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. https://arxiv.org/abs/2303.15343 Sigmoid loss for language image pre-training . Preprint, arXiv:2303.15343

    2023 arXiv

  34. [42]

    Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. 2024. Long context transfer from language to vision. arXiv preprint arXiv:2406.16852

    2024 arXiv

  35. [43]

    Minyi Zhao, Bingjia Li, Jie Wang, Wanqing Li, Wenjing Zhou, Lan Zhang, Shijie Xuyang, Zhihang Yu, Xinkun Yu, Guangze Li, and 1 others. 2022. Towards video text visual question answering: Benchmark and baseline. Advances in Neural Information Processing Systems, 35:35549--35562

    2022

  36. [44]

    Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. 2016. Visual7w: Grounded question answering in images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4995--5004

    2016

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.