Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

VideoVista-CulturalLingo: 360$^\circ$ Horizons-Bridging Cultures, Languages, and Domains in Video Comprehension

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new benchmark shows AI video models underperform on Chinese-centric questions.

desk verdict A genuinely reusable bilingual/tricultural video benchmark, but the headline model comparisons rest on an acknowledged protocol confound that needs a control experiment. read the letter →

arxiv 2504.17821 v2 pith:N7U6OH3X submitted 2025-04-23 cs.CV cs.CL

Xinyu Chen , Yunxin Li , Haoyuan Shi , Baotian Hu , Wenhan Luo , Yaowei Wang , Min Zhang This is my paper · ORCID
classification cs.CVcs.CL
keywords videocomprehensionbenchmarkculturalbiasbilingualQAmultimodallargelanguagemodelseventlocalizationChinesecultureevaluationscientificreasoningopen-source
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents VideoVista-CulturalLingo, a video question-answering benchmark intended to be the first that spans cultures, languages, and domains at once. It contains 1,389 videos and 3,134 multiple-choice questions, about half in Chinese and half in English, drawn from everyday and scientific videos and organized into fourteen tasks. The benchmark is built by having large models draft questions, then having human annotators score, keep, and refine them; the pipeline removes about 60 percent of candidate items. Evaluating 24 open-source and proprietary multimodal models, the paper finds that models answer Western-centric questions better than Chinese-centric ones, that open-source models reach only 45.2 percent on event localization, and that open-source models lag most on mathematics. If the benchmark is a fair instrument, it gives the field a reusable way to measure cultural and temporal video understanding rather than English-only, Western-only video understanding.

What carries the argument

The load-bearing object is the benchmark itself, organized as 3,134 multiple-choice QA pairs over 2,052 clips from 1,389 videos, grouped into four task families, Event, Object, Culture, and Science, with fourteen subtasks. The mechanism that carries the argument is the three-stage annotation pipeline: video preprocessing segments clips and cleans transcripts; automatic annotation uses a vision-language model for event and object labeling, text language models for question generation, and retrieval of Wikipedia background for cultural questions; and a human-check stage scores each item for correctness, task-type relevance, and video relevance, keeping only top-scoring items and manually correcting borderline ones. Culture questions are independently checked by two annotators, one native to the relevant culture. This pipeline is what converts raw videos into gold-standard QA pairs, so the entire evaluation rests on it.

What would settle it

Run all evaluated models on the same 3,134 questions with no video input, only the question text and possibly transcripts; if text-only accuracy approaches the reported video-input accuracy, the benchmark would not be measuring video comprehension, while a small gap would show many questions are answerable without watching the video.

Watch

Extended reading notes

Core claim

The paper's central claim is that current video models carry a measurable cultural and linguistic bias, and that a benchmark built from Western- and Chinese-sourced videos can expose it. On its 3,134 bilingual QA pairs, the strongest proprietary model reaches 76.3 percent overall accuracy while the strongest open-source video model reaches 61.3 percent. The gap is not uniform: on Chinese-culture questions the best open-source video model scores 65.8 percent, below its performance on European and American culture, and performance falls further on questions tied to Chinese history and cuisine. Across all models, event localization, locating when an event happens in a video, is the weakest task, with the best open-source score at 45.2 percent, while in science the leading open-source models come within a few points of proprietary models on physics, chemistry, and computer science but trail by about ten points on mathematics. The paper reads these results as evidence that training-data and benchmark-coverage biases still shape what video models can understand.

Load-bearing premise

The benchmark assumes that its machine-generated, human-refined questions are correct gold standards that genuinely require watching the video to answer, and that videos collected from the chosen platforms fairly represent the cultures they stand for.

Editorial extensions

If this is right

  • Model rankings on this benchmark split by culture, so the Chinese-English divide and the Chinese-culture deficit should be reported as a standard evaluation axis in future video-model comparisons.
  • Event localization is the clearest unsolved capability for open-source video models; the 45.2 percent ceiling means claims of open-source parity with proprietary systems should be conditioned on the task.
  • Mathematics, not general science, is where open-source video models fall furthest behind, pointing to video math reasoning as a distinct weakness.
  • All tested models, including the strongest proprietary one, lose accuracy as videos get longer, so long-video understanding remains a shared limitation.
  • Because the dataset and annotation pipeline are released, other labs can rerun the 24-model comparison and use it as a baseline for future video models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension is to apply the same question-generation and human-check protocol to other languages, such as Spanish, German, or Japanese; if models again score higher on Western content in those languages, the bias is cultural rather than merely English-centric.
  • One could separate platform effects from cultural effects by re-sampling Chinese-culture videos from additional sources or regions; if scores shift, part of the reported gap is about video style and source distribution, not cultural knowledge as such.
  • If the Western-centric advantage is driven by training-data exposure, then fine-tuning or continued pretraining on Chinese cultural video should raise Chinese-culture accuracy without much changing Western-culture accuracy; this prediction is checkable.
  • The benchmark's science questions are deliberately general rather than deep, as the paper's limitation section states, so the reported math gap is best read as a lower bound on the true gap for specialized mathematical video reasoning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces VideoVista-CulturalLingo, a video question-answering benchmark containing 1,389 videos, 2,052 clips, and 3,134 QA pairs, designed to span Chinese/English languages, Chinese/American/European cultures, and a broad range of domains. The annotation pipeline is hybrid: Qwen2-VL-72B and DeepSeek-V3/R1 generate candidate questions, which are then filtered and refined by human annotators, with two independent cultural annotators for cultural items. The paper evaluates 24 open-source and proprietary LMMs and reports three main findings: models score lower on Chinese-centric than Western-centric questions, open-source models are markedly weak at Event Localization (best 45.2%), and open-source models trail proprietary models in mathematics. The dataset, code, and project pages are publicly released.

Significance. If the evaluation protocol is matched across model families, the benchmark addresses a genuine gap: most video benchmarks are English-only and Western-centric, while VideoVista-CulturalLingo explicitly targets cultural, linguistic, and domain diversity with 14 subtasks and public release. The paper also ships useful auxiliary analyses, including frame-sampling upper bounds, audio-transcript impact, and temporal-position effects, which are cheap to reproduce given the released data. However, the central comparative claims are currently undermined by a protocol confound in the proprietary-model evaluation, and the ground-truth quality is not quantitatively audited. With those issues resolved, the benchmark could be a reusable measurement instrument.

major comments (3)
  1. [Appendix B.3] The evaluation protocol is not matched across model families: proprietary LMMs (GPT-4o, Gemini-1.5-Flash, Gemini-2.0-Flash-Lite, Gemini-2.0-Flash) receive multiple questions for each video in a single API call and return a dictionary of predictions, while open-source models are evaluated one question per call with re-ingestion of the video frames. This confound directly affects the headline comparisons in the abstract and Section 4.2, such as Gemini-2.0-Flash surpassing Qwen2.5-VL-72B by 15.0%, the Event Localization maximum of 45.2% for open-source models, and the mathematics gap. The paper acknowledges that this 'may introduce some evaluation bias' but provides no control experiment. To support the paper's comparative claims, the authors should re-run at least one open-source model under the batched protocol, or one proprietary model under the single-question protocol, and report whether the performance gaps persist; alternatively, the claims should be explicitly scoped as protocol-dependent.
  2. [Section 3.3] The benchmark's value as a measurement instrument depends on the correctness and video-dependence of the 3,134 gold QA pairs, but the manuscript reports no inter-annotator agreement statistics, no quantitative human-error audit, and no data on how often the two independent cultural annotators disagreed. The pipeline states that questions with score 4 are manually corrected or refined, and that roughly 60% of low-quality questions are eliminated, but there is no reliability metric for the human checks or a sample of cases where the annotators overturned or corrected model-generated answers. The authors should provide a small-scale audit (e.g., re-answer rates by an external annotator, disagreement rates on cultural items, and a breakdown of corrections by task) to substantiate the claim that the benchmark is gold-standard.
  3. [Section 3.1 and Section 3.4] The paper equates video source with cultural content: English videos from YouTube are treated as American/European cultural material, and Chinese videos from Xiaohongshu and BiliBili are treated as Chinese cultural material. Since the central finding is that models perform worse on 'Chinese-centric' questions, the validity of that comparison rests on the assumption that platform provenance and search-keyword domains reliably indicate the intended culture. The paper does not report any validation that the collected videos are in fact about Chinese, American, or European culture (e.g., a human cultural-tagging subset, or an analysis of the retrieved Wikipedia entries' relevance). Without such evidence, the culture-based performance gaps could partly reflect video-content selection rather than cultural comprehension. A validation strategy—such as having annotators label a random subset of videos by culture and reporting agreement—should be added.
minor comments (6)
  1. [Appendix E] The descriptions of Event Sequence and Event Localization are duplicated: the Event Localization paragraph repeats the definition of Event Sequence almost verbatim. The intended distinction between ordering events and locating a specific event should be stated clearly.
  2. [Section 1 and Conclusion] The claim of being 'the first video evaluation benchmark' to bridge culture, language, and domain would be more precise as 'the first to combine all three dimensions', since multi-domain and multi-duration benchmarks such as Video-MME precede this work.
  3. [Section 1] In the contribution list, 'autonomic video annotation framework' should be 'automatic video annotation framework'.
  4. [Section 4.3 and Figure 9] The sentence in the Science task description says 'the questions are relatively simple and can be answered with one or two-hop reasoning', but the subsequently reported mathematics gap is asserted without a difficulty calibration. A brief justification for why the math questions are not simply harder than the other science questions would strengthen the claim.
  5. [Appendix B.2] The description of the Molmo evaluation is clear about the horizontal concatenation of eight frames, but the potential effect of this unusual input format on Molmo's scores is not discussed in Section 4.2 when interpreting the image-LMM results.
  6. [Appendix D.5] The phrase 'backup-index-bydb.html' appears to be a leftover fragment of a URL or filename, not a proper reference. It should be removed or replaced with a correct citation for the Wikipedia extraction tool.

Circularity Check

0 steps flagged · score 0.0 of 10

The benchmark construction and evaluation results are self-contained; no circular derivation is present.

full rationale

The paper constructs a new video QA benchmark and reports model evaluations; it does not derive a scientific constant from an assumed premise. The central claims are empirical measurements on a new dataset, and the annotation pipeline (LLM generation with human verification) does not define the reported results into existence. The use of Qwen2-VL as an annotator and the later evaluation of Qwen2.5-VL is a possible lineage-related bias, but it is not a circular reduction: the benchmark answers were human-checked and the reported scores are external model outputs, not re-statements of the annotation model's own predictions. The self-citation to VideoVista (Li et al., 2024b) is contextual and not load-bearing for the main contributions. The admitted evaluation-bias passage in Appendix B.3 ('Although this may introduce some evaluation bias') concerns protocol confounds between proprietary and open-source models, which is a validity threat rather than a circularity of derivation. No fitted parameter is relabeled as a prediction, and no uniqueness theorem or ansatz is imported from the authors' prior work. Therefore, the derivation chain is independent of its inputs, and the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central contribution is a dataset, not a theory, so the ledger is dominated by dataset-construction assumptions. There are no invented particles or mediators. The main ledger items are annotation-quality assumptions and sampling choices.

free parameters (3)
  • Annotation score cutoffs = score=5 selected, score=4 manually revised
    Choice of quality thresholds (max score across correctness, type relevance, video relevance) is arbitrary and directly determines which questions enter the benchmark.
  • Frame sampling upper bounds = max frames 300 (Qwen2-VL/Qwen2.5-VL), 1 fps (Gemini, GPT-4o), 8 frames (image LMMs)
    Frame sampling strategy is chosen per model family and not controlled for, so part of the performance gap between models could be due to different visual information access rather than model capability.
  • Culture QA count limits = Chinese Culture 231, American Culture 200, European Culture 200
    The balanced counts are a design choice; the relatively small number of culture questions may limit reliability of the culture-level conclusions.
assumptions (4)
  • domain assumption The chosen LLM question generators (Qwen2-VL, DeepSeek-V3/R1) and human annotators produce factually correct gold answers, especially for cultural facts.
    The paper provides no inter-annotator agreement statistics and no independent audit of the gold answers; a single wrong answer in a small subtask can move task accuracy by more than 1%.
  • domain assumption WhisperX transcription is accurate enough, after homophone correction, for science and event questions to be answerable from the transcript.
    Audio transcription errors could make some questions unanswerable or mis-answerable; the paper does not quantify transcription error rates on the final dataset.
  • domain assumption Videos collected from BiliBili and Xiaohongshu are representative of Chinese culture, and YouTube videos are representative of American and European cultures.
    The culture labels come from a classifier plus the platform, and the paper does not report classifier precision or distribution of actual cultural content.
  • domain assumption The LLM-based human-check filtering (CircularEval with Qwen2.5-7B) successfully removes video-agnostic questions.
    The paper reports that about 60% of questions were eliminated by the pipeline, but does not report precision or recall of the filter, so residual video-agnostic questions could inflate or deflate model scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VideoVista-CulturalLingo: 360$^\circ$ Horizons-Bridging Cultures, Languages, and Domains in Video Comprehension." pith.science (2026). https://pith.science/paper/N7U6OH3X

@misc{pith2026250417821,
  author       = {Pith},
  title        = {Pith review of: VideoVista-CulturalLingo: 360$^\circ$ Horizons-Bridging Cultures, Languages, and Domains in Video Comprehension},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N7U6OH3X}},
  note         = {Machine review of arXiv:2504.17821}
}
read the original abstract

Assessing the video comprehension capabilities of multimodal AI systems can effectively measure their understanding and reasoning abilities. Most video evaluation benchmarks are limited to a single language, typically English, and predominantly feature videos rooted in Western cultural contexts. In this paper, we present VideoVista-CulturalLingo, the first video evaluation benchmark designed to bridge cultural, linguistic, and domain divide in video comprehension. Our work differs from existing benchmarks in the following ways: 1) Cultural diversity, incorporating cultures from China, North America, and Europe; 2) Multi-linguistics, with questions presented in Chinese and English-two of the most widely spoken languages; and 3) Broad domain, featuring videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent open-source or proprietary video large models. From the experiment results, we observe that: 1) Existing models perform worse on Chinese-centric questions than Western-centric ones, particularly those related to Chinese history; 2) Current open-source models still exhibit limitations in temporal understanding, especially in the Event Localization task, achieving a maximum score of only 45.2%; 3) Mainstream models demonstrate strong performance in general scientific questions, while open-source models demonstrate weak performance in mathematics.

Figures

Figures reproduced from arXiv: 2504.17821 by the authors.

Figure 1
Figure 1. An example of Chinese Culture in VideoVista-CulturalLingo. The correct answer is high￾lighted in yellow. domains, including text, image, video, and au￾dio over several years. Particularly in the past year, there has been a surge in the development of LMMs capable of processing video inputs. The dramatic expansion in the length of video frame sequences—from just a few frames to several hun￾dred—demonstrates significa… view at source ↗
Figure 2
Figure 2. (Left) Comprehensive statistics from different perspectives. The durations reported are based on the statistics from the 2,052 video clips. The question and answer length is count in tokens; (Right) Videos in VideoVista-CulturalLingo is sourced hundreds of domains from 3 popular video websites across the world. In the video sourced from Xiaohongshu (RedNote), we only present 42 of the all domains. as shown in [PITH… view at source ↗
Figure 3
Figure 3. The three-stage annotation process of VideoVista-CulturalLingo. preserving original semantic content. 3.2 Automatic QA Annotation The annotation framework comprises four distinct tasks: Event, Culture, Object, and Science. Our pipeline employs Qwen2-VL-72B as the primary annotator, Qwen2.5-72B for text-only annotation tasks, and paraphrase-multilingual-MiniLM-L12- v2 for embedding generation. For non-scientific task… view at source ↗
Figures from the paper (28 more)
Figure 4
Figure 4. Figure 4: The LMMs performance divided by Culture, Language and Duration. The Duration in (c): <2 minutes (Short), 2-10 minutes (Medium), >10 minutes (Long). that can compete with proprietary LMMs in this task. In the event task, VideoLLaMA3 also out￾performs all other 7B models…
Figure 5
Figure 5. Figure 5: The LMMs performance divided by domains from 3 video sources: Gemini-2.0-Flash, GPT-4o, Qwen2.5-VL-72B, VideoLLaMA3, InternVideo2.5, MiniCPM-o 2.6. In Figures 5a and Figures 5b, we present only the 18 domains with the highest number of videos. In Figure 5c, we exclude …
Figure 6
Figure 6. Figure 6: Two cases from VideoVista-CulturalLingo. weaker recognition of Chinese Culture. Science. The Science task consists of four sub￾tasks: Summarization & Synthesis, Comparison & Contrast, Application & Procedure, and Scien￾tific Principle. The first three sub-tasks involve…
Figure 7
Figure 7. Figure 7: (a) shows the quantity statistics for the 14 task categories under both Chinese and English languages. (b) [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: An example of eight images combined in a horizontal layout. Full Name Abbreviation YouTube Domains News & Politics NP Sports Spt Entertainment Ent Howto & Style HS People & Blogs PB Autos & Vehicles AV Education Edu Travel & Events TE Film & Animation FA Comedy Com Che…
Figure 9
Figure 9. Figure 9: The Evaluation results in 4 disciplines and 4 math sub-disciplines. The experimental results in the figure represent the average values of the four scientific sub-tasks. In (a),we have list the four disciplines covered by the scientific videos in VideoVista-CulturalLin…
Figure 11
Figure 11. Figure 11 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: The Evaluation results of the Temporal Relationship Between the Event of Interest and the Corresponding Video in our model evaluation. C.4 Impact of Temporal Relation We conduct experiments to investigate the temporal relationship between the event of interest posed i…
Figure 14
Figure 14. Figure 14: Prompt for Audio Refine. prompt to split the video based on audio in be￾low of [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Prompt for Event Annotation. You are an AI assistant tasked with generating detail-oriented questions based on segmented video content. # Input Information The input consists of a sequence of video clips divided based on semantic content. For the *i-th* clip, the foll…
Figure 16
Figure 16. Figure 16: Prompt for Event Description Quetions, Options and Answer Generation. language model. In [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Prompt for Chinese Culture Quetions, Options and Answer Generation. You are an AI assistant tasked with generating science reasoning questions focused on experimental phenomena or procedural principles from segmented video content in chemistry or physics experiments. …
Figure 18
Figure 18. Figure 18: Prompt for Scientific Principle Quetions, Options and Answer Generation. are relatively simple, often yielding high accuracy, making speed our primary evaluation criterion. For bounding box extraction, we also tested models like Florence2 and Grounding-DINO 1.5, but s…
Figure 19
Figure 19. Figure 19: Gradio Interface for scoring. Specifically, during the data annotation process, the Whisper, SAM2, Qwen series models, and InternVL series models were deployed for infer￾ence on local GPU servers. The DeepSeek-V3 and DeepSeek-R1 models is utilized the API services pro…
Figure 20
Figure 20. Figure 20: An Example of Event Description from VideoVista-CulturalLingo. Category: Event-Event Prediction Question:在介绍完鸡爪后，女子接下来会做什么？ (After introducing the chicken feet, what will the woman do next?) A.她会尝试茅台制作成的果冻。。 (She will try jelly made from Maotai.) B.她会尝试红烧肉作为晚餐。 (She w…
Figure 21
Figure 21. Figure 21: An Example of Event Prediction from VideoVista-CulturalLingo [PITH_FULL_IMAGE:figures/full_fig_p022_21.png]
Figure 22
Figure 22. Figure 22: An Example of Event Sequence from VideoVista-CulturalLingo. Category: Event-Event Localization Question: When does the video approximately start tasting the pizza? A. 0:06:35 B. 0:05:35 C. 0:04:35 D. 0:07:35 [PITH_FULL_IMAGE:figures/full_fig_p023_22.png]
Figure 23
Figure 23. Figure 23: An Example of Event Localization from VideoVista-CulturalLingo. Category: Object-Object Temporal Localization Question: At what time does the statue of Dr. Julius Kugy first appear in the video? A. 0:00:16 B. 0:00:20 C. 0:00:12 D. 0:00:24 [PITH_FULL_IMAGE:figures/ful…
Figure 24
Figure 24. Figure 24: An Example of Object Temporal Localization from VideoVista-CulturalLingo [PITH_FULL_IMAGE:figures/full_fig_p023_24.png]
Figure 25
Figure 25. Figure 25: An Example of Object Temporal Sequence from VideoVista-CulturalLingo. Category: Object-Object Spatial Localization Question:Which of the following bounding boxes most accurately depicts the position of the silver trophy with red, white, and blue ribbons at 4 seconds?\…
Figure 26
Figure 26. Figure 26: An Example of Object Spatial Localization from VideoVista-CulturalLingo. Category: Culture-Chinese Culture Question: 视频中提到的这道菜的主要的流行地点是? (What are the main popular locations for the dish mentioned in the video?) A.山西、山东(Shanxi, Shandong) B.北京、河北(Beijing, Hebei) C.江苏、上…
Figure 27
Figure 27. Figure 27: An Example of Chinese Culture from VideoVista-CulturalLingo [PITH_FULL_IMAGE:figures/full_fig_p024_27.png]
Figure 28
Figure 28. Figure 28: An Example of American Culture from VideoVista-CulturalLingo. Category: Culture-European Culture Question: In which country is the beverage mentioned in the video primarily associated with anti-social behavior? A. England B. Scotland C. Ireland D. Wales [PITH_FULL_IM…
Figure 29
Figure 29. Figure 29: An Example of European Culture from VideoVista-CulturalLingo. Category: Science-Summarization & Synthesis Question: According to the video, what key quantum physics concepts are essential for understanding the quantum world? A. Quantum tunneling, Heisenberg uncertaint…
Figure 30
Figure 30. Figure 30: An Example of Summarization & Synthesis from VideoVista-CulturalLingo [PITH_FULL_IMAGE:figures/full_fig_p025_30.png]
Figure 31
Figure 31. Figure 31: An Example of Comparison & Contrast from VideoVista-CulturalLingo. Category: Science-Application & Procedure Question:视频中所讲述的最后一种搜索算法的正确实施步骤是以下哪一项？ (Which of the following is the correct implementation step for the last search algorithm described in the video?) A.维护k个…
Figure 32
Figure 32. Figure 32: An Example of Application & Procedure from VideoVista-CulturalLingo [PITH_FULL_IMAGE:figures/full_fig_p026_32.png]
Figure 33
Figure 33. Figure 33: An Example of Scientific Principle from VideoVista-CulturalLingo [PITH_FULL_IMAGE:figures/full_fig_p027_33.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A survey organizes multimodal reasoning research into a staged roadmap and proposes native large multimodal reasoning models that unify perception, generation, and agentic planning.

Reference graph

Works this paper leans on

46 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, et al. 2024 a . Sharegpt4video: Improving video understanding and generation with better captions. arXiv preprint arXiv:2406.04325

    arXiv 2024

  2. [2]

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, and Sergey Tulyakov. 2024 b . Panda-70m: Captioning 70m videos with multiple cross-modality teachers. arXiv preprint arXiv:2402.19479

    arXiv 2024

  3. [3]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024 c . Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271

    arXiv 2024

  4. [4]

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. 2024. https://arxiv.org/abs/2406.07476 Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms . arXiv preprint arXiv:2406.07476

    arXiv 2024

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv 2025

  6. [6]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huaj...

    arXiv 2024

  7. [7]

    Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvon...

    arXiv 2024

  8. [8]

    Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. 2024. Mmbench-video: A long-form multi-shot benchmark for holistic video understanding. arXiv preprint arXiv:2406.14515

    arXiv 2024

Show all 46 references
  1. [9]

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2024. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075

    2024 arXiv

  2. [10]

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengme...

    2022

  3. [11]

    Li KunChang, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355

    2023 arXiv

  4. [12]

    Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. 2018. Tvqa: Localized, compositional video question answering. In EMNLP

    2018

  5. [13]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024 a . Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  6. [14]

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. 2023. Mvbench: A comprehensive multi-modal video understanding benchmark. arXiv preprint arXiv:2311.17005

    2023 arXiv

  7. [15]

    Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy. 2025 a . https://arxiv.org/abs/2501.13919 Temporal preference optimization for long-form video understanding . Preprint, arXiv:2501.13919

    2025 arXiv

  8. [16]

    Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Alejandro Jaimes, and Jiebo Luo. 2016. Tgif: A new dataset and benchmark on animated gif description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4641--4650

    2016

  9. [17]

    Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, and Min Zhang. 2024 b . https://arxiv.org/abs/2406.11303 Videovista: A versatile benchmark for video understanding and reasoning . Preprint, arXiv:2406.11303

    2024 arXiv

  10. [18]

    Yunxin Li, Baotian Hu, Xinyu Chen, Lin Ma, Yong Xu, and Min Zhang. 2024 c . https://doi.org/10.1109/TMM.2024.3428317 Lmeye: An interactive perception network for large language models . IEEE Transactions on Multimedia, 26:10952--10964

    2024

  11. [19]

    Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, and Min Zhang. 2025 b . https://doi.org/10.1109/TPAMI.2025.3532688 Uni-moe: Scaling unified multimodal llms with mixture of experts . IEEE Transactions on Pattern Analysis and Machine Intelli...

    2025

  12. [20]

    Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. 2023 a . Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122

    2023 arXiv

  13. [21]

    Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han. 2023 b . https://arxiv.org/abs/2312.07533 Vila: On pre-training for visual language models . Preprint, arXiv:2312.07533

    2023 arXiv

  14. [22]

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. 2023 a . Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499

    2023 arXiv

  15. [23]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. 2023 b . Mmbench: Is your multi-modal model an all-around player? arXiv:2307.06281

    2023 arXiv

  16. [24]

    Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. 2024 a . Tempcompass: Do video llms really understand videos? arXiv preprint arXiv: 2403.00476

    2024 arXiv

  17. [25]

    Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. 2024 b . Oryx mllm: On-demand spatial-temporal understanding at arbitrary resolution. arXiv preprint arXiv:2409.12961

    2024 arXiv

  18. [26]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. 2024. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    2024

  19. [27]

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. 2024. Egoschema: A diagnostic benchmark for very long-form video language understanding. In NeurIPS

    2024

  20. [28]

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2024 arXiv

  21. [29]

    Qwen Team. 2025. https://qwenlm.github.io/blog/qwen2.5-vl/ Qwen2.5-vl

    2025

  22. [30]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024 a . Qwen2-vl: Enhancing vision-language m...

    2024 arXiv

  23. [31]

    Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. 2024 b . https://arxiv.org/abs/2406.08035 Lvbench: An extreme long video understanding benchmark . Preprint, arXiv:2406.08035

    2024 arXiv

  24. [32]

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kai Chen, Wenhai Wang, Yu Qiao, Yali Wang, and Limin Wang. 2025. Internvideo2.5: Empowering video mllms with long and rich context modeling. arXiv...

    2025 arXiv

  25. [33]

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang,...

    2024 arXiv

  26. [34]

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. 2021. Next-qa: Next phase of question-answering to explaining temporal actions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9777--9786

    2021

  27. [35]

    Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, and Yueting Zhuang. 2017. Video question answering via gradually refined attention over appearance and motion. In ACM Multimedia

    2017

  28. [36]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  29. [37]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. 2024. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800

    2024 arXiv

  30. [38]

    Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2024. https://arxiv.org/abs/2408.04840 mplug-owl3: Towards long image-sequence understanding in multi-modal large language models . Preprint, arXiv:2408.04840

    2024 arXiv

  31. [39]

    Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. 2019. Activitynet-qa: A dataset for understanding complex web videos via question answering. In AAAI, pages 9127--9134

    2019

  32. [40]

    Zihao Yue, Qi Zhang, Anwen Hu, Liang Zhang, Ziheng Wang, and Qin Jin. 2023. https://doi.org/10.18653/v1/2023.acl-long.257 Movie101: A new movie understanding benchmark . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2023 doi

  33. [41]

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. 2025. https://arxiv.org/abs/2501.13106 Videollama 3: Frontier multimodal foundation model...

    2025 arXiv

  34. [42]

    Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. 2024 a . https://arxiv.org/abs/2406.16852 Long context transfer from language to vision . arXiv preprint arXiv:2406.16852

    2024 arXiv

  35. [43]

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. 2024 b . https://arxiv.org/abs/2410.02713 Video instruction tuning with synthetic data . Preprint, arXiv:2410.02713

    2024 arXiv

  36. [44]

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264

    2024 arXiv

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.