Pith. sign in

REVIEW 4 major objections 6 minor 19 cited by

ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ARC-Hunyuan-Video-7B is a 7B-parameter model that fuses audio with timestamped frames to parse short videos into event captions, summaries, answers, and temporal localizations, with top reported results on its benchmark and grounding tests.

desk verdict Strong engineering with a real contamination risk on the headline temporal-grounding claim; deserves review conditional on data disclosure. read the letter →

arxiv 2507.20939 v1 pith:6QGR4UNH submitted 2025-07-28 cs.CV

Yuying Ge , Yixiao Ge , Chen Li , Teng Wang , Junfu Pu , Yizhuo Li , Lu Qiu , Jin Ma
show 10 more authors
Lisheng Duan Xinyu Zuo Jinwen Luo Weibo Gu Zexuan Li Xiaojing Zhang Yangyu Tao Han Hu Di Wang Ying Shan
This is my paper
classification cs.CV
keywords structuredvideocomprehensionshort-videounderstandingtemporalgroundingaudio-visualfusiontimestampoverlayGRPOreinforcementlearningShortVid-Benchsearchandrecommendation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

What the paper is trying to establish is that a 7B-parameter model—not a much larger one—can genuinely understand the fast, dense, user-generated short videos that dominate mobile platforms, by reading raw video, audio, and text together and by knowing exactly when things happen. The authors build ARC-Hunyuan-Video-7B around two mechanisms: an audio encoder fused with visual frames at one-second granularity, and timestamps rendered directly onto frames; they then train it in five stages on millions of in-house videos. Their central quantitative claims are 74.3% accuracy on their new ShortVid-Bench and mIoU of 54.8 on Charades-STA and 41.7 on ActivityNet, above strong open baselines, plus a finding that reinforcement learning on verifiable tasks is what makes the model learn high-quality subjective summaries. A sympathetic reader would care because this is the capability that could turn raw short-video streams into searchable, recommendable structured metadata at production cost.

What carries the argument

Two architectural mechanisms carry the argument. First, a timestamp overlay: frames are sampled at 1 fps with a maximum of 150 frames, and each frame has its HH:MM:SS timestamp rendered in the corner before ViT encoding, giving the LLM a directly readable clock for localization. Second, fine-grained visual-audio synchronization: raw audio is encoded with Whisper into 30-second chunks, and the audio tokens are zero-padded and added to the 112 visual tokens of the corresponding frame, so every fused embedding covers the same time interval with no learned alignment parameters. The training machinery is the five-stage regimen—pre-training, instruction fine-tuning, cold start, GRPO reinforcement learning, and final instruction fine-tuning—where the RL stage uses only verifiable rewards: binary correctness for 100K multiple-choice questions and IoU against ground truth for 35K temporal grounding instances. The paper's core thesis is that these verifiable tasks, rather than the subjective summaries themselves, are what unlock high-quality subjective understanding.

What would settle it

Check whether any Charades-STA or ActivityNet video appears in the 0.5M temporal-grounding pre-training instances or the 10K academic instruction-tuning grounding samples by matching video IDs or frame hashes; if test videos were seen in training, the reported mIoU gains could reflect memorization. Separately, evaluate the model on a fresh set of platform shorts with human-annotated multiple-choice questions that never passed through the automated annotation pipeline; a large accuracy drop relative to ShortVid-Bench would indicate that the benchmark rewards the training distribution rather than general comprehension.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that structured video comprehension—decomposing a video into events with time spans, summarizing intent, answering open-ended questions, and grounding events—can be achieved by a compact multimodal model rather than by task-specific pipelines. The evidence is the model's reported performance: 74.3% on ShortVid-Bench compared with 67.8 to 68.3% for the strongest compared baselines, and temporal grounding mIoU of 54.8 on Charades-STA and 41.7 on ActivityNet, well above the same baselines. The paper further claims a training-curriculum finding: directly fine-tuning on human-annotated summaries gives little gain (6.42 to 6.67 in LLM-as-judge score), DPO gives no gain (6.50), but GRPO post-training on verifiable multiple-choice and grounding tasks followed by summary fine-tuning lifts the score to 6.99. The interpretation the authors offer is that objective, verifiable tasks teach the model the perception and temporal-reasoning substrate that subjective annotation learning requires, and that explicit timestamp overlays plus audio-visual alignment provide that substrate.

Load-bearing premise

The load-bearing premise is that the evaluation protocol is unbiased: ShortVid-Bench measures general short-video comprehension rather than familiarity with the training distribution, and the public datasets used for temporal-grounding training do not include the Charades-STA or ActivityNet test videos.

Editorial extensions

If this is right

  • With 1,000 training samples per downstream task, the model can be fine-tuned for brief summaries, detailed summaries, and extended browsing words, so video search and recommendation metadata can be generated automatically.
  • The reported production results—retrieval CTR up 5.88%, long-click rate up 3.34%, and satisfied QV proportion up 1.77%—imply that structured comprehension outputs are directly usable as retrieval targets and recommendation features.
  • The Charades-STA and ActivityNet grounding results suggest that timestamp-overlay training transfers beyond platform-specific shorts to standard academic temporal grounding.
  • The pilot numbers (6.42 to 6.99 from GRPO plus fine-tuning) indicate that verifiable-reward RL should precede learning from human subjective annotations, not just for video but for any multimodal task with an objectively checkable subtask.
  • At 10 seconds of inference per one-minute video on an H20 GPU under vLLM, the approach is cheap enough for near-real-time video indexing at platform scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark-overlap concern is resolved, the same architecture and training recipe should transfer to other dense, fast-paced video domains such as livestreams, meetings, and egocentric footage, where audio carries intent and visuals carry actions.
  • The reported advantage of audio fusion predicts a specific ablation result: on questions whose answer is carried mainly by non-speech audio, such as music mood or sound effects, removing the audio stream should produce a large accuracy drop, while speech-transcript-only models should fail on those questions.
  • The pilot finding that DPO on human summaries fails while GRPO on verifiable tasks succeeds suggests that preference optimization on subjective outputs may be less effective than first shaping perception and temporal reasoning with objective rewards; a similar comparison could be run on text summarization or image captioning.
  • ShortVid-Bench's six dimensions could serve as a reusable evaluation template, but its construction from in-house automated questions plus human annotation means its validity depends on public release and fresh-video testing; otherwise accuracy could partly reflect the training distribution of the annotation pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents ARC-Hunyuan-Video-7B, a 7B-parameter multimodal model that processes raw video, audio, and text jointly for structured comprehension of real-world short videos. The model builds on the Hunyuan-7B VLM, adds an audio encoder with fine-grained visual-audio synchronization, and overlays timestamps on sampled frames to provide explicit temporal awareness. Training proceeds through five stages: pre-training on automatically annotated data, initial instruction fine-tuning, a cold-start CoT phase, GRPO-based reinforcement learning on verifiable tasks (multiple-choice QA and temporal grounding), and a final instruction fine-tuning stage with human-annotated data and rejection-sampled trajectories. The authors introduce ShortVid-Bench, a human-annotated multiple-choice benchmark spanning six comprehension dimensions, and report results on it together with Charades-STA, ActivityNet, MVBench, VCR-Bench, and Video-Holmes. They claim state-of-the-art performance on short-video comprehension and temporal grounding, and report downstream deployment gains in search and recommendation metrics.

Significance. If the evaluation concerns are resolved, this paper would be a solid system contribution: it combines an end-to-end audio-visual architecture with a multi-stage training recipe, uses verifiable RL rewards to improve subjective video understanding, and provides concrete deployment results. The open-sourced model checkpoint, API, and inference code are clear strengths that support reproducibility of the model itself. The timestamp-overlay mechanism is a simple and potentially effective way to give an LLM explicit temporal grounding. However, the headline claims currently rest on an unreleased, self-constructed benchmark and on temporal-grounding training data that is not disclosed. These gaps are load-bearing: without naming the training datasets and demonstrating disjointness from the evaluation sets, the reported mIoU gains are not verifiable, and without releasing ShortVid-Bench, the central benchmark claim is not independently assessable. The paper is therefore promising but needs substantive revision before the claims can be accepted.

major comments (4)
  1. [§3.2.2, §3.3.2, Table 1] The temporal grounding training data is not disclosed: Section 3.2.2 states that 0.5M temporally grounded instances come from 'multiple public datasets', and Section 3.3.2 mentions 10K samples from 'academic datasets', but no dataset names are given. Table 1 reports mIoU of 54.8 on Charades-STA and 41.7 on ActivityNet, substantially above Qwen2.5-VL-7B's 46.9 and 25.1. If any of the unnamed training datasets overlap with the Charades-STA or ActivityNet test videos, the reported gains would be contaminated and the headline 'state-of-the-art' claim for temporal grounding would be invalid. The authors must list the training datasets, clarify the exact splits used, and explicitly rule out overlap with the test sets used in Table 1. They should also provide error bars or significance tests to support the magnitude of the gains.
  2. [§4.2.1, Abstract, §4.3] ShortVid-Bench is self-constructed and the paper does not state that it will be released; the open-sourced assets listed in the abstract and Section 4.3 are the model checkpoint, API, and inference code, not the benchmark. Without public access to the benchmark, the central claim of 'exceptional performance in real-world video understanding' on this benchmark is not independently verifiable. Moreover, the benchmark construction uses an automated pipeline and human annotation (Section 4.2.1) that appears methodologically similar to the bootstrapped annotation pipeline used to generate training data (Section 3.2.1), raising a risk that the evaluation distribution overlaps with the training distribution. The authors should release ShortVid-Bench (or a substantial subset), describe how the benchmark videos were selected, and provide evidence that they are disjoint from the in-house training videos.
  3. [§5, Table 1] The conclusion states that ARC-Hunyuan-Video-7B 'achieves state-of-the-art performance on short video comprehension benchmarks and temporal grounding benchmarks'. This overstates the evidence: Table 1 shows the model is slightly below Qwen2.5-VL-7B on MVBench (62.6 vs. 62.9), VCR-Bench (50.5 vs. 53.7), and Video-Holmes (40.9 vs. 41.6), and the only 'short video comprehension' benchmark used is the unreleased ShortVid-Bench. The claim should be restricted to the specific benchmarks that support it, and even there should be conditioned on the resolution of the data-disclosure and benchmark-access issues raised above.
  4. [Table 1, §4.2.2] No error bars, confidence intervals, or significance tests are reported for any of the quantitative results. Some differences are small (e.g., MVBench 62.6 vs. 62.9), making it impossible to tell whether the model is statistically indistinguishable from or worse than the baseline on general video understanding. For the temporal-grounding results, the large relative gain on ActivityNet (41.7 vs. 25.1) is notable but unexplained; reporting repeated evaluations or bootstrap confidence intervals would clarify whether these differences are reliable. This is especially important given the unresolved training/test overlap concern in the first major comment.
minor comments (6)
  1. [§3.1] The synchronization strategy is described as 'adaptive and parameter-free'; the description (zero-padding audio tokens and adding them to visual tokens) makes the 'adaptive' part unclear, and a more precise name or explanation would help.
  2. [§4.1.2] The baseline name 'Keye-VL-8B-8B' appears in the first paragraph; this seems to be a typo for 'Keye-VL-8B'.
  3. [§4.3.1, Figure 11] The example in Figure 11 contains a long untranslated Chinese passage in the 'Extended Browsing Words' output; providing an English translation of this output (or a separate English example) would make the figure self-contained for an international readership.
  4. [Table 1] For the general benchmarks, the differences among models are small in some cases; consider adding a note about whether the differences are within the noise of a single evaluation run, or report variance if available.
  5. [§3.3.1] The notation 'GRPO-sft 6.99' is used without defining 'sft'; please spell out the acronym on first use.
  6. [§4.3.3] The production metrics (e.g., CTR increase of 5.88%) are reported without a description of the evaluation period, the control/baseline setup, or any statistical significance; a brief methodological note would make these claims more credible.

Circularity Check

0 steps flagged · score 1.0 of 10

No definitional circularity; the bootstrapped training-data loop and unnamed public training datasets are transparency/contamination risks, not reductions of the evaluation to the training inputs.

full rationale

I traced the paper's support chain: architecture (audio encoder + timestamp overlay) -> five-stage training on in-house annotated data -> ShortVid-Bench and external grounding/general benchmarks. No equation in the paper defines a predicted quantity in terms of a fitted quantity. The bootstrapped annotation pipeline (Sec. 3.2.1) does re-insert the model's own descriptions/summaries into the pre-training annotations, but this is a data-refinement loop for training, and the quantitative evaluation is not shown to use those same outputs: ShortVid-Bench is human-curated (Sec. 4.2.1) and Table 1 also reports external benchmarks. The temporal-grounding SOTA (54.8 Charades-STA, 41.7 ActivityNet) rests on unnamed 'multiple public datasets' (Sec. 3.2.2) and 'academic datasets' (Sec. 3.3.2); if those include the evaluation sets, the numbers would be contaminated. That is an unverified evaluation-integrity risk, not an exhibited circular reduction. The self-citations (Video-Holmes, and co-authored benchmark papers such as MvBench/Seed-Bench) are used as public evaluation instruments, not as a load-bearing justification for the model's design. I therefore find no circular step that meets the quoted-reduction standard.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim does not rest on a mathematical derivation, so the ledger is dominated by domain assumptions about data quality, benchmark validity, and training transfer. The key free parameters are the hand-chosen temporal sampling and audio fusion budgets, which set the ceiling on temporal grounding. No new entities are introduced.

free parameters (4)
  • Frame sampling rate = 1 fps, max 150 frames
    Hand-selected; sets the temporal granularity for all localization and captioning tasks and therefore directly determines the grounding ceiling.
  • Audio chunk and token budget = 30 s chunks, 1500 tokens; 2 s truncation for long videos
    Design choice that defines how much audio context the LLM receives per fused embedding; affects the audio-visual synchronization claim in Section 3.1.
  • Visual resolution = 640x640
    Fixed downsampling chosen to enable visual-audio fusion; affects fine-grained visual reasoning scores.
  • GRPO KL coefficient = 0.1
    RL hyperparameter reported in Section 3.3.4; chosen by hand and not swept, though it influences post-training behavior.
assumptions (6)
  • domain assumption Pre-trained ViT and Whisper encoder features remain useful for short-video content after fine-tuning.
    Section 3.1 relies on these encoders without evaluating their saturation; the MLP adapter is the only learned projection.
  • ad hoc to paper Timestamp overlay renders frames with HH:MM:SS text that the LLM can read and map to actual event times.
    Section 3.1 introduces this mechanism as the main temporal localization signal; no ablation isolates its contribution from the audio and video inputs.
  • ad hoc to paper Bootstrapped annotation pipeline yields low-noise ground truth.
    Section 3.2.1 describes iterative refinement where the model's own outputs are fed back into a closed-source LLM; the quality of this synthetic supervision is not independently measured.
  • domain assumption GRPO on verifiable tasks (MCQ, grounding) transfers to subjective summarization quality.
    Section 3.3.1 supports this with a 140-video pilot (6.42 vs 6.99) without significance testing; the full recipe depends on this transfer.
  • ad hoc to paper ShortVid-Bench human annotations and distractors are valid and unbiased.
    Section 4.2.1 constructs the benchmark in-house; no inter-annotator agreement or distractor validation is reported, and the benchmark is not released.
  • ad hoc to paper Unlisted public grounding datasets used for training do not overlap with Charades-STA and ActivityNet test sets.
    Sections 3.2.2 and 3.3.2 mention academic datasets without names; if the evaluation benchmarks are among them, the grounding scores are inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts." pith.science (2026). https://pith.science/paper/6QGR4UNH

@misc{pith2026250720939,
  author       = {Pith},
  title        = {Pith review of: ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6QGR4UNH}},
  note         = {Machine review of arXiv:2507.20939}
}
read the original abstract

Real-world user-generated short videos, especially those distributed on platforms such as WeChat Channel and TikTok, dominate the mobile internet. However, current large multimodal models lack essential temporally-structured, detailed, and in-depth video comprehension capabilities, which are the cornerstone of effective video search and recommendation, as well as emerging video applications. Understanding real-world shorts is actually challenging due to their complex visual elements, high information density in both visuals and audio, and fast pacing that focuses on emotional expression and viewpoint delivery. This requires advanced reasoning to effectively integrate multimodal information, including visual, audio, and text. In this work, we introduce ARC-Hunyuan-Video, a multimodal model that processes visual, audio, and textual signals from raw video inputs end-to-end for structured comprehension. The model is capable of multi-granularity timestamped video captioning and summarization, open-ended video question answering, temporal video grounding, and video reasoning. Leveraging high-quality data from an automated annotation pipeline, our compact 7B-parameter model is trained through a comprehensive regimen: pre-training, instruction fine-tuning, cold start, reinforcement learning (RL) post-training, and final instruction fine-tuning. Quantitative evaluations on our introduced benchmark ShortVid-Bench and qualitative comparisons demonstrate its strong performance in real-world video comprehension, and it supports zero-shot or fine-tuning with a few samples for diverse downstream applications. The real-world production deployment of our model has yielded tangible and measurable improvements in user engagement and satisfaction, a success supported by its remarkable efficiency, with stress tests indicating an inference time of just 10 seconds for a one-minute video on H20 GPU.

Discussion (0). Sign in to comment.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning

    cs.CV 2026-07 accept novelty 7.0 of 10

    Latent event planning plus event-factorized attention restructures the AR dependency graph so dense video captions can be decoded in parallel with higher accuracy and 3–4× wall-clock speedup.

  2. Learning to Deny: Action Denial in Multimodal Large Language Models

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    MLLMs drop from over 85% accuracy on action presence to under 50% on matched action-denial videos, exposing a causal verification gap that causal graph prompts partially close.

  3. StoryTR: Narrative-Centric Video Temporal Retrieval with Theory of Mind Reasoning

    cs.AI 2026-04 unverdicted novelty 7.0 of 10

    StoryTR is a new benchmark and agentic data pipeline that adds explicit Theory of Mind reasoning chains to train smaller video retrieval models, yielding a 15% relative IoU gain over larger baselines on narrative content.

  4. OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic Video

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    OmniScript is a new 8B omni-modal model that turns long cinematic videos into scene-by-scene scripts and matches top proprietary models on temporal localization and semantic accuracy.

  5. Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation

    cs.CV 2026-02 unverdicted novelty 7.0 of 10

    Video-OPD uses on-policy distillation from a frontier teacher to turn sparse episode rewards into dense step-wise signals for more efficient post-training of MLLMs on temporal video grounding.

  6. OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A training-free two-stage token compression framework preserves 92.9% of Qwen2.5-Omni-7B's audio-visual understanding accuracy while using 6.8% of the original multimodal-token FLOPs.

  7. Allocation Before Ranking: Decoupled Token Compression for OmniLLMs

    cs.AI 2026-08 conditional novelty 6.0 of 10

    MACER decouples cross-modal token-budget allocation from within-modality token ranking and improves accuracy over shared top-K compression in Qwen2.5-Omni and OmniVinci models.

  8. RefCaptioner: Multi-Reference Image-Grounded Video Captioning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Mixed-data SFT plus Hierarchical Coverage-Discounted GRPO yields open-source SOTA multi-reference image-grounded video captions on MRVBench without hurting general captioning.

  9. O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    O-MARC is a compression distillation framework that lets compact omnimodal models maintain or exceed full-token performance on video QA while cutting latency and memory by about 35%.

  10. Streaming Video Instruction Tuning

    cs.CV 2025-12 unverdicted novelty 6.0 of 10

    Streamo is a streaming video LLM trained end-to-end on the new Streamo-Instruct-465K dataset that unifies multiple real-time video tasks with claimed strong temporal reasoning and generalization.

  11. AdaTooler-V: Adaptive Tool-Use for Images and Videos

    cs.CV 2025-12 conditional novelty 6.0 of 10

    AdaTooler-V trains MLLMs to adaptively use vision tools via AT-GRPO reinforcement learning and new datasets, reaching 89.8% on V* and outperforming GPT-4o.

  12. OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models

    cs.CV 2025-11 conditional novelty 6.0 of 10

    OmniZip introduces an audio-guided dynamic token compression framework that achieves 3.42X inference speedup and 1.4X memory reduction for omnimodal LLMs without any training.

  13. PercepCap: Video Captioner with Structured Spatio-Temporal Perception

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Explicitly generating object trajectories and event timestamps before the final caption improves detailed video captioning on multiple benchmarks, though the improvement is largely driven by an external perception oracle.

  14. CogniRoute: Learning to Route Social Evidence in Omni-Modal Models

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    CogniRoute adds a cognitive schema and route-aware RL to an omni-modal MoE, reaching 59.38% accuracy on a new 118K-example social video QA benchmark and beating prior baselines by 15-27 points.

  15. Stage-adaptive Token Selection for Efficient Omni-modal LLMs

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    SEATS adaptively selects and removes non-text tokens before and inside the LLM layers of omni-modal models, yielding 9.3x FLOPs reduction and 4.8x prefill speedup at 10% token retention while keeping 96.3% performance.

  16. OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models

    cs.AI 2026-05 unverdicted novelty 5.0 of 10

    OmniRefine introduces alignment-aware chunk refinement via similarity and dynamic programming followed by modality-cooperative token compression, achieving near-baseline accuracy at 44% token retention on WorldSense.

  17. Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation

    cs.CV 2026-02 conditional novelty 5.0 of 10

    Video-OPD post-trains video LLMs for temporal grounding by distilling a GRPO-trained teacher via token-level reverse KL on on-policy trajectories, outperforming GRPO with lower cost.

  18. Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning

    cs.CV 2026-07 unverdicted novelty 4.0 of 10

    TCA-Captioner introduces an Observer-Checker-Corrector refinement loop and TCA-Bench to address modality detachment and temporal incoherence in audiovisual video captioning.

  19. Empowering Nanoscale Connectivity through Molecular Communication: A Case Study of Virus Infection

    cs.NI 2025-08 unverdicted novelty 4.0 of 10

    A position paper proposing molecular communication as the link layer for epidemic-control bio-nano networks, with an ORF3a-based mutation identification simulation; the provided manuscript body does not match this abstract.

Reference graph

Works this paper leans on

19 extracted references · 2 canonical work pages · cited by 18 Pith papers

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923,

    arXiv

  2. [8]

    Seed-bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125,

    Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125,

    arXiv

  3. [10]

    Egoplan-bench2: A benchmark for multimodal large language model planning in real-world scenarios.arXiv preprint arXiv:2412.04447,

    Lu Qiu, Yi Chen, Yuying Ge, Yixiao Ge, Ying Shan, and Xihui Liu. Egoplan-bench2: A benchmark for multimodal large language model planning in real-world scenarios.arXiv preprint arXiv:2412.04447,

    arXiv

  4. [11]

    Audio-visual llm for video understanding.arXiv preprint arXiv:2312.06720,

    Fangxun Shu, Lei Zhang, Hao Jiang, and Cihang Xie. Audio-visual llm for video understanding.arXiv preprint arXiv:2312.06720,

    arXiv

  5. [12]

    video-salmonn: Speech-enhanced audio-visual large language models.arXiv preprint arXiv:2406.15704,

    Guangzhi Sun, Wenyi Yu, Changli Tang, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, Yuxuan Wang, and Chao Zhang. video-salmonn: Speech-enhanced audio-visual large language models.arXiv preprint arXiv:2406.15704,

    arXiv

  6. [13]

    video- salmonn-o1: Reasoning-enhanced audio-visual large language model.arXiv preprint arXiv:2502.11775,

    Guangzhi Sun, Yudong Yang, Jimin Zhuang, Changli Tang, Yixuan Li, Wei Li, Zejun Ma, and Chao Zhang. video- salmonn-o1: Reasoning-enhanced audio-visual large language model.arXiv preprint arXiv:2502.11775,

    arXiv

  7. [14]

    video- salmonn 2: Captioning-enhanced audio-visual large language models.arXiv preprint arXiv:2506.15220,

    Changli Tang, Yixuan Li, Yudong Yang, Jimin Zhuang, Guangzhi Sun, Wei Li, Zejun Ma, and Chao Zhang. video- salmonn 2: Captioning-enhanced audio-visual large language models.arXiv preprint arXiv:2506.15220,

    arXiv

  8. [15]

    Kwai keye-vl technical report.arXiv preprint arXiv:2507.01949,

    Kwai Keye Team, Biao Yang, Bin Wen, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, et al. Kwai keye-vl technical report.arXiv preprint arXiv:2507.01949,

    arXiv

Show all 19 references
  1. [17]

    Video-llama: An instruction-tuned audio-visual language model for video understanding.arXiv preprint arXiv:2306.02858,

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding.arXiv preprint arXiv:2306.02858,

    arXiv

  2. [18]

    Self-supervised product title rewrite for product listing ads

    Xue Zhao, Dayiheng Liu, Junwei Ding, Liang Yao, Mahone Yan, Huibo Wang, and Wenqing Yao. Self-supervised product title rewrite for product listing ads. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human La...

    2022

  3. [19]

    Pre-trained language model based ranking in baidu search

    Lixin Zou, Shengqiang Zhang, Hengyi Cai, Dehong Ma, Suqi Cheng, Shuaiqiang Wang, Daiting Shi, Zhicong Cheng, and Dawei Yin. Pre-trained language model based ranking in baidu search. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 401...

    2021

  4. [2015]

    Egoplan- bench: Benchmarking multimodal large language models for human-level planning.arXiv preprint arXiv:2312.06722,

    Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu. Egoplan- bench: Benchmarking multimodal large language models for human-level planning.arXiv preprint arXiv:2312.06722,

    arXiv

  5. [2017]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    arXiv

  6. [2020]

    Clip2video: Mastering video-text retrieval via image clip.arXiv preprint arXiv:2106.11097,

    Han Fang, Pengfei Xiong, Luhui Xu, and Yu Chen. Clip2video: Mastering video-text retrieval via image clip.arXiv preprint arXiv:2106.11097,

    arXiv

  7. [2021]

    Vcr-bench: A comprehensive evaluation framework for video chain-of-thought reasoning.arXiv preprint arXiv:2504.07956,

    Yukun Qi, Yiming Zhao, Yu Zeng, Xikun Bao, Wenxuan Huang, Lin Chen, Zehui Chen, Jie Zhao, Zhongang Qi, and Feng Zhao. Vcr-bench: A comprehensive evaluation framework for video chain-of-thought reasoning.arXiv preprint arXiv:2504.07956,

    arXiv

  8. [2022]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, et al. Qwen2. 5-omni technical report.arXiv preprint arXiv:2503.20215,

    arXiv

  9. [2023]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271,

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271,

    arXiv

  10. [2024]

    Video-holmes: Can mllm think like holmes for complex video reasoning?arXiv preprint arXiv:2505.21374,

    Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, and Ying Shan. Video-holmes: Can mllm think like holmes for complex video reasoning?arXiv preprint arXiv:2505.21374,

    arXiv

  11. [2025]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:...

    2010 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.