Pith. sign in

REVIEW 3 major objections 6 minor 43 references

Video captions can name which reference images match which local phrases, not just describe the scene.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 05:06 UTC pith:M6RRDG2T

load-bearing objection Solid systems paper that cleanly defines multi-ref phrase-grounded video captioning, ships a usable benchmark and post-training recipe, and earns its open-source lead—with judge coupling and thin human/downstream evidence as real but bounded caveats. the 3 major comments →

arxiv 2607.28509 v1 pith:M6RRDG2T submitted 2026-07-30 cs.CV

RefCaptioner: Multi-Reference Image-Grounded Video Captioning

Tengfei Liu , Yang Shi , Yuran Wang , Xiaohan Zhang , Yuqing Wen , Yuqi Tang , Qixun Wang , Zhuoran Zhang
show 11 more authors
Xuanyu Zhu Weihong Lin Xinlei Yu Yujie Wei Xinwei Long Fengxiang Wang Xinlong Chen Yue Ding Jialu Chen Haotian Wang Yuanxing Zhang
This is my paper
classification cs.CV
keywords video captioningmulti-reference groundingphrase-level bindingGRPOdistractor rejectionMRVBenchmultimodal LLMsvideo reconstruction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard video captioners describe what happens on screen but never say which of several reference photos those details come from. This paper defines multi-reference image-grounded video captioning: write a factual caption and attach each usable image tag right after the phrase it supports, while ignoring distractors and grouping multiple views of the same subject. RefCaptioner is a two-stage post-training recipe—mixed supervised fine-tuning plus Hierarchical Coverage-Discounted GRPO—that jointly trains reference selection, phrase-level binding, distractor rejection, and cross-reference consistency without giving up ordinary caption quality. The authors release MRVBench (real and AI-generated videos) and show the open 8B model leads open-source systems on that benchmark, stays competitive on standard caption benchmarks, and produces captions humans prefer and that reconstruct source videos more faithfully when fed to video generators.

Core claim

Multi-reference grounding can be learned together with factual video captioning: after mixed-data SFT, Hierarchical Coverage-Discounted GRPO—with a factuality branch and a grounding branch that rewards correct bindings, suppresses distractors (DAES), and enforces same-entity tag grouping (CRSC)—yields an open model that leads open-source systems on MRVBench while remaining competitive on general caption benchmarks and improving human preference and caption-conditioned reconstruction.

What carries the argument

Hierarchical Coverage-Discounted GRPO (HCD-GRPO): a dual-branch group-relative policy reward in which coverage of correct facts or correctly bound references is the positive signal and measurable errors (QA mistakes, bad bindings, distractor use, split same-entity tags) discount that signal.

Load-bearing premise

The training rewards and benchmark scores both lean on large language-model judges as stand-ins for human judgments of factual coverage and whether a phrase truly matches a reference image.

What would settle it

If independent human raters (not the automatic judge) reverse the ranking on phrase-level binding, distractor rejection, and reconstruction fidelity—especially against the same proprietary models used as judges—or if post-hoc tag insertion matched one-pass RefCaptioner under blind human scoring, the central claim would fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Reference-conditioned video generation and editing can take phrase-tagged captions as explicit subject–attribute–scene wiring instead of free text alone.
  • Open captioners can approach proprietary multi-reference grounding without sacrificing ordinary detailed video description.
  • Post-hoc rewriting of plain captions is a weak substitute for learning selection and binding jointly with generation.
  • MRVBench becomes a shared test for whether models pick the right images, bind them locally, reject distractors, and group multi-view subjects.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same coverage-discounted split between ‘say what is true’ and ‘cite only supported evidence’ could transfer to multi-image document or long-video grounding.
  • Heavy dependence on one family of judge models for both rewards and leaderboard scores suggests a follow-up with held-out human-only labels to stress-test the reported margins.
  • Caption-conditioned reconstruction is an early signal that grounded captions may act as a portable interface layer between understanding models and generators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces multi-reference image-grounded video captioning: given a video and candidate reference images (including distractors), a model must produce a factual caption with phrase-level <Image_i> tags, correct selection, local binding, distractor rejection, and same-entity grouping. RefCaptioner post-trains Qwen3-VL-8B via mixed multi-reference/general-caption SFT, then Hierarchical Coverage-Discounted GRPO with a dual-branch reward (factual coverage discounted by VQA errors; grounding coverage discounted by binding errors, DAES, and CRSC). The authors build a 20k-video training corpus and MRVBench (462 held-out real/AIGC videos, 3,831 references, 2,172 QA pairs). On MRVBench, RefCaptioner leads open-source models (MRVScore 0.882) and is competitive with proprietary systems; it also improves or holds on VDC and VCapsBench. Ablations, a CaptionRefine control, reference-count robustness, human preference (n=40), and caption-conditioned reconstruction GSB are reported as supporting evidence.

Significance. If the results hold under stronger human validation, this is a clear and timely contribution: it defines a practically motivated task at the intersection of video captioning and multi-reference generation/editing, ships a dedicated benchmark with explicit distractors and multi-view entity groups, and shows that joint post-training (not post-hoc tag insertion) is needed for phrase-level grounding. Strengths include a held-out test set, broad open-source and proprietary baselines, CaptionRefine and reward ablations (Tables 2 and 5), retention of general captioning ability (Tables 3–4), and public release of MRVBench evaluation data. The work is empirical systems research rather than theory; its value is the task formulation, training recipe, and evidence that multi-reference grounding can be added without collapsing standard caption quality.

major comments (3)
  1. [§5.1, Table 1, Appendix A.4–A.5] §5.1, Appendix A.4–A.5, and Table 1: Gemini-3.1-Pro is both a top proprietary baseline and the primary automatic judge for KP-Cov, VQA/VQA-Cov, Ref-Bind, and Subj-R/F1. This coupling can systematically favor styles the judge prefers and inflate margins relative to Gemini (and possibly other models). The paper should (i) quantify judge–human agreement on a stratified subset for binding and subject consistency, (ii) report at least one alternate-judge or cross-model re-score for the main metrics, and (iii) state clearly which claims rest only on automatic scores versus human checks. Without this, the SOTA comparison to Gemini in Table 1 is not fully trustworthy even if open-source rankings remain informative.
  2. [§6.7, Fig. 6, Appendix E.1, F.1–F.2] §6.7, Fig. 6, and Appendix E.1/F.1–F.2: The abstract and conclusion claim captions are “preferred by annotators” and enable “more source-faithful video reconstruction.” Human ranking/binding uses only 40 videos and three graduate annotators (F.1); reconstruction GSB uses the same small expert pool and conditions the generator on both caption text and the cited reference subset (E.1). Wins can therefore come from better reference selection alone rather than better phrase-level grounded language. Please either (a) run a controlled reconstruction where all methods receive the same gold positive references (text-only difference), and/or (b) substantially enlarge and statistically report the human study, and temper abstract/conclusion language to match the actual support.
  3. [§4.3, Table 5, Appendix B.3] §4.3 and Appendix B.3: HCD-GRPO’s load-bearing signals (keypoint scores, VQA error discounts, binding accuracy, CRSC) are themselves LLM-judge outputs with fixed ordinal scales and hand-set λb, λd, λc, λqa and 0.5/0.5 branch weights. Table 5 shows component necessity but not sensitivity to judge choice or λ’s. A short sensitivity sweep (e.g., ±λ or an alternate binding judge) and an explicit limitation that reward and eval judges share the same family of proxies would strengthen the claim that gains are from the hierarchical coverage-discount design rather than judge-specific overfitting.
minor comments (6)
  1. [§5, Appendix A.2] Appendix A.2 notes the 20k training corpus will not be released. Please state this prominently in the main text (contributions or experiments) and clarify what artifacts will be released (code, MRVBench, prompts, reward configs) so reproducibility expectations are clear.
  2. [Table 1] Table 1 header says “best result in each column is highlighted in green,” but the provided text does not convey highlighting; ensure camera-ready formatting makes open-source bests unambiguous, and that proprietary rows remain clearly excluded from “best.”
  3. [§4.3.1–4.3.2] Eqs. (1)–(4): define more explicitly how item-level binding decisions enter C_ref (whether a tag must pass the same judge as A_bind), and whether malformed-tag capping (τ_struct=0.2) was ever triggered on validation—useful for interpreting reward scale.
  4. [§6.5, Figure 4] Figure 4’s robustness score is a product of recall × binding × distractor rejection; briefly justify multiplicativity versus a weighted sum, and report per-component curves in the appendix to avoid hiding trade-offs.
  5. [§2, §6.1] Minor polish: “V ATEX” spacing (Related Work); consistent naming of Qwen3.6 vs Qwen3-VL variants; ensure all baselines listed in §6.1 (e.g., VideoLLaMA3, LLaVA-Video) appear in the relevant tables or are marked as appendix-only.
  6. [Appendix F.1] Human study (F.1) reports mean ranks and binding accuracy but no inter-annotator agreement (e.g., Kendall’s W or pairwise agreement). Adding IAA would help readers weight Fig. S5.

Circularity Check

0 steps flagged

No derivation circularity: empirical SFT+GRPO system with held-out bench; judge–metric alignment is validity risk, not input=prediction by construction.

full rationale

RefCaptioner is a systems/post-training paper, not a first-principles derivation. The claimed chain is: define multi-reference grounded captioning → mixed-data SFT + HCD-GRPO with explicit reward terms (R_cap, R_ref, DAES, CRSC) → evaluate on held-out MRVBench (462 videos fully held out from the 20k training corpus) plus external VDC/VCapsBench, human preference, and reconstruction. None of the load-bearing performance claims reduce by construction to fitted scalars renamed as predictions, nor to a self-cited uniqueness theorem that forbids alternatives. Overlapping-author citations (e.g., MultiRef-Compass, CapRiCorn-1K) appear as related-work context for multi-reference generation/editing, not as the sole justification of the method or scores. Reward axes intentionally mirror evaluation dimensions—standard RL post-training practice—but training targets come from annotated banks and online LLM judges, while test metrics are computed on frozen held-out annotations; that coupling is an evaluation-validity concern, not Eq.X≡Eq.Y circularity. Human GSB/ranking and caption-conditioned reconstruction supply checks outside the automatic judge loop. No self-definitional loop, fitted-input-as-prediction, or renaming-of-known-law pattern is quotable in the manuscript.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 3 invented entities

Central claims rest on standard MLLM post-training practice plus several paper-specific modeling and evaluation choices: that phrase-after-tag supervision plus coverage-discounted GRPO rewards capture the desired grounding behavior; that LLM judges are adequate oracles for coverage, binding, and consistency; and that private multi-ref annotations generalize to held-out real/AIGC videos. Free parameters are reward weights and training hyperparameters chosen by the authors rather than derived.

free parameters (3)
  • HCD-GRPO reward weights λb, λd, λc, λqa and branch mix = λb=0.3, λd=0.4, λc=0.3, λqa=0.5, R=0.5 Rcap+0.5 Rref, τstruct=0.2
    Binding, distractor, CRSC, and QA-discount coefficients plus 0.5/0.5 cap/ref mix and structural cap τstruct=0.2 are hand-set (Appendix B.3) and directly shape the optimized policy.
  • SFT mixture ratio and LoRA/GRPO optimization hyperparameters = equal-ratio SFT; LoRA r=64 α=128; GRPO LR 1e-6; KL 0.005
    Equal multi-ref vs general-caption sampling, LoRA rank/alpha, LRs, group size (6 responses × 16 prompts), KL=0.005, etc., are design choices that affect the reported operating point.
  • Keypoint/VQA judge ordinal scores {0, 0.5, 1} = {0, 0.5, 1} ordinal mapping
    Partial-credit scales in Rcap and evaluation metrics are discrete author-chosen encodings of judge labels, not measured continuous ground truth.
axioms (5)
  • domain assumption Next-token SFT on mixed multi-reference and general captions initializes useful tag format without permanently destroying general captioning.
    Stated in §4.2 and supported empirically by VDC/VCapsBench retention, but not guaranteed a priori for other bases or mixtures.
  • ad hoc to paper Coverage-discounted rewards with DAES and CRSC are valid proxies for reference selection, phrase binding, distractor rejection, and same-entity grouping.
    Core of §4.3; ablations in Table 5 support complementarity, but the functional form is paper-specific.
  • domain assumption Structured MLLM judges (esp. Gemini-3.1-Pro) can score caption factuality and phrase–image binding reliably enough for ranking models and for GRPO advantages.
    Evaluation protocol §5.1 / Appendix A.4–A.5 and reward judges in §4.3; common in captioning eval but load-bearing here.
  • domain assumption Multi-reference captioning concerns observable semantic correspondence, not causal generation provenance of the video.
    Explicit problem formulation in §3; defines what counts as a correct tag.
  • domain assumption Standard clipped GRPO with frozen vision encoder updates only the language model sufficiently to improve grounding.
    §4.3 training setup; engineering assumption shared with many VL post-training recipes.
invented entities (3)
  • Multi-reference image-grounded video captioning task independent evidence
    purpose: Define the input/output contract: factual caption plus local <Image_i> tags with distractor omission and same-entity grouping.
    Task is newly named and specified relative to standard captioning and multi-ref generation benchmarks cited in §2.
  • HCD-GRPO (Hierarchical Coverage-Discounted GRPO) with DAES and CRSC no independent evidence
    purpose: Jointly optimize factual caption quality and multi-reference grounding via dual-branch discounted rewards.
    Paper-specific reward architecture in §4.3; evidence is internal ablations and benchmark gains, not an external physical entity.
  • MRVBench independent evidence
    purpose: Held-out benchmark for factuality plus reference selection/binding/distractor/subject-consistency metrics on real and AIGC videos.
    New evaluation resource (§5); independent as a dataset once released, though constructed by the authors.

pith-pipeline@v1.2.0-daily-grok45 · 27647 in / 3939 out tokens · 69853 ms · 2026-07-31T05:06:14.149803+00:00 · methodology

0 comments
read the original abstract

Existing video captioning models generate natural descriptions of video content but cannot explicitly ground local visual elements to multiple reference images. We introduce multi-reference image-grounded video captioning, a new task requiring factual video descriptions with phrase-level reference grounding, and propose RefCaptioner, a two-stage post-training framework for this task. RefCaptioner combines mixed-data SFT with Hierarchical Coverage-Discounted GRPO to jointly improve reference selection, phrase-level binding, distractor rejection, and cross-reference consistency while preserving general video-captioning ability. To support training, we construct a corpus containing $20,000$ videos and 171,354 reference images. We further introduce MRVBench, a benchmark for evaluating caption factuality and multi-reference grounding on both real-world and AI-generated videos. Experiments show that RefCaptioner achieves the best overall performance among the open-source models while remaining competitive on standard video captioning benchmarks. Human evaluation further confirms that its captions are preferred by annotators and enable more source-faithful video reconstruction with both open-source and proprietary video generators.

Figures

Figures reproduced from arXiv: 2607.28509 by Fengxiang Wang, Haotian Wang, Jialu Chen, Qixun Wang, Tengfei Liu, Weihong Lin, Xiaohan Zhang, Xinlei Yu, Xinlong Chen, Xinwei Long, Xuanyu Zhu, Yang Shi, Yuanxing Zhang, Yue Ding, Yujie Wei, Yuqing Wen, Yuqi Tang, Yuran Wang, Zhuoran Zhang.

Figure 1
Figure 1. Figure 1: Motivation: Video captioning models can generally describe video content clearly, but still have difficulty correctly aligning multiple reference images with the corresponding visual facts. Abstract Existing video captioning models generate natural descrip￾tions of video content but cannot explicitly ground lo￾cal visual elements to multiple reference images. We introduce multi-reference image-grounded vid… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our post-training framework. Starting from Qwen3-VL-8B, multi-reference SFT initializes the caption format and local image-tag interface. GRPO then uses a layered reward evaluator to improve reference-to-visual-unit binding. timodal models, including Tarsier and AuroraCap, gen￾erate detailed descriptions of subjects, attributes, actions, scenes, and camera changes [3, 26]. Recent work has furth… view at source ↗
Figure 3
Figure 3. Figure 3: MRVBench data construction pipeline. The pipeline includes multi-domain video collection, keyframe extraction, six-dimension [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Reference-grounding robustness under increasing num [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison on MRVBench. Baseline captions exhibit distractor intrusion, incorrect or incomplete reference binding, and inconsistent subject grouping. RefCaptioner selects the supported references and places their tags after the corresponding visual phrases. form penalizes models that are weak in any one compo￾nent. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pairwise Good/Same/Bad (GSB) evaluation compar [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 19 linked inside Pith

  1. [1]

    LLaV A-OneVision-2: Towards next-generation perceptual intelligence.arXiv preprint arXiv:2605.25979, 2026

    Xiang An et al. LLaV A-OneVision-2: Towards next-generation perceptual intelligence.arXiv preprint arXiv:2605.25979, 2026. 6, 7

    Pith/arXiv arXiv 2026

  2. [2]

    Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025. 6, 7

    Pith/arXiv arXiv 2025

  3. [3]

    Wenhao Chai, Enxin Song, Yilun Du, Chenlin Meng, Vashisht Madhavan, Omer Bar-Tal, Jenq-Neng Hwang, Sain- ing Xie, and Christopher D. Manning. AuroraCap: Efficient, performant video detailed captioning and a new benchmark. InInternational Conference on Learning Representations,

  4. [4]

    Multi-subject open-set personalization in video generation

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aber- man, Jun-Yan Zhu, Ming-Hsuan Yang, and Sergey Tulyakov. Multi-subject open-set personalization in video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6099–6110, 2025. 2

    2025

  5. [5]

    VidCapBench: A compre- hensive benchmark of video captioning for controllable text- to-video generation

    Xinlong Chen, Yuanxing Zhang, Chongling Rao, Yushuo Guan, Jiaheng Liu, Fuzheng Zhang, Chengru Song, Qiang Liu, Di Zhang, and Tieniu Tan. VidCapBench: A compre- hensive benchmark of video captioning for controllable text- to-video generation. InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 8543–8563, 2025. 3

    2025

  6. [6]

    CapRiCorn-1K: A comprehensive bench- mark for video captioning and subject referential consistency across temporal scales.arXiv preprint arXiv:2606.21949,

    Xinlong Chen, Jiafu Tang, Yue Ding, Yizhuo Jia, Bozhou Li, Bohan Zeng, Yang Shi, Shihao Li, Yiyan Ji, Qiang Liu, Weihong Lin, Yuanxing Zhang, Pengfei Wan, Liang Wang, and Tieniu Tan. CapRiCorn-1K: A comprehensive bench- mark for video captioning and subject referential consistency across temporal scales.arXiv preprint arXiv:2606.21949,

    Pith/arXiv arXiv

  7. [7]

    MA- GREF: Masked guidance for any-reference video generation with subject disentanglement

    Yufan Deng, Yuanyang Yin, Xun Guo, Yizhi Wang, Ja- cob Zhiyuan Fang, Shenghai Yuan, Yiding Yang, Angtian Wang, Bo Liu, Haibin Huang, and Chongyang Ma. MA- GREF: Masked guidance for any-reference video generation with subject disentanglement. InInternational Conference on Learning Representations, 2026. 3

    2026

  8. [8]

    ARC-Hunyuan- Video-7B: Structured video comprehension of real-world shorts.arXiv preprint arXiv:2507.20939, 2025

    Yuying Ge, Yixiao Ge, Chen Li, Teng Wang, Junfu Pu, Yizhuo Li, Lu Qiu, Jin Ma, Lisheng Duan, Xinyu Zuo, Jin- wen Luo, Weibo Gu, Zexuan Li, Xiaojing Zhang, Yangyu Tao, Han Hu, Di Wang, and Ying Shan. ARC-Hunyuan- Video-7B: Structured video comprehension of real-world shorts.arXiv preprint arXiv:2507.20939, 2025. 2, 6

    Pith/arXiv arXiv 2025

  9. [9]

    Video Re- Cap: Recursive captioning of hour-long videos

    Md Mohaiminul Islam, Ngan Ho, Xitong Yang, Tushar Na- garajan, Lorenzo Torresani, and Gedas Bertasius. Video Re- Cap: Recursive captioning of hour-long videos. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18198–18208, 2024. 3

    2024

  10. [10]

    Movie Weaver: Tuning-free multi-concept video personalization with anchored prompts

    Feng Liang, Haoyu Ma, Zecheng He, Tingbo Hou, Ji Hou, Kunpeng Li, Xiaoliang Dai, Felix Juefei-Xu, Samaneh Azadi, Animesh Sinha, Peizhao Zhang, Peter Vajda, and Di- ana Marculescu. Movie Weaver: Tuning-free multi-concept video personalization with anchored prompts. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2025

  11. [11]

    Video-LLaV A: Learning united visual repre- sentation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-LLaV A: Learning united visual repre- sentation by alignment before projection. InProceedings of the 2024 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5971–5984, 2024. 2

    2024

  12. [12]

    SwinBERT: End-to-end transformers with sparse attention for video cap- tioning

    Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, and Lijuan Wang. SwinBERT: End-to-end transformers with sparse attention for video cap- tioning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17949– 17958, 2022. 2 9

    2022

  13. [13]

    Kiwi-Edit: Versatile video editing via instruction and reference guidance.arXiv preprint arXiv:2603.02175, 2026

    Yiqi Lin, Guoqiang Liang, Ziyun Zeng, Zechen Bai, Yanzhe Chen, and Mike Zheng Shou. Kiwi-Edit: Versatile video editing via instruction and reference guidance.arXiv preprint arXiv:2603.02175, 2026. 2

    Pith/arXiv arXiv 2026

  14. [14]

    Longav-compass: Towards unified eval- uation of minute-scale audio-visual generation across t2av, i2av, and v2av.arXiv preprint arXiv:2605.26244, 2026

    Tengfei Liu, Yang Shi, Xuanyu Zhu, Jiafu Tang, Liu Yang, Qixun Wang, Zhuoran Zhang, Yuqi Tang, Fengxiang Wang, Yuhao Dong, et al. Longav-compass: Towards unified eval- uation of minute-scale audio-visual generation across t2av, i2av, and v2av.arXiv preprint arXiv:2605.26244, 2026. 2

    Pith/arXiv arXiv 2026

  15. [15]

    ReBind: Multi-reference video editing via structured instruc- tions with explicit reference relationships.arXiv preprint arXiv:2607.14681, 2026

    Xinyu Liu, Shihao Li, Weihong Lin, Xinlong Chen, Yang Shi, Yujin Han, Yiyang Cai, Yanghao Wang, Ruibin Yuan, Yuanxing Zhang, Pengfei Wan, Wenhan Luo, and Yike Guo. ReBind: Multi-reference video editing via structured instruc- tions with explicit reference relationships.arXiv preprint arXiv:2607.14681, 2026. 2

    Pith/arXiv arXiv 2026

  16. [16]

    UniVL: A unified video and language pre-training model for multimodal understanding and generation.arXiv preprint arXiv:2002.06353, 2020

    Huaishao Luo, Lei Ji, Botian Shi, Haoyang Huang, Nan Duan, Tianrui Li, Jason Li, Taroon Bharti, and Ming Zhou. UniVL: A unified video and language pre-training model for multimodal understanding and generation.arXiv preprint arXiv:2002.06353, 2020. 2

    Pith/arXiv arXiv 2002

  17. [17]

    Video-ChatGPT: Towards detailed video un- derstanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-ChatGPT: Towards detailed video un- derstanding via large vision and language models. InPro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585–12602, 2024. 2

    2024

  18. [18]

    Mavors: Multi-granularity video representation for multimodal large language model

    Yang Shi, Jiaheng Liu, Yushuo Guan, Zhenhua Wu, Yuanx- ing Zhang, Zihao Wang, Weihong Lin, Jingyun Hua, Zekun Wang, Xinlong Chen, et al. Mavors: Multi-granularity video representation for multimodal large language model. InPro- ceedings of the 33rd ACM International Conference on Mul- timedia, pages 10994–11003, 2025. 2

    2025

  19. [19]

    Mme-videoocr: Evaluating ocr-based capabilities of multimodal llms in video scenarios.Advances in Neural Information Processing Systems, 38, 2026

    Yang Shi, Huanqian Wang, Xie Xie, Huanyao Zhang, Lijie Zhao, Xinfeng Li, Chaoyou Fu, Zhuoer Wen, Wenting Liu, Zhuoran Zhang, et al. Mme-videoocr: Evaluating ocr-based capabilities of multimodal llms in video scenarios.Advances in Neural Information Processing Systems, 38, 2026. 2

    2026

  20. [20]

    MV-S2V: Multi-view subject-consistent video generation

    Ziyang Song, Xinyu Gong, Bangya Liu, and Zelin Zhao. MV-S2V: Multi-view subject-consistent video generation. arXiv preprint arXiv:2601.17756, 2026. 2

    Pith/arXiv arXiv 2026

  21. [21]

    VideoBERT: A joint model for video and language representation learning

    Chen Sun, Austin Myers, Carl V ondrick, Kevin Murphy, and Cordelia Schmid. VideoBERT: A joint model for video and language representation learning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7464–7473, 2019. 2

    2019

  22. [22]

    Keyframe-compass: Towards comprehen- sive evaluation of keyframe-conditioned video generation

    Yuqi Tang, Tengfei Liu, Yizheng Lai, Yuran Wang, Yang Shi, Wanshun Su, Zhuoran Zhang, Qixun Wang, Xiaohan Zhang, Xinlei Yu, et al. Keyframe-compass: Towards comprehen- sive evaluation of keyframe-conditioned video generation. arXiv preprint arXiv:2607.14202, 2026. 2

    Pith/arXiv arXiv 2026

  23. [23]

    Artifact-bench: Evaluating mllms on detecting and assessing the artifacts of ai-generated videos

    Yuqi Tang, Yang Shi, Zhuoran Zhang, Qixun Wang, Xuehai Bai, Yue Ding, Ruizhe Chen, Bohan Zeng, Xinlong Chen, Xuanyu Zhu, et al. Artifact-bench: Evaluating mllms on detecting and assessing the artifacts of ai-generated videos. arXiv preprint arXiv:2605.18984, 2026. 2

    Pith/arXiv arXiv 2026

  24. [24]

    Qwen3.6-35B-A3B: Agentic coding power, now open to all, 2026

    Qwen Team. Qwen3.6-35B-A3B: Agentic coding power, now open to all, 2026. 6

    2026

  25. [25]

    Qwen3.6-27B: Flagship-level coding in a 27B dense model, 2026

    Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model, 2026. 6

    2026

  26. [26]

    Tarsier: Recipes for training and evaluating large video description models.arXiv preprint arXiv:2407.00634, 2024

    Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training and evaluating large video description models.arXiv preprint arXiv:2407.00634, 2024. 3

    Pith/arXiv arXiv 2024

  27. [27]

    Monet: Reasoning in latent visual space beyond images and language.arXiv preprint arXiv:2511.21395, 2025

    Qixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang, Pengfei Wan, Kun Gai, Xianghua Ying, and Yisen Wang. Monet: Reasoning in latent visual space beyond images and language.arXiv preprint arXiv:2511.21395, 2025. 2

    arXiv 2025

  28. [28]

    InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 6, 7

    Pith/arXiv arXiv 2025

  29. [29]

    VaTeX: A large-scale, high- quality multilingual dataset for video-and-language research

    Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. VaTeX: A large-scale, high- quality multilingual dataset for video-and-language research. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4581–4591, 2019. 3

    2019

  30. [30]

    Intern- Video2: Scaling foundation models for multimodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, Tianxiang Jiang, Songze Li, Jilan Xu, Hongjie Zhang, Yifei Huang, Yu Qiao, Yali Wang, and Limin Wang. Intern- Video2: Scaling foundation models for multimodal video understanding. InComputer Vision – ECCV 2024, pages 396–416. Springer Nature S...

    2024

  31. [31]

    LongCaptioning: Unlocking the power of long video caption generation in large multimodal models.arXiv preprint arXiv:2502.15393, 2025

    Hongchen Wei, Zhihong Tan, Yaosi Hu, Chang Wen Chen, and Zhenzhong Chen. LongCaptioning: Unlocking the power of long video caption generation in large multimodal models.arXiv preprint arXiv:2502.15393, 2025. 3

    Pith/arXiv arXiv 2025

  32. [32]

    MiMo-VL technical report

    LCT Xiaomi and Core Team. MiMo-VL technical report. arXiv preprint arXiv:2506.03569, 1(2):5, 2025. 6, 7

    Pith/arXiv arXiv 2025

  33. [33]

    LumosX: Relate any identities with their attributes for personalized video generation

    Jiazheng Xing, Fei Du, Hangjie Yuan, Pengwei Liu, Hong- bin Xu, Hai Ci, Ruigang Niu, Weihua Chen, Fan Wang, and Yong Liu. LumosX: Relate any identities with their attributes for personalized video generation. InInternational Confer- ence on Learning Representations, 2026. 2

    2026

  34. [34]

    MSR-VTT: A large video description dataset for bridging video and lan- guage

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. MSR-VTT: A large video description dataset for bridging video and lan- guage. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5288–5296, 2016. 3

    2016

  35. [35]

    Progress-aware video frame captioning

    Zihui Xue, Joungbin An, Xitong Yang, and Kristen Grau- man. Progress-aware video frame captioning. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13639–13650, 2025. 3

    2025

  36. [36]

    Kwai Keye-VL 1.5 technical report.arXiv preprint arXiv:2509.01563, 2025

    Biao Yang et al. Kwai Keye-VL 1.5 technical report.arXiv preprint arXiv:2509.01563, 2025. 2, 6, 7

    Pith/arXiv arXiv 2025

  37. [37]

    OpenS2V-Nexus: A detailed benchmark and million-scale dataset for subject- to-video generation.arXiv preprint arXiv:2505.20292, 2025

    Shenghai Yuan, Xianyi He, Yufan Deng, Yang Ye, Jinfa Huang, Bin Lin, Jiebo Luo, and Li Yuan. OpenS2V-Nexus: A detailed benchmark and million-scale dataset for subject- to-video generation.arXiv preprint arXiv:2505.20292, 2025. 3

    Pith/arXiv arXiv 2025

  38. [38]

    VideoLLaMA 3: Frontier mul- 10 timodal foundation models for image and video understand- ing.arXiv preprint arXiv:2501.13106, 2025

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Li- dong Bing, and Deli Zhao. VideoLLaMA 3: Frontier mul- 10 timodal foundation models for image and video understand- ing.arXiv preprint arXiv:2501.13106, 2025. 6

    Pith/arXiv arXiv 2025

  39. [39]

    Video-LLaMA: An instruction-tuned audio-visual language model for video un- derstanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-LLaMA: An instruction-tuned audio-visual language model for video un- derstanding. InProceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing: System Demonstrations, pages 543–553, 2023. 2

    2023

  40. [40]

    VCapsBench: A large-scale fine-grained benchmark for video caption quality evaluation

    Shi-Xue Zhang, Hongfa Wang, Duojun Huang, Xin Li, Xi- aobin Zhu, and Xu-Cheng Yin. VCapsBench: A large-scale fine-grained benchmark for video caption quality evaluation. InProceedings of the AAAI Conference on Artificial Intelli- gence, pages 12726–12734, 2026. 3

    2026

  41. [41]

    MultiRef-Compass: Towards comprehensive evaluation of multi-reference-to-audio-video generation.arXiv preprint arXiv:2607.14189, 2026

    Xiaohan Zhang, Yuqing Wen, Junlin Chen, Yuqi Tang, Yiting He, Lizhuo Shao, Weiming Zhu, Tengfei Liu, Yang Shi, Jialu Chen, Yuanxing Zhang, and Huaxiong Li. MultiRef-Compass: Towards comprehensive evaluation of multi-reference-to-audio-video generation.arXiv preprint arXiv:2607.14189, 2026. 3

    Pith/arXiv arXiv 2026

  42. [42]

    Debiasing multimodal large language models via penal- ization of language priors

    YiFan Zhang, Yang Shi, Weichen Yu, Qingsong Wen, Xue Wang, Wenjing Yang, Zhang Zhang, Liang Wang, and Rong Jin. Debiasing multimodal large language models via penal- ization of language priors. InProceedings of the 33rd ACM International Conference on Multimedia, pages 4232–4241,

  43. [43]

    from <Image_N>

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Zi- wei Liu, and Chunyuan Li. LLaV A-Video: Video instruction tuning with synthetic data.Transactions on Machine Learn- ing Research, 2025. 6 11 RefCaptioner: Multi-Reference Image-Grounded Video Captioning Supplementary Material A. MRVBench Construction and Evaluation A.1. Benchmark Overview MRVBench co...

    2025