Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

KeyFrame-Compass, a 386-sample benchmark, claims that current video generators trade faithful keyframe reproduction for natural motion — and that keyframe execution and general quality are separate abilities that must be scored independentl

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 02:50 UTC pith:Z4CTYFEQ

load-bearing objection Useful benchmark with real novelty and careful calibration hygiene, but the six keyframe metrics all inherit an unvalidated LLM shot-segmentation step; fix that before trusting the rankings. the 3 major comments →

arxiv 2607.14202 v1 pith:Z4CTYFEQ submitted 2026-07-15 cs.CV

KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation

Yuqi Tang , Tengfei Liu , Yizheng Lai , Yuran Wang , Yang Shi , Wanshun Su , Zhuoran Zhang , Qixun Wang
show 13 more authors
Xiaohan Zhang Xinlei Yu Xuehai Bai Xuanyu Zhu Bohan Zeng Bozhou Li Shujie Li Yifan Dai Yujie Wei Shixuan Liu Haotian Wang Jialu Chen Yuanxing Zhang
This is my paper
classification cs.CV
keywords keyframe-conditioned video generationvideo generation benchmarkkeyframe fidelitytemporal organizationmultimodal LLM evaluationstoryboard gridcontrollabilityvideo quality assessment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that keyframe execution and overall video quality are different abilities, that neither can substitute for the other in evaluation, and that current video generators fail them in a systematic way. It introduces KeyFrame-Compass, a benchmark of 386 curated samples in which a model must reproduce an ordered sequence of reference images inside a generated video, varied across three domains, two shot structures, four keyframe densities, two prompt granularities, and two input formats. Evaluation is split into six keyframe-response metrics (presence, fidelity, ordering, timing, persistence, uniqueness) and four quality groups, so that a failure can be localized to a specific ability. On nine commercial and open-source systems the benchmark's headline finding is a trade-off: models that follow the keyframes closely connect them with abrupt, slideshow-like transitions, while models that generate natural motion drift from the keyframes, and no system does well on both axes. A sympathetic reader should care because keyframe-first production is becoming a standard workflow, and the paper's claim is that we now have an instrument that can say precisely which ability is missing in a given model.

Core claim

The central claim is that keyframe execution and general video quality are non-interchangeable capabilities, and that only a benchmark scoring them separately can diagnose modern video generators. KeyFrame-Compass's 386 samples each pair an ordered keyframe sequence with a text prompt, spread across five controlled factors; an automated pipeline recovers each generated video's actual shot structure, matches keyframes inside expected temporal windows using semantic (DINOv3) and pixel (PSNR/SSIM) criteria, and yields six keyframe-response metrics alongside four evidence-grounded quality groups. Evaluating nine systems, the authors find a persistent trade-off: the model with the strongest keyfr

What carries the argument

The load-bearing mechanism is the keyframe matching pipeline. Because generated videos need not preserve the requested one-keyframe-per-shot structure, an MLLM (Gemini 3.1 Pro) first recovers the video's actual shot structure and assigns each keyframe an expected temporal window; candidate frames inside the window are filtered by DINOv3 semantic similarity (threshold 0.80) plus at least one pixel criterion (PSNR ≥ 12 dB or SSIM ≥ 0.65), and the best semantic match is canonical. All six keyframe-response metrics derive from these matches: presence, fidelity, position accuracy, order (Kendall's τ), persistence, and uniqueness. General quality is judged by an evidence-grounded protocol: GPT-5.5

Load-bearing premise

The load-bearing premise is that the automatic shot-segmentation pass (Gemini 3.1 Pro's 'Actual Video Structure Discovery') finds the true temporal structure of each generated video: all six keyframe metrics are computed inside the temporal windows it assigns, yet its accuracy is never measured against human-annotated shot boundaries, and the paper's human validation (Section 4.4) covers the four quality groups, not these six core metrics.

What would settle it

Have human annotators mark true shot boundaries and keyframe visibility on a stratified sample of the benchmark's generated videos (one-take and multi-shot), then recompute hit rate, similarity, position accuracy, persistence, and uniqueness using human-derived windows instead of LLM-derived windows; material disagreement would show the six-metric decomposition misattributes model behavior. The trade-off claim would additionally be falsified by a single system ranking first in both keyframe fidelity and general quality on the same 115-sample common set.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Keyframe execution and overall video quality must be reported as separate axes: the paper's rank reversals (LTX-2.3 first in keyframe fidelity but fourth overall; Gemini-Omni-Flash the reverse) show that a single aggregate score masks which ability is failing.
  • The remaining bottleneck for strong keyframe-following models is transition synthesis — generating plausible, physically coherent motion between anchors — rather than anchor reproduction.
  • Denser keyframe constraints reduce instruction adherence: visual instruction adherence drops from 0.849 with 3 keyframes to 0.756 with 9–12 keyframes, with the decline concentrated in video compliance, not audio.
  • Open-source models have not acquired storyboard-grid comprehension: hit rates below 0.28 and static grid-collage outputs indicate a training-data gap rather than a capacity limit.
  • Models differ systematically in whether they treat keyframes as strict visual anchors (Seedance 2.0) or loose semantic references (Gemini-Omni-Flash); applications must choose which behavior they need.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the trade-off is intrinsic to current training objectives (perceptual smoothness versus conditioning pressure), a testable next step is a decoupled architecture — one module enforcing anchor alignment, another generating inter-keyframe dynamics — which should beat both extremes on the joint axis this benchmark defines.
  • Because every keyframe metric inherits the LLM's shot-segmentation windows, the benchmark's most direct extension is a human audit of those windows: if segmentation disagrees with human-annotated boundaries on even a small stratified sample, per-metric scores would need re-derivation rather than recalibration.
  • The density-degradation finding suggests a concrete training intervention — curriculum schedules over keyframe count, or conditioning dropout at high densities — whose effect could be measured directly on the 3/6/9/12 stratification the benchmark ships with.
  • The storyboard-grid failure in open-source models could be probed further with a minimal synthetic test: a two-panel grid in known temporal order, where success requires more than copying the layout, isolating grid comprehension from keyframe reproduction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript introduces KeyFrame-Compass, a benchmark and automated evaluation protocol for multi-keyframe-conditioned video generation. It curates 386 test cases with controlled variation over application domain, video structure, keyframe count (3/6/9/12), prompt granularity, and visual input format (multi-image list vs. storyboard grid). The evaluation pipeline decomposes keyframe execution into six metrics -- Hit Rate, Keyframe Similarity, Order Consistency, Position Accuracy, Persistence, and Response Uniqueness -- via an MLLM-based shot-segmentation step followed by DINOv3/PSNR/SSIM matching, and separately scores four general-quality groups with evidence-grounded MLLM judges plus perception-model auxiliary signals. Nine video generation systems are evaluated. The main empirical claims are a fidelity/naturalness trade-off, degradation under denser keyframe constraints, and open-source models' failure to interpret storyboard-grid inputs as temporally ordered keyframe sequences. Human alignment is reported for the general-quality groups.

Significance. If the keyframe-response metrics are valid, this is a timely and useful benchmark for a growing production workflow, and the controlled metadata enable genuinely stratified analysis. The calibration hygiene is commendable: the DINOv3 threshold is fixed at an equal-error point on a held-out 1,027-pair human-labeled set, the KFS pixel parameters are swept with rank-stable results (minimum Spearman 0.997), and five repeated MLLM judge runs give standard deviations near 0.001. However, the paper's central quantitative claims rest on the six keyframe-response metrics, and those metrics are not independently validated -- in particular, the MLLM shot-segmentation step that defines the temporal windows for four of the six metrics has no human audit, and the only human-alignment study covers general quality, not keyframe response. The significance is therefore conditional on closing that validation gap.

major comments (3)
  1. [§3.3, Fig. 4, App. E.2] The load-bearing validity gap is that HR, KFS, KPA, and PAK are defined inside the 'expected temporal window' produced by Gemini 3.1 Pro's Actual Video Structure Discovery (Eqs. 1-3, 5). This segmentation is never validated: no human-annotated shot-boundary ground truth, no audit of assigned windows, no inter-annotator reliability. A spurious split or missed cut directly changes HR (match/no-match), KFS (which candidates are in-window), KPA (target vs. matched timestamp), and PAK (response interval). The qualitative grid/morphing failures in Fig. 6 are independent support, but the quantitative six-metric decomposition inherits the unmeasured segmentation step. Please add a human-annotated boundary/keyframe-assignment study on a stratified subset, report boundary precision/recall and window accuracy, and show whether leaderboard conclusions survive when oracle human windows are substitute
  2. [§4.4, Table 6, App. D] The only human-alignment analysis validates the four general-quality groups (Video Quality, Spatiotemporal Coherence, Instruction Adherence, A/V Coordination). The six keyframe-response metrics -- HR, KFS, KOC, KPA, PAK, RU -- which are the paper's core contribution, are never compared against human judgments. Since the headline trade-off (e.g., LTX-2.3 high KFS / low DVQ vs. Gemini low KFS / high DVQ in Table 3) and the density/storyboard-grid findings in Tables 4-5 are computed from these metrics, the central empirical claims need a human agreement study for the keyframe-response dimensions as well, not only for general quality.
  3. [§3.3, Eqs. (5)-(6)] PAK and RU are not fully specified. The flash and freeze scores q_flash_i and q_freeze_i in Eq. (5) are described only verbally; there is no mapping from response duration/temporal variation to the scores. In Eq. (6), the clustering gap is g = max(g_floor, T/N_kf * r), but the values of g_floor and r are never given. Without these definitions, PAK and RU are not reproducible and their reported values in Tables 3-4 cannot be independently checked. Please provide exact formulas, parameter values, and reference pseudocode or a released implementation.
minor comments (4)
  1. [Fig. 4 vs. App. B] The matching-pipeline figure shows an SSIM threshold of 0.4 in its illustrative example, while Appendix B sets tau_s = 0.65. Align the figure and appendix, or state explicitly that the figure is illustrative only.
  2. [Eq. (8)] The audio fusion weight lambda_audio is called 'predefined' but its value is not reported. Please give the value or range used for the reported results.
  3. [Table 3 and Table 4] The Overall aggregation multiplies Keyframe Fidelity and Temporal Organization before averaging groups, and short-video scores use different sample intersections for proprietary and open-source models. This is a consequential design choice; report a sensitivity analysis with simple averaging and state exact sample sizes for each table block.
  4. [§3.3, General Quality Metrics] Because Gemini 3.1 Pro serves as the segmentation/quality judge while a Gemini-family model is evaluated, add a cross-judge robustness check on a subset using an independent non-Gemini judge to rule out systematic family bias.

Circularity Check

0 steps flagged

No significant circularity: the benchmark metrics are defined independently of the evaluated models, thresholds are fixed before test evaluation, and the central trade-off finding rests on model outputs rather than on fitted inputs or self-citations.

full rationale

I found no circular step that reduces a claimed prediction to its inputs. The keyframe-response metrics (Eqs. 1-6) are computed from a matching pipeline (Fig. 4, Sec. 3.3) that is fixed before evaluation. The only fitted quantity, the DINOv3 semantic threshold γ*=0.80, is calibrated on a held-out validation set in App. B ('The threshold is fixed before test evaluation') and does not determine the model rankings; it is a scorer-calibration constant, not a model parameter. Representative-keyframe KPA's binary rule is scoring the intended property (appearance inside the assigned segment), not defining it into existence. The expected temporal windows produced by Gemini 3.1 Pro's 'Actual Video Structure Discovery' are a potential validity risk because they are not human-audited, and this risk propagates to HR/KFS/KPA/PAK; however, that is an unmeasured-assumption problem, not an identity of input and output. The judged model (Gemini-Omni-Flash) and the judge/segmentation model (Gemini 3.1 Pro) are distinct systems, so this is methodological adjacency rather than self-judgment. Self-citations such as [5, 21, 37] appear only as related-work context in Sec. 2.2 and are not used as load-bearing evidence for the central benchmark or the trade-off result. Sec. 4.4 provides independent human alignment for general-quality dimensions, and App. B/C provide stability checks. No Eq. X is equivalent to Eq. Y by construction, and no fitted parameter is renamed as a prediction; therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The benchmark's two load-bearing free choices are the DINOv3/PSNR/SSIM matching thresholds (honestly calibrated on held-out data and frozen) and the hand-chosen fusion weights. Two metrics (PAK, RU) contain unspecified parameters that a replicator must guess, and the audio fusion weight is unreported. Everything else is assembled from existing models and standard statistics; no new physical or theoretical entities are introduced.

free parameters (6)
  • Matching thresholds τd, τp, τs = τd=0.80 (DINOv3), τp=12 dB (PSNR), τs=0.65 (SSIM)
    Calibrated on a held-out 1,027-pair human-labeled validation set at the equal-error point and frozen before test evaluation (App. B). Honest calibration, but these thresholds determine the matched set M and hence HR, KFS, KPA, and PAK.
  • KFS fusion weights w_pix, w_dino = w_pix=0.4, w_dino=0.6
    Hand-chosen; Eq. 2. The sensitivity analysis sweeps only pixel-score parameters, not these weights.
  • PSNR/SSIM normalization anchors = PSNR 12 dB lower / 50 dB saturation; SSIM 0.65 lower / 1.0 saturation
    Set empirically (App. B). The KFS sensitivity sweep shows mean shift ≤0.0244 within tested ranges.
  • PAK flash/freeze scoring (q_flash, q_freeze)
    Referenced in §3.3 but the functional forms, response-interval selection rules, and thresholds are never given, making the metric non-replicable as specified.
  • RU clustering gap g = max(g_floor, T/Nkf·r)
    The parameter r and g_floor are never assigned values in the paper, so Response Uniqueness cannot be re-implemented exactly.
  • Audio fusion weight λ_audio
    Introduced in Eq. 8 for combining Gemini and CLAP audio-adherence scores; the numerical value is not reported.
axioms (5)
  • standard math Cosine similarity and Kendall's tau are valid aggregation tools for the keyframe metrics
    Used in Eqs. 1–6; unproblematic background mathematics.
  • domain assumption DINOv3 global-average-pooled cosine similarity is a valid semantic match signal for keyframe-to-video-frame correspondence
    Invoked in §3.3 following [17]; the transfer of an object-retrieval method to frame-level keyframe matching is assumed, not ablated against alternative descriptors.
  • domain assumption Gemini 3.1 Pro shot segmentation and keyframe assignment (App. E.2) are accurate enough to serve as ground truth for expected temporal windows
    Load-bearing and unvalidated: no human-annotated shot-boundary comparison, no window audit, no inter-annotator check reported.
  • domain assumption Checklist-based MLLM scoring tracks human judgments
    Partially validated in §4.4: item-level agreement 0.729–0.840, Krippendorff's α median 0.29–0.74 across four quality dimensions; A-V Coordination is the weakest.
  • domain assumption AI-generated keyframes (GPT-Image-2, Nano Banana Pro) are valid, internally consistent visual anchors
    Data construction includes MLLM filtering and human review, but acceptance statistics (annotator count, inter-rater agreement on keyframe quality/identity) are not reported.

pith-pipeline@v1.3.0-alltime-deepseek · 36472 in / 18880 out tokens · 173739 ms · 2026-08-02T02:50:44.902478+00:00 · methodology

0 comments
read the original abstract

Video generation increasingly relies on keyframe-based workflows, where creators specify a sequence of reference images to guide generation. Although recent models support multi-keyframe conditioning, it remains unclear whether they can faithfully reproduce the prescribed keyframes while maintaining overall video quality. We present KeyFrame-Compass, the first comprehensive benchmark for evaluating keyframe-conditioned video generation. The benchmark contains 386 carefully curated samples spanning three application domains, two video structures, two prompt granularities, two conditioning formats, and four keyframe densities, enabling controlled analysis under diverse generation settings. We further introduce an automated evaluation framework that jointly measures keyframe execution and overall video quality. Specifically, we decompose keyframe execution into six complementary metrics covering presence, fidelity, temporal ordering, localization, persistence, and uniqueness, while assessing overall video quality through evidence-grounded MLLM judgments augmented with specialized perception models. Experiments on nine representative video generation systems reveal several fundamental limitations. Current models exhibit a clear trade-off between faithful keyframe execution and natural video synthesis. Their performance further degrades as keyframe constraints become denser and most open-source models also fail to interpret storyboard-grid inputs as temporally ordered keyframe sequences.

Figures

Figures reproduced from arXiv: 2607.14202 by Bohan Zeng, Bozhou Li, Haotian Wang, Jialu Chen, Qixun Wang, Shixuan Liu, Shujie Li, Tengfei Liu, Wanshun Su, Xiaohan Zhang, Xinlei Yu, Xuanyu Zhu, Xuehai Bai, Yang Shi, Yifan Dai, Yizheng Lai, Yuanxing Zhang, Yujie Wei, Yuqi Tang, Yuran Wang, Zhuoran Zhang.

Figure 1
Figure 1. Figure 1: Illustration of KeyFrame-Compass. The benchmark spans three application domains (daily capture, product visualization, and cinematic narrative) and covers both realistic and stylized visual content. Each test case provides an ordered set of keyframes that the generated video must reproduce at the intended moments. Abstract Video generation increasingly relies on keyframe-based workflows, where creators spe… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of KeyFrame-Compass. Benchmark Design: KeyFrame-Compass evaluates keyframe-conditioned video generation from a text prompt and an ordered set of keyframes, covering diverse domains. Evaluation Framework: Keyframe response is assessed by a staged matching pipeline that combines MLLM-assisted shot segmentation with pixel-level and semantic matching to locate keyframe occurrences within exact tempora… view at source ↗
Figure 3
Figure 3. Figure 3: Data construction of KeyFrame-Compass. Narrative sources are curated and converted into structured story summaries, which are transformed into scene specifications for keyframe generation and video prompt construction. Candidate samples are then filtered through multimodal verification, human review, and safety screening to ensure quality, consistency, specification compliance, and safety. 2 [25] and Nano … view at source ↗
Figure 4
Figure 4. Figure 4: Keyframe matching pipeline of KeyFrame-Compass. Since generated videos may not preserve the assumed one-keyframe￾per-shot structure, we first use Gemini 3.1 Pro to recover the actual shot structure and assign input keyframes to the corresponding generated segments. Expected temporal windows are then derived for one-to-one, multi-to-one, and unassigned cases. Within each window, candidate frames are filtere… view at source ↗
Figure 5
Figure 5. Figure 5: Dataset statistics of KeyFrame-Compass. From left to right: the scene- type word cloud, the distribution of characters per story, the distribution of keyframe counts across one-take and multi-shot samples, and the co-occurrence patterns between camera motions and shot scales. These statistics summarize the diversity of story structure, scene semantics, and camera-control requirements in the benchmark [PIT… view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: An example of a fable under minimal prompt. All models consistently preserve the visual style and closely follow the intended [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: An example of a cinematic story under minimal prompt. The outputs of all models generally reproduce the shot composition and [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: An example of a cinematic story under segment-specific prompt. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: An example of a commercial under segment-specific prompt. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: An example of a commercial advertisement under segment-specific prompt. This sample features rapid scene transitions [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: An example under segment-specific prompt. In this sample, the visual differences between scenes corresponding to different [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: An example under minimal prompt. Some models produce abnormal and irrelevant content in this sample. Surprisingly, [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: An example under minimal prompt. LTX 2.3 suffers from repeated texture artifacts, but it still achieves the best overall [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RefCaptioner: Multi-Reference Image-Grounded Video Captioning

    cs.CV 2026-07 conditional novelty 6.0

    Mixed-data SFT plus Hierarchical Coverage-Discounted GRPO yields open-source SOTA multi-reference image-grounded video captions on MRVBench without hurting general captioning.

Reference graph

Works this paper leans on

78 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Wan image-to-video api reference

    Alibaba Cloud. Wan image-to-video api reference. https://www.alibabacloud.com/help/en/ model-studio/image-to-video-general- api-reference, 2026. Accessed: 2026-06-23. 9

    2026

  2. [2]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025. 2

    Pith/arXiv arXiv 2025

  3. [3]

    Elasticface: Elastic margin loss for deep face recognition, 2022

    Fadi Boutros, Naser Damer, Florian Kirchbuchner, and Ar- jan Kuijper. Elasticface: Elastic margin loss for deep face recognition, 2022. 8

    2022

  4. [4]

    VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Condition- ing

    Minghong Cai, Qiulin Wang, Zongli Ye, Wenze Liu, Quande Liu, Weicai Ye, Xintao Wang, Pengfei Wan, Kun Gai, and Xiangyu Yue. VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Condition- ing. 2, 3, 4

  5. [5]

    T2av-compass: Towards unified evaluation for text- to-audio-video generation

    Zhe Cao, Tao Wang, Jiaming Wang, Yanghai Wang, Yuanx- ing Zhang, Jialu Chen, Miao Deng, Jiahao Wang, Yubin Guo, Chenxi Liao, Yize Zhang, Zhaoxiang Zhang, and Jiaheng Liu. T2av-compass: Towards unified evaluation for text- to-audio-video generation. InInternational Conference on Machine Learning (ICML), 2026. 3

    2026

  6. [6]

    Skyreels- v2: Infinite-length film generative model, 2025

    Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma, Weiming Xiong, Wei Wang, Nuo Pang, Kang Kang, Zhiheng Xu, Yuzhe Jin, Yupeng Liang, Yubing Song, Peng Zhao, Boyuan Xu, Di Qiu, De- 13 bang Li, Zhengcong Fei, Yang Li, and Yahui Zhou. Skyreels- v2: Infinite-length film generative...

    2025

  7. [7]

    Cinedance: Towards next- generation multi-shot long-form cinematic audio-video gen- eration, 2026

    Yuheng Chen, Teng Hu, Yuji Wang, Qingdong He, Zhu- cun Xue, Qianyu Zhou, Jason Li, Lizhuang Ma, Jiangn- ing Zhang, and Dacheng Tao. Cinedance: Towards next- generation multi-shot long-form cinematic audio-video gen- eration, 2026. 4

    2026

  8. [8]

    Aigcbench: Comprehensive evaluation of image-to-video content generated by ai.BenchCouncil Transactions on Benchmarks, Standards and Evaluations, page 100152,

    Fanda Fan, Chunjie Luo, Wanling Gao, and Jianfeng Zhan. Aigcbench: Comprehensive evaluation of image-to-video content generated by ai.BenchCouncil Transactions on Benchmarks, Standards and Evaluations, page 100152,

  9. [9]

    Tc-bench: Benchmark- ing temporal compositionality in text-to-video and image-to- video generation, 2024

    Weixi Feng, Jiachen Li, Michael Saxon, Tsu jui Fu, Wenhu Chen, and William Yang Wang. Tc-bench: Benchmark- ing temporal compositionality in text-to-video and image-to- video generation, 2024. 2, 3

    2024

  10. [10]

    ImageBind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. ImageBind: One embedding space to bind them all. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 8

    2023

  11. [11]

    Introducing nano banana pro.https:// blog.google/technology/ai/nano-banana- pro/, 2025

    Google. Introducing nano banana pro.https:// blog.google/technology/ai/nano-banana- pro/, 2025. . 5

    2025

  12. [12]

    Gemini 3.1 pro model card.https: //deepmind.google/models/model-cards/ gemini-3-1-pro/, 2026

    Google DeepMind. Gemini 3.1 pro model card.https: //deepmind.google/models/model-cards/ gemini-3-1-pro/, 2026. Accessed: 2026-06. 2, 5

    2026

  13. [13]

    Gemini omni flash model card.https: //deepmind.google/models/model-cards/ gemini-omni-flash/, 2026

    Google DeepMind. Gemini omni flash model card.https: //deepmind.google/models/model-cards/ gemini-omni-flash/, 2026. Accessed: 2026-06-23. 9

    2026

  14. [14]

    Ltx-video: Realtime video latent diffusion.arXiv preprint arXiv:2501.00103,

    Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion.arXiv preprint arXiv:2501.00103,

    Pith/arXiv arXiv

  15. [15]

    Vabench: A comprehensive benchmark for audio- video generation, 2026

    Daili Hua, Xizhi Wang, Bohan Zeng, Xinyi Huang, Hao Liang, Junbo Niu, Xinlong Chen, Quanqing Xu, and Wentao Zhang. Vabench: A comprehensive benchmark for audio- video generation, 2026. 3

    2026

  16. [16]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Reco...

    2024

  17. [17]

    Hudson, Ye Xia, Skanda Koppula, Andre Araujo, Joao Carreira, and Niloy J

    Rishabh Kabra, Maks Ovsjanikov, Drew A. Hudson, Ye Xia, Skanda Koppula, Andre Araujo, Joao Carreira, and Niloy J. Mitra. A mixed diet makes dino an omnivorous vision en- coder, 2026. 6

    2026

  18. [18]

    Musiq: Multi-scale image quality transformer,

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer,

  19. [19]

    DreaMontage: Arbitrary Frame-Guided One-Shot Video Generation

    Jiawei Liu, Junqiao Li, Jiangfan Deng, Gen Li, Siyu Zhou, Zetao Fang, Shanshan Lao, Zengde Deng, Jianing Zhu, Tingting Ma, Jiayi Li, Yunqiu Wang, Qian He, and Xing- long Wu. DreaMontage: Arbitrary Frame-Guided One-Shot Video Generation. 2, 3

  20. [20]

    Javisdit: Joint audio-video diffusion trans- former with hierarchical spatio-temporal prior synchroniza- tion

    Kai Liu, Wei Li, Lai Chen, Shengqiong Wu, Yanhao Zheng, Jiayi Ji, Fan Zhou, Jiebo Luo, Ziwei Liu, Hao Fei, and Tat-Seng Chua. Javisdit: Joint audio-video diffusion trans- former with hierarchical spatio-temporal prior synchroniza- tion. 2026. 8

    2026

  21. [21]

    Longav-compass: To- wards unified evaluation of minute-scale audio-visual gener- ation across t2av, i2av, and v2av, 2026

    Tengfei Liu, Yang Shi, Xuanyu Zhu, Jiafu Tang, Liu Yang, Qixun Wang, Zhuoran Zhang, Yuqi Tang, Fengxiang Wang, Yuhao Dong, Xinlong Chen, Bozhou Li, Bohan Zeng, Yue Ding, Xiaohan Zhang, Jialu Chen, Haotian Wang, Yuanxing Zhang, Pengfei Wan, and Leye Wang. Longav-compass: To- wards unified evaluation of minute-scale audio-visual gener- ation across t2av, i2...

    2026

  22. [22]

    Evalcrafter: Benchmarking and evaluating large video generation models.arXiv preprint arXiv:2310.11440, 2023

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Ray- mond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models.arXiv preprint arXiv:2310.11440, 2023. 2, 3

    Pith/arXiv arXiv 2023

  23. [23]

    SAM 3.1: Faster and more accessible real-time video segmentation.https://ai.meta.com/blog/ segment-anything-model-3/, 2026

    Meta AI. SAM 3.1: Faster and more accessible real-time video segmentation.https://ai.meta.com/blog/ segment-anything-model-3/, 2026. Accessed: 2026-06-27. 7

    2026

  24. [24]

    A corpus and evaluation framework for deeper understanding of commonsense stories, 2016

    Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and evaluation framework for deeper understanding of commonsense stories, 2016. 4

    2016

  25. [25]

    Introducing chatgpt images 2.0.https: //openai.com/index/introducing-chatgpt- images-2-0/, 2026

    OpenAI. Introducing chatgpt images 2.0.https: //openai.com/index/introducing-chatgpt- images-2-0/, 2026. Accessed: 2026-06-18. 5

    2026

  26. [26]

    Gpt-5.5 system card.https://openai.com/ index/gpt-5-5-system-card/, 2026

    OpenAI. Gpt-5.5 system card.https://openai.com/ index/gpt-5-5-system-card/, 2026. Accessed: 2026-06. 5

    2026

  27. [27]

    OmniWeaving: Towards Unified Video Gener- ation with Free-form Composition and Reasoning

    Kaihang Pan, Qi Tian, Jianwei Zhang, Weijie Kong, Jiangfeng Xiong, Yanxin Long, Shixue Zhang, Haiyi Qiu, Tan Wang, Zheqi Lv, Yue Wu, Liefeng Bo, Siliang Tang, and Zhao Zhong. OmniWeaving: Towards Unified Video Gener- ation with Free-form Composition and Reasoning. 2, 3

  28. [28]

    Seedance 2.0: Advanc- ing video generation for world complexity.arXiv preprint arXiv:2604.14148, 2026

    Team Seedance, De Chen, Liyang Chen, Xin Chen, Ying Chen, Zhuo Chen, Zhuowei Chen, Feng Cheng, Tianheng Cheng, Yufeng Cheng, et al. Seedance 2.0: Advanc- ing video generation for world complexity.arXiv preprint arXiv:2604.14148, 2026. 9

    Pith/arXiv arXiv 2026

  29. [29]

    Msvbench: Towards human-level evaluation of multi-shot video generation, 2026

    Haoyuan Shi, Yunxin Li, Nanhao Deng, Zhenran Xu, Xinyu Chen, Longyue Wang, Baotian Hu, and Min Zhang. Msvbench: Towards human-level evaluation of multi-shot video generation, 2026. 3, 4, 8

    2026

  30. [30]

    Mavors: Multi-granularity video representation for multimodal large language model

    Yang Shi, Jiaheng Liu, Yushuo Guan, Zhenhua Wu, Yuanx- ing Zhang, Zihao Wang, Weihong Lin, Jingyun Hua, Zekun Wang, Xinlong Chen, et al. Mavors: Multi-granularity video representation for multimodal large language model. InPro- ceedings of the 33rd ACM International Conference on Mul- timedia, pages 10994–11003, 2025. 2

    2025

  31. [31]

    SII-GAIR, Sand. ai, :, Ethan Chern, Hansi Teng, Hanwen Sun, Hao Wang, Hong Pan, Hongyu Jia, Jiadi Su, Jin Li, Junjie Yu, Lijie Liu, Lingzhi Li, Lyumanshan Ye, Min Hu, Qiangang Wang, Quanwei Qi, Steffi Chern, Tao Bu, Tao- ran Wang, Teren Xu, Tianning Zhang, Tiantian Mi, Weixian 14 Xu, Wenqiang Zhang, Wentai Zhang, Xianping Yi, Xiaojie Cai, Xiaoyang Kang, Y...

    2026

  32. [32]

    Orianne Siméoni, Huy V . V o, Maximilian Seitzer, Fed- erico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khali- dov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, ...

    2025

  33. [33]

    T2v-compbench: A comprehen- sive benchmark for compositional text-to-video generation

    Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v-compbench: A comprehen- sive benchmark for compositional text-to-video generation. arXiv preprint arXiv:2407.14505, 2024. 2

    Pith/arXiv arXiv 2024

  34. [34]

    Kling-omni technical report.arXiv preprint arXiv:2512.16776, 2025

    Kling Team, Jialu Chen, Yuanzheng Ci, Xiangyu Du, Zipeng Feng, Kun Gai, Sainan Guo, Feng Han, Jingbin He, Kang He, et al. Kling-omni technical report.arXiv preprint arXiv:2512.16776, 2025. 9

    Pith/arXiv arXiv 2025

  35. [35]

    Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell

    Ting-Hao, Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell. Visual storytelling,

  36. [36]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025. 9

    Pith/arXiv arXiv 2025

  37. [37]

    Msavbench: Towards comprehensive and reliable evaluation of multi-shot audio-video generation, 2026

    Yujie Wei, Yujin Han, Zhekai Chen, Yongming Li, Kaixun Jiang, Zhihang Liu, Quanhao Li, Zhiwu Qing, Xiang Wang, Zhen Xing, Ruihang Chu, Lingyi Hong, Yefei He, Junjie Zhou, Junqiu Yu, Yang Shi, Difan Zou, Kai Zhu, Shiwei Zhang, Yingya Zhang, Yu Liu, Xihui Liu, and Hongming Shan. Msavbench: Towards comprehensive and reliable evaluation of multi-shot audio-vi...

    2026

  38. [38]

    Hunyuanvideo 1.5 technical report,

    Bing Wu, Chang Zou, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Jack Peng, Jianbing Wu, Jiangfeng Xiong, Jie Jiang, Linus, Patrol, Peizhen Zhang, Peng Chen, Peng- hao Zhao, Qi Tian, Songtao Liu, Weijie Kong, Weiyan Wang, Xiao He, Xin Li, Xinchi Deng, Xuefei Zhe, Yang Li, Yanxin Long, Yuanbo Peng, Yue Wu, Yuhong Liu, Zhenyu Wang, Zuozhuo Dai, Bo Peng, C...

  39. [39]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives, 2023

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jing- wen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives, 2023. 7

    2023

  40. [40]

    Captain Cinema: To- wards Short Movie Generation

    Junfei Xiao, Ceyuan Yang, Lvmin Zhang, Shengqu Cai, Yang Zhao, Yuwei Guo, Gordon Wetzstein, Maneesh Agrawala, Alan Yuille, and Lu Jiang. Captain Cinema: To- wards Short Movie Generation. 2, 3

  41. [41]

    Inceptionnext: When inception meets convnext, 2025

    Weihao Yu, Pan Zhou, Shuicheng Yan, and Xinchao Wang. Inceptionnext: When inception meets convnext, 2025. 8

    2025

  42. [42]

    Ui2v-bench: An understanding-based image-to- video generation benchmark, 2025

    Ailing Zhang, Lina Lei, Dehong Kong, Zhixin Wang, Jiaqi Xu, Fenglong Song, Chun-Le Guo, Chang Liu, Fan Li, and Jie Chen. Ui2v-bench: An understanding-based image-to- video generation benchmark, 2025. 2, 3, 4

    2025

  43. [43]

    Muss: A large-scale dataset and cinematic narrative benchmark for multi-shot subject-to-video generation.arXiv preprint arXiv:2604.23789, 2026

    Haojie Zhang, Di Wu, Bingyan Liu, Linjie Zhong, Yuancheng Wei, Xingsong Ye, Nanqing Liu, and Yaling Liang. Muss: A large-scale dataset and cinematic narrative benchmark for multi-shot subject-to-video generation.arXiv preprint arXiv:2604.23789, 2026. 4

    Pith/arXiv arXiv 2026

  44. [44]

    Monst3r: A simple approach for estimating geometry in the presence of motion, 2025

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jam- pani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming- Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion, 2025. 8

    2025

  45. [45]

    STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative

    Peixuan Zhang, Zijian Jia, Kaiqi Liu, Shuchen Weng, Si Li, and Boxin Shi. STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative. 2, 3

  46. [46]

    Smartdirector: Keyframe- conditioned cinematic video generation with narrative pac- ing control, 2026

    Zhida Zhang, Jie Ma, Zhan Peng, Haoxue Wu, Yang Han, Jun Liang, Jie Cao, and Jing Li. Smartdirector: Keyframe- conditioned cinematic video generation with narrative pac- ing control, 2026. 2, 3

    2026

  47. [47]

    VBench-2.0: Advancing video genera- tion benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. VBench-2.0: Advancing video genera- tion benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025. 3

    Pith/arXiv arXiv 2025

  48. [48]

    Videomemory: Toward consis- tent video generation via memory integration.arXiv preprint arXiv:2601.03655, 2026

    Jinsong Zhou, Yihua Du, Xinli Xu, Luozhou Wang, Zijie Zhuang, Yehang Zhang, Shuaibo Li, Xiaojun Hu, Bolan Su, and Ying-cong Chen. Videomemory: Toward consis- tent video generation via memory integration.arXiv preprint arXiv:2601.03655, 2026. 3

    arXiv 2026

  49. [49]

    Avgen- bench: A task-driven benchmark for multi-granular evalua- tion of text-to-audio-video generation, 2026

    Ziwei Zhou, Zeyuan Lai, Rui Wang, Yifan Yang, Zhen Xing, Yuqing Yang, Qi Dai, Lili Qiu, and Chong Luo. Avgen- bench: A task-driven benchmark for multi-granular evalua- tion of text-to-audio-video generation, 2026. 3

    2026

  50. [50]

    ViStoryBench: Comprehensive Benchmark Suite for Story Visualization

    Cailin Zhuang, Ailin Huang, Yaoqi Hu, Jingwei Wu, Wei Cheng, Jiaqi Liao, Hongyuan Wang, Xinyao Liao, Wei- wei Cai, Hengyuan Xu, Xuanyang Zhang, Xianfang Zeng, Zhewei Huang, Gang Yu, and Chi Zhang. ViStoryBench: Comprehensive Benchmark Suite for Story Visualization. 2, 3, 4 15 KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video...

    arXiv

  51. [51]

    Watch the generated video without using the benchmark spec to decide where cuts should exist

  52. [52]

    Start a new shot only at an actual edit boundary such as a hard cut, dissolve, wipe, or other visible transition to a different take

    A shot is one continuous camera take. Start a new shot only at an actual edit boundary such as a hard cut, dissolve, wipe, or other visible transition to a different take

  53. [53]

    Do not create a new shot for an action phase, subject state change, camera movement, reframing, occlusion, or prompt- requested event when the take remains continuous

  54. [54]

    If it is a one-take video, return exactly one segment covering the full video

    If the generated video has multiple actual shots, return one segment per actual shot in chronological order. If it is a one-take video, return exactly one segment covering the full video. Step 2 - Assign keyframes after the shot boundaries are fixed:

  55. [55]

    Use the benchmark spec and visible content only to assign each input keyframe to every already-fixed actual shot time range that visibly contains or represents it

  56. [56]

    Keyframe assignment must never add, remove, split , merge, shift, or reorder actual shot boundaries

  57. [57]

    List the keyframe in unassigned_keyframes instead

    If no actual shot visibly responds to a keyframe, do not invent an assignment. List the keyframe in unassigned_keyframes instead. Output contract:

  58. [58]

    Each segment is an actual generated-video shot, not an expected prompt shot and not a keyframe-specific semantic phase

  59. [59]

    S1", "S2

    Use actual shot ids "S1", "S2", ... in chronological order. For an actual single-shot video, the only shot id is "S1", even when the benchmark spec requests multiple shots

  60. [60]

    SEG001",

    Use segment ids "SEG001", "SEG002", ... in the same chronological order

  61. [61]

    Segment times are seconds in [0, duration_sec], sorted, contiguous, non-overlapping, and together cover the complete generated video

  62. [62]

    If it appears in zero segments, it must appear exactly once in unassigned_keyframes

    Each input keyframe id may appear in zero, one, or multiple segments' assigned_keyframe_ids. If it appears in zero segments, it must appear exactly once in unassigned_keyframes

  63. [63]

    A segment may contain zero, one, or multiple assigned keyframes

  64. [64]

    not_visible_in_generated_video

    unassigned_keyframes[*].reason must be exactly one of: "not_visible_in_generated_video" or "ambiguous". Use "not_visible_in_generated_video" only when the generated video has no Contract for Video Segmentation (continued) visible response to that keyframe. Use "ambiguous " when the evidence is too weak or conflicting to assign a visible response to a segment

  65. [65]

    S1" and assign all keyframes to that segment. - If the spec requests a one-take video but the generated video visibly has two edited shots, return two segments with shot_ids

    Confidence reflects confidence in both the observed shot boundary and the keyframe assignment. Keep reason concise and evidence-based. Do not: - Copy expected shot ids from the benchmark spec unless they coincidentally match the actual generated-video shot order. - Force the generated video to have the prompt- requested number of shots. - Split a continuo...

  66. [66]

    spatial_orientation.relative_position_stability: - left/right, front/behind, near/far, facing direction, containment, and ordering between named subjects remain stable when no visible action explains a change. - Judging rule: a relation change is acceptable when visible subject movement, camera viewpoint change, or an explicit scene transition explains it...

  67. [67]

    cross_shot

    spatial_orientation.scale_proportion_rationality: - Proportional size between subjects, objects, and environment landmarks remains plausible 14 Spatial Orientation Consistency Procedure (contin- ued) across camera scales and angles. Item construction: - Prefer 2-5 checklist_items. - Name the two subjects or subject/environment pair being checked. - Use on...

  68. [68]

    video_adherence. camera_parameter

    camera_parameter (dimension: "video_adherence. camera_parameter"): - camera motion, scale, angle, or framing matches the prompt/spec

  69. [69]

    video_adherence .shot_count_structure

    shot_count_structure (dimension: "video_adherence .shot_count_structure"): - generated shot count or continuous-take structure matches the prompt/spec. - for reviewer-facing human_question and inspect text, call a single-shot structure "one-take (a single continuous shot with no cuts)". Do not use the internal label "single-shot" by itself. - when shot co...

  70. [70]

    video_adherence. narrative_rhythm

    narrative_logic (dimension: "video_adherence. narrative_rhythm" or "video_adherence. story_focus"): - transitions and duration distribution support the intended story focus

  71. [71]

    video_adherence. segment_presence

    segment_presence (dimension: "video_adherence. segment_presence"): - required characters, props, actions, and scenes appear in the intended segments. Item construction: - Prefer 3-7 checklist_items. - Each item tests one requested prompt/spec requirement. - Use "shot" or "segment" for localized requirements . - Use "cross_shot" for transition/rhythm check...

  72. [72]

    audio_adherence. semantic_alignment

    semantic_alignment (dimension: "audio_adherence. semantic_alignment"): - requested sounds, music, dialogue, or ambience are present and match what the prompt/spec asks for

  73. [73]

    audio_adherence. style_emotion

    style_emotion (dimension: "audio_adherence. style_emotion"): - overall audio style, mood, genre, or emotional tone matches the prompt/spec's intended atmosphere. Item construction: - Use evidence_scope "audio_track". - Prefer 1-3 checklist_items when audio requirements exist. - If no audio requirement exists, create one item stating that no audio-specific...

  74. [74]

    video_adherence. segment_presence

    segment_presence (dimension: "video_adherence. segment_presence"): - all core subjects listed in subject_classification.subjects appear in the generated video unless the story_summary clearly makes one background-only. - required central characters, props, animals, vehicles, or scene elements are visually present enough for a judge to recognize them

  75. [75]

    video_adherence .shot_count_structure

    shot_count_structure (dimension: "video_adherence .shot_count_structure"): - multi-shot samples contain multiple distinct shots separated by cuts or transitions. - one-take samples remain a single continuous shot with no cuts unless the keyframes imply a transition. - for reviewer-facing human_question and inspect text, use "one-take (a single continuous ...

  76. [76]

    video_adherence. narrative_rhythm

    narrative_logic (dimension: "video_adherence. narrative_rhythm" or "video_adherence. 16 Video Modality Adherence Minimal-Prompt Proce- dure (continued) story_focus"): - visual differences between adjacent keyframes inform expected rhythm. - the generated video represents the main events implied by keyframes and story_summary. Item construction: - Prefer 3...

  77. [77]

    spatial_orientation.relative_position_stability

    relative_position_stability (dimension: " spatial_orientation.relative_position_stability "): - left/right, front/behind, near/far, facing, and containment relations. - Judging rule: a relation change is acceptable when visible subject movement, camera viewpoint change, or a scene transition explains it; treat only unexplained changes as failures

  78. [78]

    spatial_orientation. scale_proportion_rationality

    scale_proportion_rationality (dimension: " spatial_orientation. scale_proportion_rationality"): - proportional size between subjects and environment remains plausible across visual segments. Item construction: - Use only relations with clear visual evidence in at least one keyframe. - Name the subjects and relation being checked. - Use "cross_shot" when c...