Pith. sign in

REVIEW 5 major objections 7 minor 49 references

A few learnable tokens, positioned at fractional latent coordinates, give a pretrained video diffusion model frame-accurate shot transitions and better cross-shot consistency than prior multi-shot approaches.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Learnable planning tokens with fractional positional timestamps let one diffusion pass generate multi-shot video with frame-accurate cuts and timed camera motion.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection ShotPlan makes a plausible new move—learnable FRoPE-positioned planning tokens for frame-accurate shot control—but the headline comparison is undercut by a metric confound. the 5 major comments →

arxiv 2607.17675 v1 pith:H4HC2Z5S submitted 2026-07-20 cs.CV

ShotPlan: Cinematic Video Generation with Learnable Planning Token

Su Guo , Guangce Liu , Haosen Yang , Jiepeng Wang , Cong Liu , Junqi Liu , Haibin Huang , Hongxun Yao
show 2 more authors
Chi Zhang Xuelong Li
This is my paper
classification cs.CV
keywords multi-shot video generationshot planninglearnable tokensfractional rotary position embeddingdiffusion transformercinematic videotemporal controlvideo diffusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that shot planning in cinematic multi-shot video generation can be made explicit and frame-accurate without disturbing the pretrained architecture of a video diffusion transformer. It does this by inserting a small number of learnable planning tokens into the token sequence, one per transition, and positioning them with a fractional variant of rotary position embedding so each token points at a precise physical frame. The authors argue this beats prior approaches—which rely on attention masks or modified positional encodings—because it preserves free cross-shot information flow, yielding tighter transition timing and stronger character and scene consistency. If true, the idea gives creators a lightweight way to script cuts, fades, and temporally localized camera moves inside a single generation pass.

Core claim

ShotPlan's central claim is that a transition is best represented as a learnable token that acts as an in-context conditioning anchor in the video transformer's own coordinate system. The planning tokens are concatenated with the patchified video tokens and processed through ordinary self-attention, so no attention structure or positional encoding of the backbone is modified. Fractional Temporal Rotary Position Embedding (FRoPE) maps a user-specified frame timestamp into a continuous latent coordinate, which matters because the video autoencoder compresses four physical frames into one latent step. On the paper's benchmark, the method reports a transition deviation of 0.64 frames versus 1.12

What carries the argument

Learnable planning tokens plus Fractional Temporal Rotary Position Embedding (FRoPE). A planning token is a small trainable vector replicated according to the number of transition events, each copy receiving a real-valued (fractional) temporal coordinate instead of a discrete latent index. Since rotary embeddings are sinusoidal, FRoPE plugs the fractional coordinate directly into the existing RoPE computation, allowing the token to point at an exact frame even under VAE temporal compression. The tokens are concatenated with visual tokens, pass through all DiT blocks as ordinary sequence elements, act as in-context conditioning, and are removed before decoding.

Load-bearing premise

The load-bearing premise is that an off-the-shelf shot-boundary detector and a single multimodal model used as both data curator and evaluation judge are faithful proxies for human perception of cut timing and cross-shot consistency; if either is biased, the reported margins may not survive human evaluation.

What would settle it

Have independent human annotators mark the perceived cut frames on a fresh set of generated videos (e.g., 50 prompts with two to four specified cut frames), then compare median absolute error in frames against the same annotators' ratings for the strongest prior baseline. If the median error for ShotPlan is not clearly lower, or if human consistency ratings do not track the reported automated feature-similarity scores, the central claim of frame-accurate, more consistent multi-shot generation is not supported. A cheaper check: replace the detector-based transition measure with manual frame lab

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Users can specify hard cuts (and fade start/end points) at exact frame numbers, and the model honors them within about a frame.
  • Because the base model's attention and positional encoding are left intact, cross-shot information flows freely, which the authors connect to better preservation of character identity and scene layout.
  • The same token mechanism extends to gradual transitions and to temporally localized camera motion, suggesting a single controller for discrete and continuous temporal events.
  • The approach needs only modest fine-tuning data (roughly 13,000 curated clips and a few thousand training steps) to adapt a large pretrained video diffusion transformer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism generalizes, the same token-anchoring idea could control other time-localized events in video (action beats, lighting changes, audio cues), since it does not depend on the semantics of a cut.
  • The evaluation leans on automated surrogates—an off-the-shelf detector for cut timing and a single multimodal judge for consistency—so an independent human study on a held-out prompt set would be the decisive test of whether the reported gains hold perceptually.
  • The camera-motion result suggests a cheaper path to motion control than training dedicated adapters, but the paper tests only six predefined moves; broader motion vocabularies would reveal the limits of token-propagation.
  • A targeted ablation of token count versus number of cuts could clarify whether the method scales to long videos with many transitions or degrades as token sequence grows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. ShotPlan proposes a framework for multi-shot cinematic video generation built on the Wan2.1-T2V-14B diffusion transformer. The method introduces learnable planning tokens that are concatenated to the visual token sequence and equipped with Fractional Temporal Rotary Position Embedding (FRoPE), which maps user-specified frame-level transition timestamps to fractional latent coordinates. Training data is curated from VideoEvent using TransNet V2 shot detection and Gemini 2.5 filtering/annotation. Experiments compare ShotPlan with CineTrans, EchoShot, HoloCine, and MultiShotMaster on transition deviation, text alignment, DINOv2-based inter-shot consistency, and Gemini-judged narrative coherence, plus a user study for temporally localized camera motion. The paper claims state-of-the-art transition accuracy and inter-shot consistency.

Significance. If the empirical claims are fully supported, ShotPlan would be a valuable contribution. The design is elegant: it preserves the pretrained DiT architecture and the original RoPE for video tokens, adds only a small set of learnable conditioning tokens, and FRoPE is a parameter-free extension that addresses VAE temporal compression. The internal ablation in Table 2 and the attention visualization in Fig. 4 are consistent with the proposed mechanism working as described. However, the headline comparison against baselines is currently not decisive because the main metric is confounded with transition style, no uncertainty quantification is provided, and parts of the evaluation rely on the same model used for data curation. The paper needs stronger, less circular validation before the central claim can be accepted.

major comments (5)
  1. [Sec. 4.1, 'Metrics' and Table 1] Transition Deviation is computed as the frame-level gap between TransNet V2 detections and user-specified timestamps. TransNet V2 is trained on real edited footage and is sensitive to transition abruptness. ShotPlan is explicitly trained to emit hard cuts at precise frames, whereas CineTrans and HoloCine are designed around soft/gradual cinematic transitions. For a dissolve or other gradual transition there is no unique boundary frame, so a detector firing at the transition center or missing the transition incurs a penalty that does not reflect temporal-control error. The 0.64 vs 1.12/2.71 gap may therefore conflate transition-style mismatch with timing accuracy. Please add a human-annotated frame-level evaluation on the same generated videos, report detection rates and confidence, and break down results by requested transition type (hard cut vs soft transition). This is central to the '
  2. [Table 1 and Sec. 4.2] No error bars, standard deviations, or significance tests are reported for any automatic metric. The benchmark consists of 100 prompts, but the number of generated videos per method and per metric is not stated. Differences such as Text Alignment 0.26 vs 0.28 and Character Consistency 0.46 vs 0.39 may be within sampling noise. Please report per-prompt paired results, confidence intervals, and a paired significance test (e.g., bootstrap or Wilcoxon) for each metric. Without this, the claim that ShotPlan 'significantly outperforms' is unsupported.
  3. [Sec. 3.4 and Sec. 4.1] Gemini 2.5 is used to construct the evaluation prompts, filter the training data, generate shot captions, and judge Narrative Coherence. This creates a risk that the method is tailored to a particular judge and that the reported narrative scores reflect agreement with Gemini rather than general cinematic quality. This is not by itself a fatal flaw, but it needs mitigation: report human agreement on evaluation samples, use an independent captioning model for text alignment, and make the judge prompts public. The manual screening protocol for the curated data should also be described in enough detail to assess selection bias.
  4. [Sec. 4.1, 'Baseline'] The statement 'For fair comparison, we adapt each prompt to the input format required by each baseline' is too vague to establish equivalence. It is unclear whether the baselines receive the same transition timestamps, the same shot count, the same structured per-shot captions, and the same hard-cut versus soft-transition specification. If a baseline cannot consume a frame-level hard-cut schedule, then the Transition Deviation comparison is unfair because the task is not identical. Please provide a concrete example of the adapted prompt for each method and document any constraints (e.g., minimum shot duration) imposed on baselines.
  5. [Sec. 4.6, Table 3] The camera-movement user study lacks essential setup details: number of participants, number of videos per condition, whether the evaluation is paired within participants, the exact wording of the 'Timing Accuracy' question, and any statistical analysis. The reported differences (e.g., Timing 97% vs 96% for Wan 2.1; Type 89% vs 81% for Kling 2.6) are not accompanied by confidence intervals or significance tests. The Wan 2.1 baseline is also not described: how is the motion instruction provided to it? Please supply the full protocol and per-item results, or weaken the claim to 'competitive with' rather than 'the highest timing accuracy.'
minor comments (7)
  1. [Abstract] Typo: 'we introduceShotPlan' should read 'we introduce ShotPlan'.
  2. [Sec. 3.3] The text says 'With temporal compression ratios t' but the symbol is later s_t; please fix the notation. Also, the equation for f = 1 + (T - 1)/s_t is broken across lines in the PDF; please reformat.
  3. [Sec. 4.4, Table 2] The paragraph for Global AdaLN says it 'shows little improvement over the uncontrolled baseline,' but no uncontrolled baseline is shown in Table 2. Either include that row or rephrase to refer to an earlier sentence.
  4. [Fig. 4 caption] The caption says 'the vertical axis represents the flag tokens' — this should be 'planning tokens.' Also, the x-axis label 'Latent Frames (Time)' could be clearer as 'Latent temporal index'.
  5. [Sec. 4.5] Minor style: '8-th DiT block' should be '8th DiT block.'
  6. [Table 3 and text] The model name appears as both 'SeedDance 1.5 Pro' and 'Seedance 1.5'; please use one spelling consistently.
  7. [References] Reference [17] (LoRA) is cited only in the supplementary material. If it is not needed in the main text, consider moving it to the supplementary reference list or citing it in the main text for completeness.

Circularity Check

0 steps flagged

No significant circularity: FRoPE mechanism is ablated and metrics rely on external detectors; Gemini/TransNet issues are validity concerns, not circularity.

full rationale

I find no circular step that reduces a claimed result to its own inputs. The main mechanism—learnable planning tokens with FRoPE—is an empirical design validated by ablations (Table 2: removing FRoPE raises Transition Deviation from 0.64 to 2.13; replacing learnable tokens with static semantic tokens degrades consistency), and the attention analysis in Fig. 4 is post-hoc evidence, not a derivation. The formal mapping t_u -> 1+(t_u-1)/s_t is just the VAE compression inverse using RoPE's continuous coordinates; it defines the conditioning, not the evaluation, and is not equivalent to the reported metric. The evaluation uses external tools (TransNet V2, ViCLIP, DINOv2) and a Gemini judge; no parameter is fitted to these judges, and the transition-control claim is also supported by internal ablations. The use of Gemini for data annotation and narrative scoring, and TransNet V2 for both data labeling and transition metrics, are benchmark-validity concerns, not circularity under the definitional/fitted-parameter criteria. No load-bearing self-citation exists: the only overlap reference (Lumiere) is unrelated background. Score 0.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 2 invented entities

The central mechanism is training a small number of trainable tokens plus a positional embedding choice. The main fitted quantities are the token contents and the data-pipeline thresholds. The strongest external anchor is the FRoPE ablation (removing FRoPE worsens Transition Deviation), which is the one clean conditional check in the paper. Everything else rests on the pretrained Wan2.1 prior, TransNet V2 detection, and Gemini-based scoring.

free parameters (2)
  • Learnable planning tokens = random init N(0, 0.02), trained
    F_p tokens are trainable and initialized randomly; their optimization is the mechanism, so their content is a fitted component.
  • Cutting/soft-detection tolerance and data filter thresholds = window length 5s, per-shot >=20 frames, frequency filtering, manual screening
    These thresholds are hand-chosen data curation choices that affect training data and therefore all downstream numbers.
axioms (3)
  • domain assumption The pretrained Wan2.1-T2V-14B generates coherent video given normal tokens; inserted extra tokens do not degrade generation.
    The entire design keeps the backbone fixed and adds tokens, so the backbone prior is assumed robust to out-of-distribution token counts.
  • domain assumption TransNet V2 shot-boundary detections on generated 81-frame videos are accurate enough to measure Transition Deviation.
    TransNet V2 is trained on natural edited videos; using it on generated content is an unvalidated transfer assumption.
  • domain assumption Gemini 2.5 reliably filters/annotates training data and judges narrative coherence in the evaluation.
    An LLM/VLM used both as data annotator and evaluator for the same style of content; errors propagate into both training and metrics.
invented entities (2)
  • Learnable planning tokens F_p (hard-cut/cross-fade/camera-motion tokens) no independent evidence
    purpose: Temporal anchors for shot transitions and camera-motion commands.
    No token-ablation or attention visualization provides a falsifiable outside prediction; only internal metrics.
  • Fractional Temporal Rotary Position Embedding (FRoPE) no independent evidence
    purpose: Position planning tokens at fractional latent timestamps to achieve frame-level cut accuracy.
    FRoPE is a positional coordinate choice; the ablation shows it affects Transition Deviation, but no external falsifiable handle is given.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ShotPlan: Cinematic Video Generation with Learnable Planning Token." pith.science (2026). https://pith.science/paper/H4HC2Z5S

@misc{pith2026260717675,
  author       = {Pith},
  title        = {Pith review of: ShotPlan: Cinematic Video Generation with Learnable Planning Token},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H4HC2Z5S}},
  note         = {Machine review of arXiv:2607.17675}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Current video generation models achieve impressive results in single-shot generation, yet remain limited in cinematic video generation, where coherent narratives and effective multi-shot composition require explicit shot planning. To address this challenge, we propose ShotPlan, a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model. Our method introduces learnable planning tokens that capture shot-level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps. Unlike standard video generation tokens, the proposed planning tokens are equipped with Fractional Temporal Rotary Position Embedding (FRoPE), enabling shot transitions to be modeled at the frame level. Experiments demonstrate that ShotPlan significantly outperforms existing cinematic video generation methods, offering more flexible shot management and stronger inter-shot consistency.

Figures

Figures reproduced from arXiv: 2607.17675 by Chi Zhang, Cong Liu, Guangce Liu, Haibin Huang, Haosen Yang, Hongxun Yao, Jiepeng Wang, Junqi Liu, Su Guo, Xuelong Li.

Figure 1
Figure 1. Figure 1: ShotPlan enables controllable multi-shot video generation within a single sequence. ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the ShotPlan framework. Given an input video, a 3D VAE encodes the frames into latent representations, which are then patchified into visual tokens. Based on user-specified cut timestamps and a structured prompt, ShotPlan introduces learnable planning tokens assigned with precise temporal coordinates via Fractional Rotary Position Embedding (FRoPE). These tokens are concatenated with visual tok… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of multi-shot video generation with CineTrans [ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Latent temporal attention weight visualization of the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of temporally localized cam [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: ShotPlan generation gallery. Each row shows a multi-shot video generated by ShotPlan, with user-specified shot transitions. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: ShotPlan generation gallery (continued). Each row shows a multi-shot video generated by ShotPlan, with user-specified shot [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 20 linked inside Pith

  1. [1]

    Recammaster: Camera-controlled generative ren- dering from a single video

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlled generative ren- dering from a single video. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 14834– 14844, 2025. 8

    2025

  2. [2]

    Lumiere: A space-time diffu- sion model for video generation

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Her- rmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere: A space-time diffu- sion model for video generation. InSIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 2

    2024

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2

    Pith/arXiv arXiv 2023

  4. [4]

    Videocanvas: Unified video completion from 8 arbitrary spatiotemporal patches via in-context conditioning

    Minghong Cai, Qiulin Wang, Zongli Ye, Wenze Liu, Quande Liu, Weicai Ye, Xintao Wang, Pengfei Wan, Kun Gai, and Xiangyu Yue. Videocanvas: Unified video completion from 8 arbitrary spatiotemporal patches via in-context conditioning. arXiv preprint arXiv:2510.08555, 2025. 4

    Pith/arXiv arXiv 2025

  5. [5]

    Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023. 2

    Pith/arXiv arXiv 2023

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7310–7320,

  7. [7]

    Seine: Short-to-long video diffu- sion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffu- sion model for generative transition and prediction. InThe Twelfth International Conference on Learning Representa- tions, 2023. 2, 3

    2023

  8. [8]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025

    Gheorghe Comanici, Eric Bieber, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. 6

    2025

  9. [9]

    Motion prompting: Controlling video generation with motion trajec- tories

    Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Yusuf Aytar, Michael Rubinstein, Chen Sun, et al. Motion prompting: Controlling video generation with motion trajec- tories. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 1–12, 2025. 8

    2025

  10. [10]

    Talecrafter: Interactive story visualization with multiple characters.arXiv preprint arXiv:2305.18247, 2023

    Yuan Gong, Youxin Pang, Xiaodong Cun, Menghan Xia, Yingqing He, Haoxin Chen, Longyue Wang, Yong Zhang, Xintao Wang, Ying Shan, et al. Talecrafter: Interactive story visualization with multiple characters.arXiv preprint arXiv:2305.18247, 2023. 3

    Pith/arXiv arXiv 2023

  11. [11]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023. 2

    Pith/arXiv arXiv 2023

  12. [12]

    Long context tuning for video generation

    Yuwei Guo, Ceyuan Yang, Ziyan Yang, Zhibei Ma, Zhijie Lin, Zhenheng Yang, Dahua Lin, and Lu Jiang. Long context tuning for video generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17281– 17291, 2025. 3

    2025

  13. [13]

    Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024. 8

    Pith/arXiv arXiv 2024

  14. [14]

    Animate-a-story: Storytelling with retrieval-augmented video generation.arXiv preprint arXiv:2307.06940, 2023

    Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, et al. Animate-a-story: Storytelling with retrieval-augmented video generation.arXiv preprint arXiv:2307.06940, 2023. 3

    Pith/arXiv arXiv 2023

  15. [15]

    Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022. 2

    Pith/arXiv arXiv 2022

  16. [16]

    Video dif- fusion models.Advances in neural information processing systems, 35:8633–8646, 2022

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models.Advances in neural information processing systems, 35:8633–8646, 2022. 2

    2022

  17. [17]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. InIn- ternational Conference on Learning Representations, 2022. 11

    2022

  18. [18]

    Shotadapter: Text-to- multi-shot video generation with diffusion models

    Ozgur Kara, Krishna Kumar Singh, Feng Liu, Duygu Cey- lan, James M Rehg, and Tobias Hinz. Shotadapter: Text-to- multi-shot video generation with diffusion models. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 28405–28415, 2025. 3

    2025

  19. [19]

    Kling video generation model.https:// kling.kuaishou.com/en, 2024

    Kuaishou. Kling video generation model.https:// kling.kuaishou.com/en, 2024. Accessed: 2026-05-

    2024

  20. [20]

    Videvent: A large dataset for understanding dynamic evolution of events in videos, 2025

    Baoyu Liang, Qile Su, Shoutai Zhu, Yuchen Liang, and Chao Tong. Videvent: A large dataset for understanding dynamic evolution of events in videos, 2025. 5

    2025

  21. [21]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 3, 11

    Pith/arXiv arXiv 2022

  22. [22]

    Sit: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers. InEuropean Conference on Com- puter Vision, pages 23–40. Springer, 2024. 2, 3

    2024

  23. [23]

    Latte: Latent diffusion transformer for video generation.arXiv preprint arXiv:2401.03048, 2024

    Xin Ma, Yaohui Wang, Xinyuan Chen, Gengyun Jia, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation.arXiv preprint arXiv:2401.03048, 2024. 2, 3

    Pith/arXiv arXiv 2024

  24. [24]

    Holocine: Holistic generation of cinematic multi-shot long video narratives.arXiv preprint arXiv:2510.20822, 2025

    Yihao Meng, Hao Ouyang, Yue Yu, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Hanlin Wang, Yixuan Li, Cheng Chen, Yanhong Zeng, et al. Holocine: Holistic generation of cinematic multi-shot long video narratives.arXiv preprint arXiv:2510.20822, 2025. 2, 3, 5, 6, 7

    arXiv 2025

  25. [25]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 6

    Pith/arXiv arXiv 2023

  26. [26]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 4195–4205,

  27. [27]

    Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Pith/arXiv arXiv

  28. [28]

    Maskˆ 2dit: Dual mask-based diffusion transformer for multi-scene long video generation

    Tianhao Qi, Jianlong Yuan, Wanquan Feng, Shancheng Fang, Jiawei Liu, SiYu Zhou, Qian He, Hongtao Xie, and Yongdong Zhang. Maskˆ 2dit: Dual mask-based diffusion transformer for multi-scene long video generation. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 18837–18846, 2025. 3 9

    2025

  29. [29]

    Seedance 1.5 pro: A native audio- visual joint generation foundation model.arXiv preprint arXiv:2512.13507, 2025

    Team Seedance, Heyi Chen, Siyan Chen, Xin Chen, Yan- fei Chen, Ying Chen, Zhuo Chen, Feng Cheng, Tianheng Cheng, Xinqi Cheng, et al. Seedance 1.5 pro: A native audio- visual joint generation foundation model.arXiv preprint arXiv:2512.13507, 2025. 8

    Pith/arXiv arXiv 2025

  30. [30]

    Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,

    Pith/arXiv arXiv

  31. [31]

    Transnet v2: An effective deep network architecture for fast shot transition detection

    Tom ´as Soucek and Jakub Lokoc. Transnet v2: An effective deep network architecture for fast shot transition detection. InProceedings of the 32nd ACM International Conference on Multimedia, pages 11218–11221, 2024. 6

    2024

  32. [32]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

  33. [33]

    Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025. 2, 3, 6, 8, 11

    Pith/arXiv arXiv 2025

  34. [34]

    Echoshot: Multi-shot portrait video generation

    Jiahao Wang, Hualian Sheng, Sijia Cai, Weizhan Zhang, Caixia Yan, Yachuang Feng, Bing Deng, and Jieping Ye. Echoshot: Multi-shot portrait video generation. InThe Thirty-ninth Annual Conference on Neural Information Pro- cessing Systems, 2025. 2, 3, 5, 6, 7

    2025

  35. [35]

    Multishotmaster: A controllable multi-shot video generation framework.arXiv preprint arXiv:2512.03041,

    Qinghe Wang, Xiaoyu Shi, Baolu Li, Weikang Bian, Quande Liu, Huchuan Lu, Xintao Wang, Pengfei Wan, Kun Gai, and Xu Jia. Multishotmaster: A controllable multi-shot video generation framework.arXiv preprint arXiv:2512.03041,

    arXiv

  36. [36]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation.arXiv preprint arXiv:2307.06942, 2023

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation.arXiv preprint arXiv:2307.06942, 2023. 6

    Pith/arXiv arXiv 2023

  37. [37]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Pa- pers, pages 1–11, 2024. 8

    2024

  38. [38]

    Videorope: What makes for good video rotary position embedding?arXiv preprint arXiv:2502.05173, 2025

    Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Jian Tong, Haodong Duan, Qipeng Guo, Jiaqi Wang, et al. Videorope: What makes for good video rotary position embedding?arXiv preprint arXiv:2502.05173, 2025. 2, 3, 5

    Pith/arXiv arXiv 2025

  39. [39]

    Hunyuanvideo 1.5 technical report.arXiv preprint arXiv:2511.18870, 2025

    Bing Wu, Chang Zou, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Jack Peng, Jianbing Wu, Jiangfeng Xiong, Jie Jiang, et al. Hunyuanvideo 1.5 technical report.arXiv preprint arXiv:2511.18870, 2025. 2

    Pith/arXiv arXiv 2025

  40. [40]

    Cinetrans: Learning to generate videos with cinematic transitions via masked diffusion models.arXiv preprint arXiv:2508.11484, 2025

    Xiaoxue Wu, Bingjie Gao, Yu Qiao, Yaohui Wang, and Xinyuan Chen. Cinetrans: Learning to generate videos with cinematic transitions via masked diffusion models.arXiv preprint arXiv:2508.11484, 2025. 2, 3, 5, 6, 7

    arXiv 2025

  41. [41]

    Captain cinema: To- wards short movie generation

    Junfei Xiao, Ceyuan Yang, Lvmin Zhang, Shengqu Cai, Yang Zhao, Yuwei Guo, Gordon Wetzstein, Maneesh Agrawala, Alan Yuille, and Lu Jiang. Captain cinema: To- wards short movie generation. InThe Fourteenth Interna- tional Conference on Learning Representations, 2025. 2, 3

    2025

  42. [42]

    Direct-a-video: Customized video generation with user- directed camera movement and object motion

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-video: Customized video generation with user- directed camera movement and object motion. InACM SIG- GRAPH 2024 Conference Papers, pages 1–12, 2024. 8

    2024

  43. [43]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024. 2, 3

    Pith/arXiv arXiv 2024

  44. [44]

    StoryMem: Multi-shot long video storytelling with memory.arXiv preprint, arXiv:2512.19539, 2025

    Kaiwen Zhang, Liming Jiang, Angtian Wang, Jacob Zhiyuan Fang, Tiancheng Zhi, Qing Yan, Hao Kang, Xin Lu, and Xingang Pan. StoryMem: Multi-shot long video storytelling with memory.arXiv preprint, arXiv:2512.19539, 2025. 3

    arXiv 2025

  45. [45]

    Moviedreamer: Hierarchical generation for coherent long vi- sual sequence.arXiv preprint arXiv:2407.16655, 2024

    Canyu Zhao, Mingyu Liu, Wen Wang, Weihua Chen, Fan Wang, Hao Chen, Bo Zhang, and Chunhua Shen. Moviedreamer: Hierarchical generation for coherent long vi- sual sequence.arXiv preprint arXiv:2407.16655, 2024. 2, 3

    arXiv 2024

  46. [46]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Jun- hao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. InEuropean Conference on Computer Vision, pages 273–290. Springer, 2024. 8

    2024

  47. [47]

    Videogen-of-thought: Step-by-step generating multi-shot video with minimal manual interven- tion.arXiv preprint arXiv:2412.02259, 2024

    Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma, Yexin Liu, Wenjie Shu, Yatian Pang, Feilong Tang, Qifeng Chen, Harry Yang, et al. Videogen-of-thought: Step-by-step generating multi-shot video with minimal manual interven- tion.arXiv preprint arXiv:2412.02259, 2024. 2, 3

    arXiv 2024

  48. [48]

    Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024. 2

    Pith/arXiv arXiv 2024

  49. [49]

    Storydiffusion: Consistent self- attention for long-range image and video generation.Ad- vances in Neural Information Processing Systems, 37: 110315–110340, 2024

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. Storydiffusion: Consistent self- attention for long-range image and video generation.Ad- vances in Neural Information Processing Systems, 37: 110315–110340, 2024. 3 10 ShotPlan: Cinematic Video Generation with Learnable Planning Token Supplementary Material This supplementary material ...

    2024

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.