Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

TokensGen: Harnessing Condensed Tokens for Long Video Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TokensGen's central claim is that minute-long video consistency comes from planning all clips in a condensed token space before rendering them, then stitching the clips with adaptive FIFO-Diffusion.

desk verdict A credible modular recipe for minute-long video generation, with a real unmeasured risk at the token-conditioning interface; worth refereeing but not yet fully convincing. read the letter →

arxiv 2507.15728 v1 pith:3WKQGEPZ submitted 2025-07-21 cs.CV

Wenqi Ouyang , Zeqi Xiao , Danni Yang , Yifan Zhou , Shuai Yang , Lei Yang , Jianlou Si , Xingang Pan This is my paper
classification cs.CV
keywords longvideogenerationdiffusionmodelssemantictokenstokenizertext-to-videotemporalconsistencyFIFO-Diffusiontokentransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long, minute-level videos can be generated consistently by planning the entire video in a compressed token space first and then rendering it clip by clip with an existing short-video diffusion model. The paper's TokensGen does this in three steps: a Video Tokenizer condenses each short clip into a small set of semantic tokens, a Text-to-Token transformer (T2To) generates the token sequence for the whole long video in one pass, and an adaptive FIFO-Diffusion schedule denoises consecutive clips together so boundaries do not jump. The central claim is that this decomposition solves the long-term drift and memory bottlenecks of direct long-video generation while reusing a pretrained short-video model, so a minute of consistent video costs roughly the same per rendered clip as a short clip. The paper supports this with quantitative gains in motion smoothness and dynamic degree, and a user study in which viewers preferred TokensGen's motion-and-content consistency over the strongest baseline by about 75% to 24%.

What carries the argument

The load-bearing object is the condensed semantic token sequence: a Video Tokenizer (CogVideoX's 3D VAE and patchify, followed by a Resampler with 3D cross-attention and a PCA channel projection) compresses each 49-frame clip into a tensor of shape 4x8x12x16, about a 3x8x8 reduction from the original 13x60x90x16 latent. T2To is a diffusion transformer trained to model the joint distribution of these token tensors across all clips, which gives the global plan. To2V is the same pretrained short-video architecture with a tunable 3D cross-attention branch that lets the semantic tokens steer each clip's layout and motion. Adaptive FIFO-Diffusion is the inference-time diagonal denoising schedule that keeps adjacent clips continuous without the distributional artifacts of naive frame padding.

What would settle it

Encode a held-out set of source clips into tokens, render them back with To2V, and compare reconstruction fidelity against the originals; then compare T2To-generated token statistics to encoder token statistics on the same prompts. If reconstruction is poor or the distributions are far apart, the claimed consistency mechanism is not doing what the paper assumes.

Watch

Extended reading notes

Core claim

TokensGen claims that long-term consistency in video generation can be separated from short-term rendering: first generate a complete, globally coherent plan of condensed semantic tokens from the text prompt, then let a pretrained short-video model render each clip under that plan. The plan is produced by T2To, a video token diffusion transformer that outputs tokens for up to 24 clips of 49 frames each in one forward pass, and the rendering is done by To2V, a CogVideoX-based model whose cross-attention branch injects the tokens alongside the text. During inference, an adaptive FIFO-Diffusion schedule connects clips diagonally, replacing the frame-replication padding that caused artifacts in the original FIFO approach. The paper reports that this yields minute-long videos with stable motion and content, with 75.69% and 75.74% of pairwise user preferences favoring TokensGen over the best baseline for text alignment and motion-content consistency respectively.

Load-bearing premise

Everything rests on the assumption that the tiny condensed tokens preserve enough spatial and motion detail, and that tokens generated from text by T2To land in the same distribution as tokens encoded from real videos; the paper checks only the final rendered video, never this intermediate representation.

Editorial extensions

If this is right

  • Minute-level text-to-video becomes a modular pipeline: a pretrained short-video model can be reused, and memory cost scales with the compact token plan rather than with the full frame sequence.
  • Because the whole token plan is sampled before any clip is rendered, long-range decisions such as scene progression and content continuity are made once, not accumulated clip by clip, which is what prevents drift.
  • The adaptive FIFO padding removes a specific class of boundary artifacts, so extending a video does not degrade the opening frames or propagate replicated-frame distortions.
  • The framework's components can be improved independently: better tokenizers, stronger token planners, or different short-term consistency schedulers can be swapped in without redoing the whole system.
  • The same pipeline supports zero-shot prompt-guided long-video editing by feeding source video tokens plus a target text prompt into To2V, preserving structure while changing semantics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the system's weakest untested link is the tokenizer itself—reconstruction fidelity is never measured directly—so the cleanest next experiment is to encode real clips, render them back with To2V, and measure per-clip fidelity against the source.
  • Beyond the paper: if T2To-generated tokens drift from encoder-produced tokens, the cross-attention conditioning will degrade; a direct distribution comparison in token space on held-out prompts would settle this more sharply than rendered-video user studies.
  • Beyond the paper: because the global plan is a sequence of compact tensors, token-space operations such as interpolating or blending two prompts' plans are a natural, untested extension that could give smooth semantic transitions without retraining.
  • Beyond the paper: the same token plan might be rendered by different short-video backbones, since the token representation is trained atop a pretrained model and could be ported rather than re-learned per model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. TokensGen proposes a two-stage framework for minute-long video generation: To2V, a short-video diffusion model conditioned on text and condensed semantic tokens extracted by a Video Tokenizer built on CogVideoX; T2To, a token-level diffusion transformer that generates the full sequence of condensed tokens for a long video from text alone; and an adaptive FIFO-Diffusion inference scheme that connects adjacent clips. The authors train on MiraData, evaluate against several long-video baselines, and report both VBench scores and a human study in which their method receives about 75% preference on alignment and motion/content consistency over the strongest baseline. Ablations cover token shape, PCA projection, and the adaptive padding scheme.

Significance. If the central claim holds, the paper offers a practical and modular route to long video generation: it upgrades a pre-trained short-video model to minute-long outputs without retraining the backbone, uses a compact token representation to keep memory costs low, and provides a transparent decomposition into inner-clip control, global consistency, and boundary smoothing. The authors give credit for a fairly extensive evaluation, including multiple baselines, VBench and VBench-Long, a human study, ablations, and a public webpage with video results. The main contribution is the token-level global planner (T2To) coupled with the token-conditioned To2V decoder; this is a conceptually clean design that appears distinct from keyframe-based hierarchical methods and from pure FIFO autoregressive extension. The paper is also candid about limitations, explicitly acknowledging gradual foreground/background drift and cases where FIFO and condensed tokens are insufficient. The empirical support is not yet conclusive because the central coupling between T2To and To2V is not directly validated and the human-study evidence is small-scale and lacks statistical detail.

major comments (4)
  1. [Sec. 4.3 and Sec. 5.1] The central claim of long-term coherence rests on the assumption that the tokens generated by T2To lie in the same distribution as the tokens produced by the Video Tokenizer from real video. The paper provides no direct evidence for this: T2To is trained with a text-conditioned diffusion loss only, without a reconstruction loss, an adversarial or MMD regularizer, or any paired-data alignment against encoded tokens. If T2To's output drifts off the encoder's manifold, the cross-attention branch in To2V (Sec. 4.2) will receive out-of-distribution keys and values, degrading both per-clip fidelity and inter-clip coherence. The PCA projection fitted on 300 samples to reduce 3072 channels to 16 (Sec. 5.1) could further widen this gap, yet no reconstruction fidelity or distribution-distance metric is reported. I request a direct token-level analysis: for the same prompt, compare To2V outputs conditioned on (a) tokens encoded from real video and (b) tokens generated by T2To, and report reconstruction metrics; additionally, report a distribution distance (e.g., FID or MMD) between encoded and generated token sets. Without this, the end-to-end video results cannot localize whether the long-term consistency comes from T2To's token planning or from other components.
  2. [Sec. 5.2, Table 1; Appendix A.2 and B] The human study is the primary support for the headline preference rates, but it is small: 24 participants rate 12 videos per method in Sec. 5.2, while Table 3 in Appendix B reports a 26-participant study, an inconsistency. No confidence intervals, per-prompt breakdowns, or significance tests are provided. Given that VBench scores are mixed (ours is lower than AP-FIFO+CogVideoX on Subject Consistency and Background Consistency), the preference percentages alone are not enough to establish a robust advantage. Please report the number of participants and videos consistently, provide a per-prompt table, and compute a statistical test or at least confidence intervals for the preference proportions.
  3. [Sec. 5.2, Tables 1 and 3] The automatic evaluation does not uniformly support the claim of improved consistency. In Table 1, TokensGen is below AP-FIFO+CogVideoX on Subject Consistency (84.57 vs 86.22) and Background Consistency (92.20 vs 92.89). In Table 3, on VBench-Long, TokensGen is below the TestSet row on Subject Consistency, Background Consistency, Motion Smoothness, and Dynamic Degree, and the paper's defense is that SC/BC/TF favor static videos. The subsequent filtering of baselines with 'subpar DD and SC/BC' appears post hoc. I ask the authors to specify a pre-defined evaluation protocol or, alternatively, to report a broader set of metrics that penalize static videos less, so that the comparison is not based on a post-hoc subset.
  4. [Sec. 4.4 and Sec. 5.3] The adaptive FIFO mechanism is described only verbally: for a latent partition with fewer than f_s frames, all frames are denoised together, and for exactly f_s frames, only the noisier latter frames are updated. This leaves the precise noise schedule, update rule, and queue semantics underspecified. Since the ablation in Sec. 5.3 shows that removing adaptive padding causes visible artifacts, the method should be described with enough detail to be reproduced, e.g., a pseudocode block or an equation for the partial-partition updates.
minor comments (4)
  1. [Appendix B, Table 3 caption] The caption contains undefined placeholder tokens ("first", "second", "subpar") and the participant count differs from the main text; please clean up the caption and harmonize the number of participants across the paper.
  2. [Sec. 4.2, Figure 4] In the Resampler description, the figure labels f_s × h_s × w_s × c_s and f_r × h_r × w_r × c_s are easy to confuse because the channel dimension c_s appears in both the source and latent shapes; adding a subscript or color coding would improve readability.
  3. [Sec. 5.1] The sentence describing the training of To2V says the model is first trained on 4×5×7×3072 tokens and then on 4×8×12×3072, but the ablation in Table 2 compares 4×5×7 and 4×8×12; please clarify whether the 13×5×7 variant also follows the same progressive schedule.
  4. [General] The paper does not mention whether code or trained models will be released; given the number of hyperparameters and the reliance on a specific base model, a statement about code/data availability would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the evaluation targets are external benchmarks and a user study, and no prediction reduces by construction to fitted inputs.

full rationale

The derivation chain in TokensGen is self-contained with respect to its evaluation targets. To2V is trained to render short clips from text plus semantic tokens produced by a Video Tokenizer; T2To is trained to generate those tokens from text alone; the final evaluation compares rendered long videos against external baselines (Video-Infinity, DiTCtrl, Kling, FIFO-Diffusion variants) using VBench metrics and a user study. No step fits a parameter to the evaluation quantity and then reports that quantity as a prediction. The PCA projection in Sec. 5.1 is fitted on 300 Resampler outputs, but it is not fitted to the VBench/user-study scores; the ablation in Tab. 2 explicitly compares with and without the projection and reports similar performance, so the central claims do not depend on that fit. T2To is trained on tokens produced by the same Video Tokenizer used at inference, but this is a standard autoencoding-style setup rather than a circularity: the training target is encoded tokens, while the evaluated output is rendered-video quality against external baselines. The paper's own limitations section acknowledges that the tokens do not preserve all fine-grained details and can fail in complex scenes, which further shows the claimed coherence is not vacuously guaranteed by construction. No self-citations are load-bearing; the cited CogVideoX and FIFO-Diffusion are external prior works. Therefore no circular step can be quoted.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

TokensGen introduces no new physical or mathematical entities. Its free parameters are architectural choices (token shape, chunking, PCA projection) that were tuned during development. The central assumptions are domain assumptions about transfer from a pre-trained short-video model and about the faithfulness of the learned token representation; these are plausible but not independently validated.

free parameters (4)
  • Semantic token shape (fr x hr x wr = 4 x 8 x 12) = 4 x 8 x 12
    Selected after ablations in Sec 5.3; smaller shapes lose layout/motion fidelity and larger shapes hurt T2To convergence. This choice directly controls the information available to the To2V generator.
  • PCA projection matrix (channel dimension cr=16) = trained on 300 Resampler output samples
    Reduces channel dimension from 3072 to 16 to stabilize T2To training. Fit to 300 samples of the model's own embeddings; no cross-validation or sample-selection details.
  • Chunk length (49 frames) and max chunks (N=24) = 49 frames, N=24
    These define the maximum video length (1176 frames, about 2 minutes at 10fps) and the granularity of token sequences; chosen for memory and training stability, not derived from a requirement.
  • Adaptive FIFO lookahead schedule = not specified
    The adaptive padding strategy changes when a partition has fewer than fs frames, but the exact noise schedule and hyperparameters are not reported, making the transition quality hard to reproduce.
assumptions (6)
  • domain assumption CogVideoX-5B provides a strong short-video prior that transfers to token-conditioned generation.
    Invoked in Sec 4.2; the paper relies on the pre-trained base model for frame quality and motion.
  • domain assumption The fixed 3D causal VAE and Patchify module from CogVideoX encode sufficient spatial-temporal information for semantic control.
    Sec 4.2 states these are inherited and fixed; no analysis of what information is lost in this encoding is provided.
  • domain assumption T2To, trained on token sequences derived from per-clip encoding, learns global consistency that transfers to novel text prompts.
    Sec 4.3 trains T2To on tokens from real videos; the paper does not measure the gap between generated tokens and encoded tokens.
  • domain assumption The filtered MiraData subset (gameplay and landscape) is representative enough for general long-video consistency.
    Sec 5.1 filters 16k videos with PySceneDetect and human evaluation; test prompts are also from MiraData, so generalization outside this domain is untested.
  • domain assumption PCA projection computed on 300 samples generalizes to all token embeddings.
    Sec 5.1 states the projection is applied without observing quality loss on the full distribution; the sample selection is not described.
  • domain assumption Adaptive padding at the start of FIFO denoising aligns the initial latents with the model's learned distribution.
    Sec 4.4 argues replicated frames deviate from the training distribution; the proposed fix is heuristic and not evaluated quantitatively beyond ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TokensGen: Harnessing Condensed Tokens for Long Video Generation." pith.science (2026). https://pith.science/paper/3WKQGEPZ

@misc{pith2026250715728,
  author       = {Pith},
  title        = {Pith review of: TokensGen: Harnessing Condensed Tokens for Long Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3WKQGEPZ}},
  note         = {Machine review of arXiv:2507.15728}
}
read the original abstract

Generating consistent long videos is a complex challenge: while diffusion-based generative models generate visually impressive short clips, extending them to longer durations often leads to memory bottlenecks and long-term inconsistency. In this paper, we propose TokensGen, a novel two-stage framework that leverages condensed tokens to address these issues. Our method decomposes long video generation into three core tasks: (1) inner-clip semantic control, (2) long-term consistency control, and (3) inter-clip smooth transition. First, we train To2V (Token-to-Video), a short video diffusion model guided by text and video tokens, with a Video Tokenizer that condenses short clips into semantically rich tokens. Second, we introduce T2To (Text-to-Token), a video token diffusion transformer that generates all tokens at once, ensuring global consistency across clips. Finally, during inference, an adaptive FIFO-Diffusion strategy seamlessly connects adjacent clips, reducing boundary artifacts and enhancing smooth transitions. Experimental results demonstrate that our approach significantly enhances long-term temporal and content coherence without incurring prohibitive computational overhead. By leveraging condensed tokens and pre-trained short video models, our method provides a scalable, modular solution for long video generation, opening new possibilities for storytelling, cinematic production, and immersive simulations. Please see our project page at https://vicky0522.github.io/tokensgen-webpage/ .

Figures

Figures reproduced from arXiv: 2507.15728 by the authors.

Figure 1
Figure 1. Given the text prompt, TokensGen generates long videos of up to 2 minutes, maintaining consistent motions and content. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. CogVideoX architecture. model from f frames (f ≪ M) to generate long videos with M frames. This method progressively denoises con￾secutive frames with increasing noise levels. Given a time step schedule 0 = τ0 < τ1 < ... < τf = T, each denoising step is defined as follows: \begin {aligned} [z^{1}_{\tau _{0}};...;z^{f}_{\tau _{f-1}}]=\Phi ([z^{1}_{\tau _{1}};...;z^{f}_{\tau _{f}}],[\tau _{1};...;\tau _{f}],c;\epsilon… view at source ↗
Figure 3
Figure 3. Overview of the model. Left: Overall Framework for TokensGen. Right: Trainable Modules. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The architecture of the Resampler. ditions. This branch consists of a Semantic Token Adaptive LayerNorm (Sem AdaLN) and a 3D Cross-Attention Mod￾ule, as depicted on the right side of [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: The qualitative comparison. We recommend readers refer to our webpage for video comparisons. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Ablation study on methods of incorporating video conditions. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Ablation study FIFO. prompts with source video data allows for transformations that preserve the essential structure of the original footage while injecting new semantics. We directly combine the tar￾get text prompt and the source video as input conditions to generate …
Figure 8
Figure 8. Figure 8: Long Video Editing. then leverages condensed semantic tokens to preserve long￾term consistency across clips. Finally, our adaptive FIFO￾Diffusion strategy overcomes boundary artifacts by main￾taining temporal continuity. This pipeline efficiently scales pre-trained sho…
Figure 9
Figure 9. Figure 9: The qualitative comparison. We recommend readers refer to our webpage for video comparisons. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Gradual variations in foreground or background objects over extended sequences. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Both the FIFO strategy and the condensed tokens are insufficient to capture intricate spatial-temporal cues, leading to perfor [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LatSearch: Latent Reward-Guided Search for Faster Inference-Time Scaling in Video Diffusion

    cs.CV 2026-03 accept novelty 6.0 of 10

    LatSearch improves video diffusion quality and efficiency by scoring intermediate latents with a trained reward model and performing reward-guided resampling plus final pruning.

  2. SemanticAudio: Audio Generation and Editing in Semantic Space

    eess.AS 2026-01 conditional novelty 6.0 of 10

    SemanticAudio improves text-to-audio alignment by generating a compact semantic plan first with a Flow Matching planner and then rendering acoustic latents from that plan, and it performs training-free audio editing b...

Reference graph

Works this paper leans on

50 extracted references · 22 canonical work pages · cited by 2 Pith papers

  1. [1]

    https://kling.kuaishou.com/, 2024

    Kling. https://kling.kuaishou.com/, 2024. 1, 2, 3, 6, 12

    work page 2024

  2. [2]

    Hunyuanvideo: A systematic framework for large video gen- erative models, 2024. 2

    work page 2024

  3. [3]

    https://openai.com/sora/, 2024

    sora. https://openai.com/sora/, 2024. 1

    work page 2024

  4. [4]

    chatgpt.com, 2025

    Gpt-4o. chatgpt.com, 2025. 6, 12

    work page 2025

  5. [5]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 1, 2, 3

    arXiv 2023

  6. [6]

    Generating long videos of dynamic scenes

    Tim Brooks, Janne Hellsten, Miika Aittala, Ting-Chun Wang, Timo Aila, Jaakko Lehtinen, Ming-Yu Liu, Alexei Efros, and Tero Karras. Generating long videos of dynamic scenes. Advances in Neural Information Processing Systems, 35:31769–31781, 2022. 3

    work page 2022

  7. [7]

    Ditctrl: Exploring attention control in multi-modal dif- fusion transformer for tuning-free multi-prompt longer video generation

    Minghong Cai, Xiaodong Cun, Xiaoyu Li, Wenze Liu, Zhaoyang Zhang, Yong Zhang, Ying Shan, and Xiangyu Yue. Ditctrl: Exploring attention control in multi-modal dif- fusion transformer for tuning-free multi-prompt longer video generation. arXiv:2412.18597, 2024. 1, 2, 3, 6, 12

    arXiv 2024

  8. [8]

    Pyscenedetect

    Brandon Castellano. Pyscenedetect. https://github. com/Breakthrough/PySceneDetect, 2024. 6

    work page 2024

Show all 50 references
  1. [9]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024. 1, 2, 12

    2024

  2. [10]

    Seine: Short-to-long video diffusion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. In ICLR,

  3. [11]

    Alabdul- mohsin, Avital Oliver, Piotr Padlewski, Alexey A

    Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim M. Alabdul- mohsin, Avital Oliver, Piotr Padlewski, Alexey A. Gritsenko, Mario Lucic, and Neil Houlsby. Patch n’ pack: N...

    2023

  4. [12]

    Exvideo: Extending video diffusion models via parameter-efficient post-tuning

    Zhongjie Duan, Wenmeng Zhou, Cen Chen, Yaliang Li, and Weining Qian. Exvideo: Extending video diffusion models via parameter-efficient post-tuning. arXiv preprint arXiv:2406.14130, 2024. 2

    2024 arXiv

  5. [13]

    Karl Pearson F.R.S. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science , 2 (11):559–572, 1901. 5

    1901

  6. [14]

    Videostu- dio: Generating consistent-content and multi-scene videos

    Ting Yao Fuchen Long, Zhaofan Qiu and Tao Mei. Videostu- dio: Generating consistent-content and multi-scene videos. In ECCV, 2024. 2

    2024

  7. [15]

    Sparsectrl: Adding sparse con- trols to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse con- trols to text-to-video diffusion models. arXiv preprint arXiv:2311.16933, 2023. 2

    2023 arXiv

  8. [16]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. Interna- tional Conference on Learning Representations, 2024. 2

    2024

  9. [17]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. 2022. 3

    2022

  10. [18]

    Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tadevosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text. arXiv preprint arXiv:2403.14773, 2024. 1, 3, 13

    2024 arXiv

  11. [19]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models....

    2024

  12. [20]

    VBench++: Comprehensive and versatile bench- mark suite for video generative models

    Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, Yaohui Wang, Xinyuan Chen, Ying- Cong Chen, Limin Wang, Dahua Lin, Yu Qiao, and Zi- wei Liu. VBench++: Comprehensive and versatile bench- mark suite ...

    2024 arXiv

  13. [21]

    Pyramidal flow matching for efficient video generative modeling

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. arXiv preprint arXiv:2410.05954 ,

  14. [22]

    Miradata: A large-scale video dataset with long durations and structured captions, 2024

    Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xin- tao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long durations and structured captions, 2024. 5, 7, 12

    2024

  15. [23]

    Fifo-diffusion: Generating infinite videos from text without training

    Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. arXiv preprint arXiv:2405.11473, 2024. 1, 2, 3, 6, 13

    2024 arXiv

  16. [24]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3

    2013 arXiv

  17. [25]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Li- uhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131, 2024. 1, 2

    2024 arXiv

  18. [26]

    Videodirectorgpt: Consistent multi-scene video generation via llm-guided planning

    Han Lin, Abhay Zala, Jaemin Cho, and Mohit Bansal. Videodirectorgpt: Consistent multi-scene video generation via llm-guided planning. 2023. 2

    2023

  19. [27]

    Freelong: Training-free long video generation with spectralblend tem- poral attention

    Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Freelong: Training-free long video generation with spectralblend tem- poral attention. arXiv preprint arXiv:2407.19918, 2024. 1, 3

    2024 arXiv

  20. [28]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 2

    2024 arXiv

  21. [29]

    Tuning-free long video generation via global-local collaborative diffu- sion

    Yongjia Ma, Junlin Chen, Donglin Di, Qi Xie, Lei Fan, Wei Chen, Xiaofei Gou, Na Zhao, and Xun Yang. Tuning-free long video generation via global-local collaborative diffu- sion. arXiv preprint arXiv:2501.05484, 2025. 1, 3

    2025 arXiv

  22. [30]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2023

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xin- tao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2023. 1, 3, 13

    2023

  23. [31]

    Rolling diffusion models

    David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. arXiv preprint arXiv:2402.09470, 2024. 1, 2, 3

    2024 arXiv

  24. [32]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  25. [33]

    Video-infinity: Distributed long video generation

    Zhenxiong Tan, Xingyi Yang, Songhua Liu, and Xinchao Wang. Video-infinity: Distributed long video generation. arXiv preprint arXiv:2406.16260, 2024. 1, 2, 3, 6, 12

    2024 arXiv

  26. [34]

    Genmo Team. Mochi 1. https: // github. com/ genmoai/models, 2024. 2

    2024

  27. [35]

    Videotetris: To- wards compositional text-to-video generation.arXiv preprint arXiv:2406.04277, 2024

    Ye Tian, Ling Yang, Haotian Yang, Yuan Gao, Yufan Deng, Jingmin Chen, Xintao Wang, Zhaochen Yu, Xin Tao, Pengfei Wan, Di Zhang, and Bin Cui. Videotetris: To- wards compositional text-to-video generation.arXiv preprint arXiv:2406.04277, 2024. 3, 13

    2024 arXiv

  28. [36]

    Gen-l-video: Multi-text to long video generation via temporal co-denoising

    Fu-Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264, 2023. 1, 3

    2023 arXiv

  29. [37]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 1, 2

    2023 arXiv

  30. [38]

    Lavie: High-quality video generation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. IJCV, 2024. 1, 2

    2024

  31. [39]

    Loong: Generating minute-level long videos with autoregressive lan- guage models

    Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive lan- guage models. arXiv preprint arXiv:2410.02757, 2024. 3

    2024 arXiv

  32. [40]

    Progressive autoregressive video diffusion models

    Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou. Progressive autoregressive video diffusion models. arXiv preprint arXiv:2410.08151, 2024. 2, 3

    2024 arXiv

  33. [41]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Xintao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors. arXiv preprint arXiv:2310.12190, 2023. 1, 2, 3

    arXiv

  34. [42]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 1, 2, 3, 4, 5, 14

    2024 arXiv

  35. [43]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation.arXiv preprint arXiv:2303.12346, 2023

    Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, et al. Nuwa-xl: Diffusion over diffusion for extremely long video generation.arXiv preprint arXiv:2303.12346, 2023. 3

    2023 arXiv

  36. [44]

    From slow bidirectional to fast causal video generators

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Free- man, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast causal video generators. arXiv preprint arXiv:2412.07772, 2024. 3

    2024

  37. [45]

    Mora: Enabling gen- eralist video generation via a multi-agent framework

    Zhengqing Yuan, Ruoxi Chen, Zhaoxu Li, Haolong Jia, Li- fang He, Chi Wang, and Lichao Sun. Mora: Enabling gen- eralist video generation via a multi-agent framework. arXiv preprint arXiv:2403.13248, 2024. 2

    2024 arXiv

  38. [46]

    I2vgen-xl: High-quality image-to-video syn- thesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qing, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video syn- thesis via cascaded diffusion models. 2023. 1, 3

    2023

  39. [47]

    Moviedreamer: Hier- archical generation for coherent long visual sequence, 2024

    Canyu Zhao, Mingyu Liu, Wen Wang, Jianlong Yuan, Hao Chen, Bo Zhang, and Chunhua Shen. Moviedreamer: Hier- archical generation for coherent long visual sequence, 2024. 1, 2

    2024

  40. [48]

    Open-sora: Democratizing efficient video production for all, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, 2024. 1, 2

    2024

  41. [49]

    Storydiffusion: Consistent self-attention for long-range image and video generation

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Ji- ashi Feng, and Qibin Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. NeurIPS 2024, 2024. 3

    2024

  42. [50]

    Vlogger: Make your dream a vlog

    Shaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, and Yali Wang. Vlogger: Make your dream a vlog. arXiv preprint arXiv:2401.09414, 2024. 2 Appendix Overview. The appendix includes sections as follows: • Details of Comparison Study (Appendix A). • Addi...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.