Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 0.6B-parameter text-to-video model generates a five-second clip on an iPhone 16 Pro Max in under five seconds, with quality scores near the top of the open-source comparison table.

desk verdict The speed claim is solid; the quality claim is orphaned from the deployed model. read the letter →

arxiv 2412.10494 v2 pith:AYDQYSLE submitted 2024-12-13 cs.CV cs.AIcs.LGcs.PF

Yushu Wu , Zhixing Zhang , Yanyu Li , Yanwu Xu , Anil Kag , Yang Sui , Huseyin Coskun , Ke Ma
show 6 more authors
Aleksei Lebedev Ju Hu Dimitris Metaxas Yanzhi Wang Sergey Tulyakov Jian Ren
This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.PF MSC 68T0768T45
keywords mobilevideogenerationdiffusionmodelsarchitecturesearchadversarialdistillationtext-to-videolatencyoptimizationmodelpruningon-deviceinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a 0.6B-parameter text-to-video diffusion model can generate a five-second, 512x512 video on an iPhone 16 Pro Max in under five seconds, while scoring on par with much larger server-side video models. The route has three stages: prune a Stable Diffusion 1.5 image UNet into a compact spatial backbone; search among temporal layer types, placements, and counts under measured mobile latency and memory limits; then adversarially fine-tune the resulting model so it needs only four denoising steps and no classifier-free guidance. The authors also compress the video VAE decoder, cutting its 17-frame decode time from 27.2 seconds to 0.5 seconds. If the claim holds, on-device text-to-video generation moves from impractical to near-realtime, changing where and how such models can be deployed.

What carries the argument

The engine of the result is a three-part combination. First, a latency-memory joint evolutionary architecture search picks which temporal layers (1D or 3D self-attention, cross-attention, or convolution) to place at which UNet blocks, ranking candidates by the ratio of VBench score gain to measured latency and memory. Second, a latent adversarial fine-tuning recipe with a spatial-temporal discriminator head and joint image-video training distills the 25-step diffusion process into 4 steps without classifier-free guidance. Third, a decoupled and pruned spatial-temporal VAE decoder reduces the decoding bottleneck from 27.2 seconds to 0.5 seconds for a 17-frame clip. Together these components fit a video generation pipeline inside the memory and latency budget of an Apple A18 Pro chip.

What would settle it

Run the 4-step model and a strong server baseline such as CogVideoX-2B on the same set of prompts and collect pairwise human preference judgments on motion, prompt alignment, and aesthetics; if human judges clearly prefer the baseline despite close VBench scores, the paper's quality claim is undercut. A second decisive test is timing the full pipeline on an iPhone 16 Pro Max from text input to displayed video; if end-to-end time exceeds five seconds on the released build, the headline speed claim fails.

Watch

Extended reading notes

Core claim

The paper argues that a video diffusion model small and fast enough for a phone can be obtained without training from scratch. Starting from a pruned image-generation UNet, the authors search for the best way to add temporal modeling under strict hardware budgets, then distill the model from 25 denoising steps to 4 using an adversarial objective with a unified spatial-temporal discriminator head. The resulting model, at 0.6B parameters, generates 51 frames at 512x512 resolution in about 4.1 seconds on an iPhone 16 Pro Max, according to the paper's benchmark, and reaches a VBench total score of 81.14, which sits above several open-source server-side models in the reported table. The paper frames this as the first demonstration of real-time text-to-video generation on a mobile device.

Load-bearing premise

The quality comparison rests on VBench, and VBench is also the metric used during the architecture search to choose the final model; if VBench scores diverge from what human viewers actually prefer, the on-par quality conclusion is not supported.

Editorial extensions

If this is right

  • If the reported speed holds on the iPhone 16 Pro Max, text-to-video generation no longer requires a cloud round-trip for short clips, which makes private, offline video creation feasible on consumer hardware.
  • The 4-step adversarial distillation recipe, combined with the compact backbone, suggests that step-count reduction techniques developed for large image models can transfer to much smaller video models when the discriminator is redesigned for joint image-video training.
  • The latency-memory search criterion is general enough that the same methodology could be applied to other hardware targets, such as Android flagships, laptops, or automotive chips, by rebuilding the latency lookup table for those devices.
  • Compressing the VAE decoder by more than fifty times shows that decoding, not just denoising, is a major bottleneck for edge video generation, and that it can be addressed separately from the denoiser.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same pipeline could be pushed toward true real-time streaming video generation, where frames are produced continuously rather than as one fixed clip; the authors explicitly note that their current system is conventional one-shot generation, so this is a natural next step rather than a demonstrated capability.
  • Applying the same latency-memory search and adversarial distillation to a DiT-based video backbone would be a telling test of whether the conclusions generalize beyond UNets, since the paper restricts itself to a UNet and cites quadratic token complexity as the reason.
  • A direct implication of the search metric being VBench is that the architecture is optimized for what VBench measures; if future benchmarks better capture motion semantics or long-range consistency, the searched architecture may need to be revisited.
  • The reported per-step latency of about one second suggests that modest further gains in neural engine efficiency, weight quantization, or step reduction could push generation time to the two-second range on the same hardware.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes SnapGen-V, a framework for mobile text-to-video generation. Starting from a pruned Stable Diffusion v1.5 spatial backbone, the method performs an evolutionary architecture search over temporal layer types and placements using a hardware latency/memory lookup table and VBench-based quality scores, trains the final model with image-video joint training, and then applies latent adversarial fine-tuning with unified spatial-temporal discriminator heads to reduce sampling to 4 denoising steps. It also compresses the OpenSora spatial-temporal VAE decoder. The headline result is a 0.6B-parameter model that generates a 5-second 512x512 video on an iPhone 16 Pro Max in 4.12 seconds (Appendix F) and achieves a VBench total score of 81.14 (Table 2), claimed to be on par with or better than server-side models.

Significance. If the claims hold, this is a substantial engineering contribution: it is the first demonstration of text-to-video diffusion on a phone, with a measured latency under five seconds, a systematic temporal-layer search, and a plausible joint image-video adversarial distillation recipe. The speed claim rests on an independent hardware benchmark rather than on a fitted parameter, and the VAE decoder compression is quantified with reconstruction metrics. The architecture ablations and the comparison against SF-V-style discriminator heads are informative. However, the quality evidence is weaker: the VBench-based architecture selection creates a selection-artifact risk for the final VBench comparison, and the evaluated checkpoint differs from the deployed mobile checkpoint. Releasing the code and model, evaluating the exact deployed artifact, and reporting end-to-end latency would materially strengthen the paper.

major comments (4)
  1. [Section 4 (Evaluation) and Appendix F] The central quality claim and the central speed claim are not shown to describe the same model. The VBench evaluation in Section 4 generates 120-frame horizontal videos at 432x768 and saves them at 5 seconds and 24 fps, while Appendix F reports a mobile demo at 51 frames, 512x512, 10 fps, and states that 'the model is further finetuned with video datasets with a framerate of 10 fps.' Consequently, the 81.14 VBench total in Table 2 and the user-study win rates in Table 3 were not measured on the checkpoint that runs on the iPhone in 4.12 seconds. Please either provide VBench and user-study numbers for the exact deployed 51-frame/10-fps checkpoint, or explicitly restrict the on-par-quality claim to the non-deployed evaluation setting.
  2. [Section 3.2 and Appendix A] The architecture search in Section 3.2 selects temporal layers by training candidates for 20K iterations and ranking them with VBench scores (Appendix A specifies a VBench subset over 268 prompts), and Table 2 then reports the final model's VBench total. The quality comparison is therefore partly a selection artifact, because the search objective and the final evaluation metric are the same. The paper should report the variance of VBench scores across evaluation seeds or prompt sets, evaluate held-out architectures that were not selected by VBench, or support the final architecture choice with a human study at the deployed operating point.
  3. [Appendix F] The sub-5-second speed is assembled from component measurements (four denoising steps at 1.02 seconds each, plus 0.5 seconds for the VAE decoder and 6 milliseconds for the text encoder) rather than from an end-to-end wall-clock measurement of a complete run. On-device pipelines can have non-compositional costs from model loading, the two-part split of the exported CoreML model, memory pressure, and scheduling. Please report the measured end-to-end latency of the full pipeline on the iPhone 16 Pro Max, ideally with a per-prompt distribution, in addition to the component benchmarks.
  4. [Section 4.2, Table 2] Many of the VBench differences in Table 2 are small (e.g., 81.14 vs. 81.01 for T2V-Turbo and 81.14 vs. 80.91 for CogVideoX-2B), and no error bars, number of seeds, or significance tests are provided. Without such information, the claim that a 0.6B 4-step mobile model 'achieves higher total score compared to recent arts' is not robustly supported. Please include confidence intervals or per-prompt score distributions for at least the main comparisons.
minor comments (5)
  1. [Appendix D] The text 'OponSora-v1.2' should read 'OpenSora-v1.2'.
  2. [Table 2] The header line 'color spatial appearance temporal overall' is missing separators; please format it so the VBench sub-metric names are clearly distinguishable.
  3. [Table 3] The user study does not state the number of prompts, annotators, or pairwise comparisons; please include these details so the win rates can be interpreted.
  4. [Table 4] The row label 'Scaling' and the column abbreviations '∆T', '∆AQ', '∆IQ', '∆DD', and '∆MS' are hard to parse; please define every symbol in the caption and state the baseline latency explicitly.
  5. [Section 3.3] The description of the temporal self-attention in the discriminator heads should specify whether it operates over the full frame sequence or over chunks, since memory constraints are central to the mobile design.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: VBench is used as both the architecture-search objective and the final quality metric, so the reported VBench quality score is partly a selection artifact; the speed claim is independent.

  1. fitted input called prediction [Appendix A (Search Algorithm); Sec. 3.2 (Latency and Memory Guided Architecture Search); Sec. 4 (Evaluation, Table 2)]
    "We choose the Vbench score [19] to evaluate the quantitative performance of each architecture, and we specifically focus on the average score of the overall consistency, the object class , and the color score instead of the complete benchmark to reduce the evaluation time. ... The architecture candidate is trained on precomputed video latents for 20K iterations with the spatial backbone frozen, and is evaluated on VBench [19] to obtain the scores as the quality metric. ... The model is evaluated following the standard benchmarking procedure of VBench [19]."

    The final quality evidence (VBench total 81.14 in Tables 1 and 2) is the same benchmark used as the objective in the evolutionary architecture search and, in Table 5, to select the adversarial noise-location hyperparameter m. The architecture and m are therefore fitted to VBench, and the reported VBench score is a selection artifact rather than an independent confirmation of the design. This is the fitted-input-called-prediction pattern: the 'quality' result is the optimized objective itself. The iPhone speed measurement (1.02 s per denoising step) and the user study provide independent content, so the circularity is partial and confined to the quality claim.

full rationale

The central speed claim is an independent hardware measurement: Appendix F reports 1.02 s per denoising step, 0.5 s for the VAE decoder, and 6 ms for the text encoder on an iPhone 16 Pro Max, totaling under five seconds. That result does not reduce to any fitted parameter. The quality claim is partially circular: the temporal architecture is selected by VBench scores after 20K-iteration candidate training (Sec. 3.2, Appendix A), the noise-level hyperparameter m is chosen by VBench (Table 5), and the same VBench benchmark is then used to report the final model's quality (Table 2). Thus part of the reported score is a selection artifact. The paper's user study (Table 3) and VAE/backbone ablations add some independent signal, but they do not eliminate the overlap between the search objective and the final evaluation metric. No load-bearing uniqueness theorem or self-citation chain is present; citations to the authors' prior SF-V and SnapFusion work are supported by in-paper descriptions and measurements. Separately, the VBench evaluation (120 frames, 432x768, 24 fps) and the mobile demo (51 frames, 512x512, 10 fps, further finetuned) describe different operating points; this is a validity concern rather than a circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on tuned hyperparameters (m, s, k, EMA rate), search settings (VBench as a proxy, 20K-iteration training), and inherited components from prior work (pruned SD1.5, OpenSora VAE, SF-V distillation). No new physical or model entities are introduced.

free parameters (6)
  • Discriminator timestep distribution location m = -1
    m controls the noise level of real and fake samples passed to the discriminator. It was swept over -2, -1, 0, and 1 in Table 5 and selected for best semantic/multi-object score rather than best total VBench.
  • Discriminator timestep distribution scale s = 1
    Set following the logit-normal sampling in Eq. (3) and not ablated; used in all final experiments.
  • Number of generator timesteps k = 4
    Chosen as the inference step count for adversarial fine-tuning; Table 6 shows 1 and 2 steps degrade performance, so 4 is a selected operating point.
  • CFG scale during architecture search = 7
    Used in Appendix A for VBench scoring of searched candidates (25 denoising steps), while the final model runs with 4 steps and no CFG. The mismatch between search and deployment settings is a potential gap.
  • Training iterations for architecture search candidates = 20000
    Search-quality scores are obtained after only 20K iterations with a frozen spatial backbone; the final model trains much longer, so search scores are weak predictors of final quality.
  • EMA rate = 0.95
    Set following SF-V [74]; this hyperparameter affects distillation stability and was not independently tuned.
assumptions (5)
  • domain assumption VBench is a valid proxy for text-to-video generation quality.
    VBench is used both as the search objective (Appendix A) and as the final evaluation metric (Table 2). If VBench does not reflect human preference, the on-par quality claim weakens.
  • domain assumption Short-training architecture search scores transfer to full training.
    Architecture candidates are evaluated after 20K iterations with a frozen backbone; the final model is trained much longer. The paper does not analyze the correlation between search-time VBench scores and final-model VBench scores.
  • domain assumption The SnapFusion-style pruning recipe preserves enough image quality to serve as a video backbone.
    The spatial backbone is obtained by pruning SD1.5 following prior work [11, 29]; pruning is performed by the same research group and no independent replication is provided.
  • domain assumption The OpenSora spatial-temporal VAE remains accurate after 54.5x decoder compression.
    The decoder is compressed and evaluated only via PSNR, SSIM, LPIPS, and VBench, not via a human study on reconstructed video.
  • domain assumption Hinge-loss adversarial training with a frozen UNet encoder as discriminator backbone is stable for video distillation.
    This design is inherited from SF-V [74] and shown to work empirically here, but there is no theoretical guarantee of stability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device." pith.science (2026). https://pith.science/paper/AYDQYSLE

@misc{pith2026241210494,
  author       = {Pith},
  title        = {Pith review of: SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYDQYSLE}},
  note         = {Machine review of arXiv:2412.10494}
}
read the original abstract

We have witnessed the unprecedented success of diffusion-based video generation over the past year. Recently proposed models from the community have wielded the power to generate cinematic and high-resolution videos with smooth motions from arbitrary input prompts. However, as a supertask of image generation, video generation models require more computation and are thus hosted mostly on cloud servers, limiting broader adoption among content creators. In this work, we propose a comprehensive acceleration framework to bring the power of the large-scale video diffusion model to the hands of edge users. From the network architecture scope, we initialize from a compact image backbone and search out the design and arrangement of temporal layers to maximize hardware efficiency. In addition, we propose a dedicated adversarial fine-tuning algorithm for our efficient model and reduce the denoising steps to 4. Our model, with only 0.6B parameters, can generate a 5-second video on an iPhone 16 PM within 5 seconds. Compared to server-side models that take minutes on powerful GPUs to generate a single video, we accelerate the generation by magnitudes while delivering on-par quality.

Figures

Figures reproduced from arXiv: 2412.10494 by the authors.

Figure 1
Figure 1. Example generation results from our mobile text-to-video model. Our model can generate high-quality and motion consistent 5-second videos on a mobile device (e.g., iPhone 16 Pro Max) within 5 seconds. Abstract We have witnessed the unprecedented success of diffusion-based video generation over the past year. Re￾cently proposed models from the community have wielded the power to generate cinematic and high-resolution… view at source ↗
Figure 2
Figure 2. Framework Overview. In the Latency and Memory Guided Architecture Search, we freeze the pretrained efficient spatial layers and conduct evolutionary search over temporal layer based on the memory and latency constraint. During the Adversarial Fine-tuning stage, we initialize the discriminator with the weights from the text-to-video model trained in the first stage. The discriminator employs the encoder of the UNet a… view at source ↗
Figure 3
Figure 3. Computation Complexity and Memory Consumption Analysis. The computation complexity and memory consumption of different temporal layer for various input size. The temporal dimension is fixed to 12 for simplicity. memory footprint for each design candidate can be eas￾ily profiled, as in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Latent Adversarial Fine-tuning. Given a latent x0 and a noise latent ϵ, we obtain the intermediate noisy latent xt through a forward diffusion process. The generator then predicts the veloc￾ity as Gθ (xt, t). Using the predicted velocity, we compute xˆ0 and calculate t…
Figure 5
Figure 5. Figure 5: Video generation on various domains. We employ our model to synthesize videos across diverse domains, with each video containing 120 frames at a resolution of 432 × 768. All results are generated through a 4-step inference process. The results demonstrate that our mode…
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices

    cs.CV 2025-02 conditional novelty 6.0 of 10

    A training-free pipeline makes diffusion text-to-video generation run on an iPhone 15 Pro with quality close to GPU output, at the cost of slower generation.

Reference graph

Works this paper leans on

79 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Dream machine

    Luma AI. Dream machine. https://lumalabs.ai/ dream-machine, . 2

    work page

  2. [2]

    Pika 1.5

    Pika AI. Pika 1.5. https://pika.art/try, . 2

    work page

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. ArXiv preprint, abs/2311.15127, 2023. 1, 2, 3, 5

    arXiv 2023

  4. [4]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, CVPR 2023, Vancou- ver, BC, Canada, June 17-24, 2023 , pages 22563–22575. IEEE, 2023. 1, 2

    work page 2023

  5. [5]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. ArXiv preprint, abs/2310.19512, 2023. 2

    arXiv

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7310– 7320, 2024. 2, 7

    work page 2024

  7. [7]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13320–13331, 2024. 2

    work page 2024

  8. [8]

    EDT: An Efficient Diffusion Transformer Framework Inspired by Human-like Sketching

    Xinwang Chen, Ning Liu, Yichen Zhu, Feifei Feng, and Jian Tang. Edt: An efficient diffusion transformer frame- work inspired by human-like sketching. ArXiv preprint , abs/2410.23788, 2024. 1, 2

    work page Pith review arXiv 2024

Show all 79 references
  1. [9]

    Swiftbrush v2: Make your one-step diffusion model better than its teacher

    Trung Dao, Thuan Hoang Nguyen, Thanh Le, Duc Vu, Khoi Nguyen, Cuong Pham, and Anh Tran. Swiftbrush v2: Make your one-step diffusion model better than its teacher. In European Conference on Computer Vision, pages 176–192. Springer, 2025. 3

    2025

  2. [10]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  3. [11]

    Structural pruning for diffusion models

    Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models. In Advances in Neural Infor- mation Processing Systems 36: Annual Conference on Neu- ral Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. 2, 3

    2023

  4. [12]

    Wave: Warping ddim inversion features for zero-shot text-to-video editing

    Yutang Feng, Sicheng Gao, Yuxiang Bao, Xiaodi Wang, Shumin Han, Juan Zhang, Baochang Zhang, and Angela Yao. Wave: Warping ddim inversion features for zero-shot text-to-video editing. In European Conference on Computer Vision, pages 38–55. Springer, 2025. 1

    2025

  5. [13]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. ArXiv preprint, abs/2307.04725, 2023. 1, 2, 3

    2023 arXiv

  6. [14]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330–348. Springer, 2025. 2

    2025

  7. [15]

    Photorealistic video generation with diffusion models.ArXiv preprint, abs/2312.06662, 2023

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and Jos ´e Lezama. Photorealistic video generation with diffusion models.ArXiv preprint, abs/2312.06662, 2023. 2

    2023 arXiv

  8. [16]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. ArXiv preprint, abs/2207.12598, 2022. 2

    2022 arXiv

  9. [17]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. In Advances in Neural Informa- tion Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, De- cember 6-12, 2020, virtual, 2020. 2, 3

    2020

  10. [18]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Ki- gali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. 2

    2023

  11. [19]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models....

    2024

  12. [20]

    Dreammotion: Space-time self-similar score distillation for zero-shot video editing

    Hyeonho Jeong, Jinho Chang, Geon Yeong Park, and Jong Chul Ye. Dreammotion: Space-time self-similar score distillation for zero-shot video editing. ArXiv preprint , abs/2403.12002, 2024. 1

    2024 arXiv

  13. [21]

    Pyramidal flow matching for efficient video generative modeling

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. ArXiv preprint, abs/2410.05954, 2024. 2

    2024

  14. [22]

    Simple reflow: Improved techniques for fast flow models

    Beomsu Kim, Yu-Guan Hsieh, Michal Klein, Marco Cuturi, Jong Chul Ye, Bahjat Kawar, and James Thornton. Simple reflow: Improved techniques for fast flow models. ArXiv preprint, abs/2410.07815, 2024. 3

    2024 arXiv

  15. [23]

    Fifo-diffusion: Generating infinite videos from text without training

    Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. ArXiv preprint, abs/2405.11473, 2024. 2

    2024 arXiv

  16. [24]

    Kuaishou. Kling. https://kling.kuaishou.com/ en. 2

  17. [25]

    Vivid-1-to-3: Novel view synthesis with video diffusion models

    Jeong-gi Kwak, Erqun Dong, Yuhe Jin, Hanseok Ko, Shweta Mahajan, and Kwang Moo Yi. Vivid-1-to-3: Novel view synthesis with video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6775–6785, 2024. 1

    2024

  18. [26]

    Open-sora-plan, 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 2

    2024

  19. [27]

    T2v- turbo: Breaking the quality bottleneck of video consis- tency model with mixed reward feedback

    Jiachen Li, Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sug- ato Basu, Wenhu Chen, and William Yang Wang. T2v- turbo: Breaking the quality bottleneck of video consis- tency model with mixed reward feedback. ArXiv preprint, abs/2405.18750, 2024. 2, 7

    2024 arXiv

  20. [28]

    T2v- turbo-v2: Enhancing video generation model post-training through data, reward, and conditional guidance design

    Jiachen Li, Qian Long, Jian Zheng, Xiaofeng Gao, Robinson Piramuthu, Wenhu Chen, and William Yang Wang. T2v- turbo-v2: Enhancing video generation model post-training through data, reward, and conditional guidance design. ArXiv preprint, abs/2410.05677, 2024. 2

  21. [29]

    Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds. In Advances in Neural Information Pro- cessing Systems 36: Annual Conference on Neural In...

    2023

  22. [30]

    Flowvid: Taming imperfect optical flows for consistent video-to-video synthesis

    Feng Liang, Bichen Wu, Jialiang Wang, Licheng Yu, Kun- peng Li, Yinan Zhao, Ishan Misra, Jia-Bin Huang, Peizhao Zhang, Peter Vajda, et al. Flowvid: Taming imperfect optical flows for consistent video-to-video synthesis. InProceedings of the IEEE/CVF Conference on Computer Visi...

    2024

  23. [31]

    Geometric gan

    Jae Hyun Lim and Jong Chul Ye. Geometric gan. ArXiv preprint, abs/1705.02894, 2017. 5

    2017 arXiv

  24. [32]

    Animatediff-lightning: Cross-model diffusion distillation

    Shanchuan Lin and Xiao Yang. Animatediff-lightning: Cross-model diffusion distillation. ArXiv preprint , abs/2403.12706, 2024. 3

    2024 arXiv

  25. [33]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. 3

    2023

  26. [34]

    Free- long: Training-free long video generation with spectralblend temporal attention

    Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Free- long: Training-free long video generation with spectralblend temporal attention. ArXiv preprint, abs/2407.19918, 2024. 2

    2024 arXiv

  27. [35]

    Osv: One step is enough for high-quality image to video generation

    Xiaofeng Mao, Zhengkai Jiang, Fu-Yun Wang, Wenbing Zhu, Jiangning Zhang, Hao Chen, Mingmin Chi, and Yabiao Wang. Osv: One step is enough for high-quality image to video generation. ArXiv preprint, abs/2409.11367, 2024. 3

    2024 arXiv

  28. [36]

    Codi: Conditional diffusion distillation for higher-fidelity and faster image generation

    Kangfu Mei, Mauricio Delbracio, Hossein Talebi, Zhengzhong Tu, Vishal M Patel, and Peyman Milanfar. Codi: Conditional diffusion distillation for higher-fidelity and faster image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  29. [37]

    Hailuo ai

    MiniMax. Hailuo ai. https://hailuoai.video/. 2

  30. [38]

    Openvid-1m: A large-scale high-quality dataset for text-to- video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to- video generation. ArXiv preprint, abs/2407.02371, 2024. 2

    2024 arXiv

  31. [39]

    Video generation models as world simula- tors

    OpenAI. Video generation models as world simula- tors. https : / / openai . com / index / video - generation-models-as-world-simulators/ . 2

  32. [40]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1- 6, 2023, pages 4172–4182. IEEE, 2023. 1, 2, 3

    2023

  33. [41]

    Movie gen: A cast of media foundation models

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. ArXiv preprint, abs/2410.13720,

    arXiv

  34. [42]

    Hier- archical spatio-temporal decoupling for text-to-video gener- ation

    Zhiwu Qing, Shiwei Zhang, Jiayu Wang, Xiang Wang, Yujie Wei, Yingya Zhang, Changxin Gao, and Nong Sang. Hier- archical spatio-temporal decoupling for text-to-video gener- ation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 6635–6645,

  35. [43]

    Freenoise: Tuning- free longer video diffusion via noise rescheduling

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xin- tao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning- free longer video diffusion via noise rescheduling. ArXiv preprint, abs/2310.15169, 2023. 2

    2023 arXiv

  36. [44]

    Customize-a-video: One-shot motion customization of text- to-video diffusion models

    Yixuan Ren, Yang Zhou, Jimei Yang, Jing Shi, Difan Liu, Feng Liu, Mingi Kwon, and Abhinav Shrivastava. Customize-a-video: One-shot motion customization of text- to-video diffusion models. ArXiv preprint, abs/2402.14780,

    arXiv

  37. [45]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 3

    2022

  38. [46]

    Gen-3 alpha

    Runway. Gen-3 alpha. https://runwayml.com/ research/introducing-gen-3-alpha . 2

  39. [47]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. InThe Tenth International Conference on Learning Representations, ICLR 2022, Vir- tual Event, April 25-29, 2022. OpenReview.net, 2022. 3

    2022

  40. [48]

    Projected gans converge faster

    Axel Sauer, Kashyap Chitta, Jens M ¨uller, and Andreas Geiger. Projected gans converge faster. In Advances in Neu- ral Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 17480–17492,

    2021

  41. [49]

    Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis

    Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila. Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis. In International Confer- ence on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, pages 30105–30118. ...

    2023

  42. [50]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. ArXiv preprint, abs/2311.17042, 2023. 3, 5

    2023 arXiv

  43. [51]

    Fast high- resolution image synthesis with latent adversarial diffusion distillation

    Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high- resolution image synthesis with latent adversarial diffusion distillation. ArXiv preprint, abs/2403.12015, 2024. 3, 4

    2024 arXiv

  44. [52]

    Improved tech- niques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved tech- niques for training consistency models. ArXiv preprint , abs/2310.14189, 2023. 3

    2023 arXiv

  45. [53]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Confer- ence on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, pages 32211–32252. PMLR, 2023. 3

    2023

  46. [54]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  47. [55]

    Mochi, 2024

    Genmo Team. Mochi, 2024. 1, 2

    2024

  48. [56]

    Gemini: a family of highly capable multimodal models.ArXiv preprint, abs/2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.ArXiv preprint, abs/2312.11805, 2023. 1, 2

    2023 arXiv

  49. [57]

    Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer...

    2025

  50. [58]

    Animatelcm: Accelerating the animation of personalized diffusion mod- els and adapters with decoupled consistency learning

    Fu-Yun Wang, Zhaoyang Huang, Xiaoyu Shi, Weikang Bian, Guanglu Song, Yu Liu, and Hongsheng Li. Animatelcm: Accelerating the animation of personalized diffusion mod- els and adapters with decoupled consistency learning. ArXiv preprint, abs/2402.00769, 2024. 3, 7

    2024 arXiv

  51. [59]

    Rectified diffusion: Straightness is not your need in rectified flow

    Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li. Rectified diffusion: Straightness is not your need in rectified flow. ArXiv preprint, abs/2410.07303,

    arXiv

  52. [60]

    Videolcm: Video latent consistency model

    Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang. Videolcm: Video latent consistency model. ArXiv preprint, abs/2312.09109,

    arXiv

  53. [61]

    Mo- tionbooth: Motion-aware customized text-to-video genera- tion

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Mo- tionbooth: Motion-aware customized text-to-video genera- tion. ArXiv preprint, abs/2406.17758, 2024. 2

    2024 arXiv

  54. [62]

    Tune-a-video: One-shot tun- ing of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tun- ing of image diffusion models for text-to-video generation. In IEEE/CVF International Conference on Computer Visi...

    2023

  55. [63]

    Freeinit: Bridging initialization gap in video dif- fusion models

    Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initialization gap in video dif- fusion models. In European Conference on Computer Vision, pages 378–394. Springer, 2025. 2

    2025

  56. [64]

    Sana: Ef- ficient high-resolution image synthesis with linear diffusion transformers

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, and Song Han. Sana: Ef- ficient high-resolution image synthesis with linear diffusion transformers. ArXiv preprint, abs/2410.10629, 2024. 1

    arXiv

  57. [65]

    Ufogen: You forward once large scale text-to-image gener- ation via diffusion gans

    Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou. Ufogen: You forward once large scale text-to-image gener- ation via diffusion gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8196–8206, 2024. 3

    2024

  58. [66]

    Consistency flow matching: Defining straight flows with velocity consistency

    Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Er- mon, and Bin Cui. Consistency flow matching: Defining straight flows with velocity consistency. ArXiv preprint , abs/2407.02398, 2024. 3

    2024 arXiv

  59. [67]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. ArXiv preprint, abs/2408.06072, 2024. 1, 2, 3, 7, 5

    2024 arXiv

  60. [68]

    Im- proved distribution matching distillation for fast image syn- thesis

    Tianwei Yin, Micha ¨el Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Im- proved distribution matching distillation for fast image syn- thesis. ArXiv preprint, abs/2405.14867, 2024. 3

    2024 arXiv

  61. [69]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Micha ¨el Gharbi, Richard Zhang, Eli Shecht- man, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 6613–6623...

    2024

  62. [70]

    Ditfastattn: Attention compression for diffusion transformer models

    Zhihang Yuan, Pu Lu, Hanling Zhang, Xuefei Ning, Lin- feng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattn: Attention compression for diffusion transformer models. ArXiv preprint, abs/2406.08552, 2024. 1, 2

    2024 arXiv

  63. [71]

    Make pixels dance: High- dynamic video generation

    Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, and Hang Li. Make pixels dance: High- dynamic video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8850–8860, 2024. 2

    2024

  64. [72]

    Motion consistency model: Accel- erating video diffusion with disentangled motion-appearance distillation

    Yuanhao Zhai, Kevin Lin, Zhengyuan Yang, Linjie Li, Jian- feng Wang, Chung-Ching Lin, David Doermann, Junsong Yuan, and Lijuan Wang. Motion consistency model: Accel- erating video diffusion with disentangled motion-appearance distillation. ArXiv preprint, abs/2406.06890, 2024. 3

    2024 arXiv

  65. [73]

    Fast and memory-efficient video dif- fusion using streamlined inference

    Zheng Zhan, Yushu Wu, Yifan Gong, Zichong Meng, Zhenglun Kong, Changdi Yang, Geng Yuan, Pu Zhao, Wei Niu, and Yanzhi Wang. Fast and memory-efficient video dif- fusion using streamlined inference. In The Thirty-eighth An- nual Conference on Neural Information Processing Systems. 2

  66. [74]

    Sf-v: Single forward video generation model

    Zhixing Zhang, Yanyu Li, Yushu Wu, Yanwu Xu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Dimitris Metaxas, et al. Sf-v: Single forward video generation model. ArXiv preprint, abs/2406.04324,

    arXiv

  67. [75]

    Avid: Any-length video inpainting with dif- fusion model

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, and Licheng Yu. Avid: Any-length video inpainting with dif- fusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 71...

    2024

  68. [76]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Jun- hao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In European Conference on Computer Vision, pages 273–290. Springer, 2025. 2

    2025

  69. [77]

    Open-sora: Democratizing efficient video production for all, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, 2024. 2, 3, 4, 5, 8, 1

    2024

  70. [78]

    Al- legro: Open the black box of commercial-level video gener- ation model

    Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Al- legro: Open the black box of commercial-level video gener- ation model. ArXiv preprint, abs/2410.15458, 2024. 1, 2

    2024 arXiv

  71. [79]

    Golden Retriever and French Bulldog go through a dark corridor of abandoned alien spacecraft. Sci-fi horror movie style,

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Ji- ashi Feng, and Qibin Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. NeurIPS 2024, 2024. 2 SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device Supplementary Ma...

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.