Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

MoViE: Mobile Diffusion for Video Editing

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that instruction-based video editing can run on a mobile phone at 12 frames per second, and shows a system that reaches this speed by pruning the denoising network, collapsing classifier-free guidance to one pass, and…

desk verdict Solid mobile-editing engineering with a real capability, but the headline 12 fps is single-frame denoising only, not end-to-end editing. read the letter →

arxiv 2412.06578 v1 pith:WIUNA5U2 submitted 2024-12-09 cs.CV

Adil Karjauv , Noor Fathima , Ioannis Lelekas , Fatih Porikli , Amir Ghodrati , Amirhossein Habibian This is my paper · ORCID
classification cs.CV
keywords mobilevideoeditingdiffusionmodelsadversarialdistillationclassifier-freeguidanceon-deviceinferenceinstruction-basedsingle-stepsamplingefficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to establish that instruction-based video editing can run in near-real time on a mobile phone, not just on server GPUs. The authors introduce MoViE, a video-editing diffusion pipeline built on InstructPix2Pix with cross-frame attention, and claim it edits a 120-frame video at 512×384 resolution in 9.6 seconds on a Snapdragon 8 Gen 3 device, i.e., about 12 frames per second. They achieve this through three optimizations: pruning attention layers in the denoiser and swapping in a lightweight autoencoder, distilling the three-pass multi-modal classifier-free guidance into a single forward pass, and distilling the multi-step denoiser into a single adversarial step that still accepts text and image guidance scales. If the claim holds, common video-editing tasks such as style transfer and attribute changes could be done locally and privately on consumer hardware without a cloud round-trip.

What carries the argument

Three components carry the result. Mobile-Pix2Pix is a pruned InstructPix2Pix U-Net with the lightweight deterministic TAESD autoencoder, which together cut the per-frame FLOPs of network and autoencoder substantially. Multimodal guidance distillation trains a student network that takes the image guidance scale and text guidance scale as sinusoidal embeddings, injected into ResNet blocks like timestep embeddings, so that one forward pass reproduces the three-way combination of conditional and unconditional predictions that classifier-free guidance normally requires. Guidance-preserving adversarial step distillation borrows the LADD recipe, using a frozen teacher that generates clean latents, a student that denoises a noisy version in one step, and a discriminator with frozen feature extractor plus trainable spatial heads; because the student and discriminator both receive the guidance scales, the distilled model keeps the user's ability to control edit strength at inference. These mechanisms reduce the number of forward evaluations per diffusion step by a factor of three and the number of denoising steps by a factor of ten.

What would settle it

Measure the wall-clock time of the complete MoViE pipeline—input frame encoding, anchor encoding, cross-frame attention, single-step denoising, and decoding—on a 120-frame, 512×384 video running on a Snapdragon 8 Gen 3 phone, and compare it with the claimed 9.6 seconds. If the full run takes longer, the central on-device 12 fps claim fails for complete videos.

Watch

Extended reading notes

Core claim

The central discovery is a compound optimization recipe that makes on-device video-to-video diffusion practical. Starting from the InstructPix2Pix image editor with a middle-frame cross-frame attention anchor, the paper replaces the heavy U-Net with Mobile-Pix2Pix, which drops self- and cross-attention at the highest resolution and uses the TAESD autoencoder; it then trains a student denoiser to reproduce the combined output of the three-pass classifier-free guidance in a single pass by conditioning on the image guidance scale and text guidance scale, and finally replaces the ten-step denoising loop with one adversarial step, carried out by a discriminator whose spatial heads are conditioned on the same guidance signals. The paper reports that a full 120-frame 512×384 edit takes 9.6 seconds on a Xiaomi 14 Pro and that editing quality, measured by CLIP-based scores and a small human preference study, stays close to the original multi-step model. It argues that this is the first on-device video-to-video diffusion model, with per-frame cost of 0.76 TFLOPs versus tens to over a hundred TFLOPs for recent server-side methods.

Load-bearing premise

The headline speed figure rests on a single-frame denoising measurement, and the paper does not report an end-to-end video-edit timing that includes anchor encoding, cross-frame attention, and autoencoder overhead; if those costs are substantial, real throughput will fall short of 12 frames per second.

Editorial extensions

If this is right

  • A 120-frame, 512×384 video can be edited on a phone in 9.6 seconds instead of requiring a multi-GPU server.
  • Per-frame compute drops to 0.76 TFLOPs, more than an order of magnitude below recent video-editing baselines, so the model can run on low-power hardware.
  • Video length no longer forces a memory blow-up: frames are generated with cost linear in the number of frames, and the only anchor is the middle frame.
  • Text and image guidance scales remain controllable after distillation, so users can trade edit strength against fidelity at inference.
  • Local editing removes the need to upload video to the cloud, which changes the privacy and connectivity requirements of video-editing tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported 12 fps is a single-frame denoising latency, not a measured end-to-end video edit; anchor encoding, cross-frame attention traffic, and autoencoder time are not separately reported, so the true per-video throughput could be lower than the headline number.
  • The same three-stage recipe (architectural pruning, multimodal CFG distillation, guidance-conditioned adversarial distillation) likely transfers to other instruction-based editors and to text-to-image models, since none of the steps is specific to video.
  • The distilled model inherits the limitations of its teacher and of InstructPix2Pix: prompts or edits the base model cannot represent will not appear in the output.
  • A direct way to verify the headline is to time the whole 120-frame pipeline on the same phone; if wall-clock time exceeds 9.6 seconds, the fps claim is not valid for full videos.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents MoViE, a mobile-oriented video editing pipeline built from InstructPix2Pix. The authors apply four optimizations: (1) a mobile-friendly UNet with attention layers removed at the highest resolutions, (2) a lightweight TAESD autoencoder, (3) multimodal guidance distillation that folds the three classifier-free-guidance forward passes into one student pass conditioned on the image and text guidance scales, and (4) adversarial step distillation that reduces sampling to a single step while conditioning on the guidance scales to preserve controllability. The claims include on-device video editing at 12 fps on a Snapdragon 8 Gen 3 mobile platform, with quality comparable to state-of-the-art video editing models on the TGVE benchmark and DAVIS videos. The experiments are organized as sequential stages, with per-stage FLOPs and latency measurements, qualitative comparisons, and an appendix human evaluation.

Significance. If the headline throughput claim is substantiated, this would be a noteworthy practical milestone: the first demonstration of an on-device video-to-video diffusion editing model, with a reported 120-frame edit in 9.6 seconds. The staged optimization story is transparent and useful, and the two methodological components—multimodal guidance distillation and guidance-preserving adversarial distillation—are well motivated and could transfer to other editing pipelines. The paper also provides quantitative comparisons on a standard benchmark (TGVE) and qualitative results on DAVIS, with per-stage ablations. However, the central quantitative claim currently rests on a denoising-only latency measurement, and several quality claims are supported by small, underpowered evaluations. The overall engineering contribution is credible, but the headline needs to be either re-measured or reworded.

major comments (5)
  1. [§4.1, Table 1, §1] The headline number "12 fps" and the associated 9.6-second video-editing time are not supported by the reported measurement protocol. Section 4.1 states "Reported latencies correspond to a single-frame denoising," and Table 1's final row gives 0.08 s phone latency; 120 × 0.08 s = 9.6 s. This measurement excludes the TAESD encoder/decoder cost for every source and edited frame, which the paper itself counts as part of the end-to-end FLOPs (0.76 TFLOPs per frame in Table 1), and it also does not appear to include the cross-frame attention overhead with the middle-frame anchor described in Section 3.1. The abstract and introduction promote the 9.6 s / 12 fps figure as the central contribution. The authors should either report a measured end-to-end phone latency that includes autoencoding and anchor processing, or clearly label the headline as denoising-only throughput.
  2. [Appendix A.1, §4.2] The human evaluation is explicitly underpowered by the authors' own admission: "due to the limited sample size, we cannot draw definitive conclusions." Yet Section 4.2 builds on this study to claim "maintaining good quality edits" and competitiveness with SOTA methods. Since the quality-preservation claim is central to the trade-off narrative, either the human study must be expanded (with more participants or pairs, and appropriate statistical testing), or the claims in the main text must be correspondingly weakened.
  3. [Table 2, §4.3] The ablations for the adversarial distillation design choices report only CLIP-Image similarity on the InstructPix2Pix validation set, with no variance, confidence intervals, or significance tests. The observed differences are small (0.759 vs. 0.769 vs. 0.765 for noise distributions; 0.781 vs. 0.786 for guidance conditioning on discriminator heads) and could be within run-to-run noise. This weakens the support for the specific design choices (mean = −1, std = 1; guidance conditioning on heads). The authors should add standard deviations or multiple seeds, and ideally report directional CLIP or a second metric.
  4. [§3.4, Fig. 4] The paper's claim that adversarial distillation "preserves controllability" is supported only by the qualitative Figure 4, which shows varying outputs for a few guidance-scale combinations. There is no quantitative measurement of how edit strength responds to sI and sT in the single-step student, nor a comparison of the guidance-response curve before and after distillation. Since preserving controllability is one of the main stated advantages over LADD [51] and SF-V [67], this needs a more rigorous demonstration, such as a plot of a relevant similarity metric versus each guidance scale.
  5. [§5 vs. §1, §4.2] There is a numerical inconsistency in the reported frame rate: the abstract and introduction state 12 fps, Section 4.2 also says 12 frames per second, but the conclusion states 12.5 frames per second. This must be harmonized, and the exact computation underlying the chosen value should be stated explicitly.
minor comments (5)
  1. [Table 1] The row labeled "RA VE" should be "RAVE," and the table would be more readable if every row reported the same set of columns; currently Fairy and EVE have missing latency entries, and the footnote about source of metric values is easy to miss.
  2. [§3.1] The sentence "Frames are generated in 10 diffusion steps, with the middle frame used as the sole anchor" refers to the base model, but the final MoViE pipeline uses one step; the wording should identify which model variant is being described to avoid confusion.
  3. [Algorithm 2] In Step 4, the notation x0 = M(ϵ, C) implies a single call to the teacher, yet the appendix explains that the teacher is evaluated with 5 LCM sampling steps; the algorithm should either show the sampling loop or clarify that M represents the full multi-step sampling procedure.
  4. [Fig. 5] The CLIP metric curves in Figure 5 are plotted as single lines without variance estimates; adding error bars or multiple runs would make the "minimum quality drop" claim more convincing.
  5. [§4.1] The latency is measured on 512×384 frames while the FLOPs and GPU latencies are reported for 480×480 resolution; the authors should state explicitly why the resolution difference does not affect the comparison, or report all numbers at the same resolution.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the optimization chain is externally benchmarked, and the headline FPS caveat is a measurement-protocol gap, not a circular step.

full rationale

The paper's efficiency claims are produced by a staged pipeline that is evaluated against external benchmarks and baselines, not by definitional identity. Stage 1 removes expensive self-/cross-attention following SnapFusion, Stage 2 creates Mobile-Pix2Pix, Stage 3 distills the three-pass CFG of Eq. (1) into one student pass via the MSE loss to the teacher's output, and Stage 4 applies adversarial distillation built on LADD; each stage is measured in Table 1 with FLOPs, GPU latency, phone latency, PickScore, and CLIPFrame on the TGVE benchmark. The guidance-scale controllability is built into the student conditioning, but the paper validates it empirically in Fig. 4 and ablates discriminator design in Table 2, so it is not merely assumed. The self-citations [12, 21] are passing related-work mentions and are not load-bearing for any contribution. The one notable caveat is measurement protocol: Section 4.1 states 'Reported latencies correspond to a single-frame denoising,' and Section 1's headline 'editing a 120-frame video ... in 9.6 seconds' equals 120 × 0.08 s from Table 1, which therefore excludes VAE encode/decode and cross-frame anchor overhead. This is a correctness/verification concern, not circularity: no parameter is fitted and renamed a prediction, and no equation is assumed equal to itself by construction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central contribution is a compression and distillation stack, so most scientific content sits in chosen hyperparameters and the trusted teacher/autoencoder; the paper does not introduce new physical entities.

free parameters (3)
  • Guidance scale sampling ranges = s_I ~ U[1,3]; s_T ~ U[2,14]
    Chosen to cover typical InstructPix2Pix settings; the student model only learns control within this envelope, so the controllability claim is conditional on these ranges.
  • Discriminator noise distribution = logit-normal, mean=-1, std=1
    Ablated in Section 4.3; the choice changes CLIP-Image scores from 0.759 to 0.769, so it is a hand-tuned component of the adversarial recipe.
  • Teacher LCM sampling steps = 5
    The teacher is evaluated with 5 LCM steps because it gives 'good quality results'; this anchors the student's quality ceiling.
assumptions (3)
  • domain assumption The frozen teacher's multi-step outputs are a reliable and sufficient target for student distillation.
    The whole one-step student inherits the teacher's errors and biases; if the 5-step LCM teacher is misaligned, the student cannot recover.
  • domain assumption TAESD's latent space is compatible with the Stable Diffusion VAE well enough for the edits to be decoded correctly.
    The final pipeline uses TAESD for both encoding and decoding; the shared-latent claim is inherited from the TAESD project, not re-derived here.
  • domain assumption Static InstructPix2Pix image triplets are a sufficient training signal for temporally consistent video editing.
    No video data or temporal loss is used; temporal consistency is expected to come from the shared anchor mechanism, which is not evaluated in isolation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MoViE: Mobile Diffusion for Video Editing." pith.science (2026). https://pith.science/paper/WIUNA5U2

@misc{pith2026241206578,
  author       = {Pith},
  title        = {Pith review of: MoViE: Mobile Diffusion for Video Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WIUNA5U2}},
  note         = {Machine review of arXiv:2412.06578}
}
read the original abstract

Recent progress in diffusion-based video editing has shown remarkable potential for practical applications. However, these methods remain prohibitively expensive and challenging to deploy on mobile devices. In this study, we introduce a series of optimizations that render mobile video editing feasible. Building upon the existing image editing model, we first optimize its architecture and incorporate a lightweight autoencoder. Subsequently, we extend classifier-free guidance distillation to multiple modalities, resulting in a threefold on-device speedup. Finally, we reduce the number of sampling steps to one by introducing a novel adversarial distillation scheme which preserves the controllability of the editing process. Collectively, these optimizations enable video editing at 12 frames per second on mobile devices, while maintaining high quality. Our results are available at https://qualcomm-ai-research.github.io/mobile-video-editing/

Figures

Figures reproduced from arXiv: 2412.06578 by the authors.

Figure 1
Figure 1. MoViE is a fast video editing model, capable of generating [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Multimodal Guidance Distillation Overview: Standard [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Adversarial Distillation: We distill a multi-step teacher [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: MoViE at text guidance [4.0, 8.0, 12.0] and image guidance [1.25, 1.75]. Our adversarial training maintains guid￾ance scales, allowing us to control edit strength during inference. (Prompt: In Van Gogh Style) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: CLIP metrics for InstructPix2Pix, Mobile-Pix2pix, [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of MoViE on DAVIS. Our method can handle complex global edits as well as perform more nuanced attribute [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: CLIP metrics for different autoencoder configurations. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison of our method to the base model. The efficiency is greatly improved whereas quality is not compromised [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison of our method MoViE to other SOTA video editing algorithms. We evaluate on two challenging editing [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Human Evaluation results comparing MoViE to [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Adversarial Head Architecture. training, we use a learning rate of 10−5 . We train on a single A100 GPU and training takes around 14 hours. Stage-4. Image-editing UNet model is an ϵ − prediction model. However, we find that prior to adversarial training, finetuning th…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MobileWan: Closing the Quality Gap for Mobile Video Diffusion

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A 5B-parameter video diffusion transformer is made deployable on mobile hardware via recurrence distillation, learnable head pruning, step distillation, and decoder optimization, achieving 83.79 VBench at 20s latency.

Reference graph

Works this paper leans on

70 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [51]

    Fast high- resolution image synthesis with latent adversarial diffusion distillation

    Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high- resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403.12015, 2024

    arXiv 2024

  2. [67]

    Sf-v: Single forward video generation model

    Zhixing Zhang, Yanyu Li, Yushu Wu, Yanwu Xu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Dimitris Metaxas, et al. Sf-v: Single forward video generation model. arXiv preprint arXiv:2406.04324, 2024

    arXiv 2024

  3. [1]

    Tiny autoencoder for stable diffusion

    Ollin Boer Bohan. Tiny autoencoder for stable diffusion

    work page

  4. [2]

    In- Base ModelMoViE Input VideoInput Video In Monet StyleTurn him into a wooden statue Figure 8

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- Base ModelMoViE Input VideoInput Video In Monet StyleTurn him into a wooden statue Figure 8. Qualitative comparison of our method to the base model. The efficiency is greatly improved whereas quality is not compromised both for style transfer and attribute edits. Please refer to the Appendix for vid...

    arXiv 2022

  5. [3]

    Edgefusion: On-device text-to-image generation

    Thibault Castells, Hyoung-Kyu Song, Tairen Piao, Shinkook Choi, Bo-Kyeong Kim, Hanyoung Yim, Changgwun Lee, Jae Gon Kim, and Tae-Ho Kim. Edgefusion: On-device text-to-image generation. arXiv preprint arXiv:2404.11925, 2024

    arXiv 2024

  6. [4]

    Pix2video: Video editing using image diffusion

    Duygu Ceylan, Chun-Hao P Huang, and Niloy J Mitra. Pix2video: Video editing using image diffusion. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 23206–23217, 2023

    work page 2023

  7. [5]

    Control-a-video: Controllable text-to-video generation with diffusion models

    Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv preprint arXiv:2305.13840, 2023

    arXiv 2023

  8. [6]

    Speed is all you need: On-device acceleration of large diffu- sion models via gpu-aware optimizations

    Yu-Hui Chen, Raman Sarokin, Juhyun Lee, Jiuqiang Tang, Chuo-Ling Chang, Andrei Kulik, and Matthias Grundmann. Speed is all you need: On-device acceleration of large diffu- sion models via gpu-aware optimizations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4651–4655, 2023

    work page 2023

Show all 70 references
  1. [7]

    Consistent video- to-video transfer using synthetic dataset, 2023

    Jiaxin Cheng, Tianjun Xiao, and Tong He. Consistent video- to-video transfer using synthetic dataset, 2023

    2023

  2. [8]

    Squeezing large-scale diffusion models for mobile

    Jiwoong Choi, Minkyu Kim, Daehyun Ahn, Taesu Kim, Yul- hwa Kim, Dongwon Jo, Hyesung Jeon, Jae-Joon Kim, and Hyungjun Kim. Squeezing large-scale diffusion models for mobile. arXiv preprint arXiv:2307.01193, 2023

    2023 arXiv

  3. [9]

    Distilling the knowledge in diffusion models

    Tim Dockhorn, Robin Rombach, Andreas Blatmann, and Yaoliang Yu. Distilling the knowledge in diffusion models. In CVPR Workshop Generative Modelsfor Computer Vision, volume 2, 2023

    2023

  4. [10]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7346–7356, 2023

    2023

  5. [11]

    Tokenflow: Consistent diffusion features for consistent video editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373, 2023

    2023 arXiv

  6. [12]

    Clockwork diffusion: Efficient generation with model-step distillation

    Amirhossein Habibian, Amir Ghodrati, Noor Fathima, Guil- laume Sautiere, Risheek Garrepalli, Fatih Porikli, and Jens Petersen. Clockwork diffusion: Efficient generation with model-step distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  7. [13]

    Instruct-nerf2nerf: Edit- ing 3d scenes with instructions

    Ayaan Haque, Matthew Tancik, Alexei A Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerf: Edit- ing 3d scenes with instructions. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19740–19750, 2023

    2023

  8. [14]

    Ptqd: Accurate post-training quantization for diffusion models

    Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Pro- cessing Systems, 36, 2024

    2024

  9. [15]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022

    2022 arXiv

  10. [16]

    Clipscore: A reference-free evaluation met- ric for image captioning, 2022

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning, 2022

    2022

  11. [17]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022

    arXiv

  12. [18]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  13. [19]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    2022

  14. [20]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022

    2022 arXiv

  15. [21]

    Object- centric diffusion for efficient video editing

    Kumara Kahatapitiya, Adil Karjauv, Davide Abati, Fatih Porikli, Yuki M Asano, and Amirhossein Habibian. Object- centric diffusion for efficient video editing. In European Conference on Computer Vision , pages 91–108. Springer, 2025

    2025

  16. [22]

    Rave: Randomized noise shuf- fling for fast and consistent video editing with diffusion mod- els

    Ozgur Kara, Bariscan Kurtkaya, Hidir Yesiltepe, James M Rehg, and Pinar Yanardag. Rave: Randomized noise shuf- fling for fast and consistent video editing with diffusion mod- els. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 65...

    2024

  17. [23]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  18. [24]

    Elucidating the design space of diffusion-based generative models, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models, 2022

    2022

  19. [25]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Comput...

    2023

  20. [26]

    Bk-sdm: A lightweight, fast, and cheap ver- sion of stable diffusion

    Bo-Kyeong Kim, Hyoung-Kyu Song, Thibault Castells, and Shinkook Choi. Bk-sdm: A lightweight, fast, and cheap ver- sion of stable diffusion. arXiv preprint arXiv:2305.15798 , 2023

    2023 arXiv

  21. [27]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017

    2017

  22. [28]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation, 2023

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation, 2023

    2023

  23. [29]

    Videopoet: A large language model for zero-shot video gen- eration

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos ´e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vigh- nesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video gen- eration. arXiv preprint arXiv:2312.14125, 2023

    2023 arXiv

  24. [30]

    Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds. Advances in Neural Information Pro- cessing Systems, 36, 2024

    2024

  25. [31]

    Rectified flow: A marginal preserving approach to optimal transport, 2022

    Qiang Liu. Rectified flow: A marginal preserving approach to optimal transport, 2022

    2022

  26. [32]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023

    2023

  27. [33]

    Video-p2p: Video editing with cross-attention control

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8599–8608, 2024

    2024

  28. [34]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022

    2022 arXiv

  29. [35]

    Simplifying, stabilizing and scaling continuous-time consistency models

    Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410.11081, 2024

    2024 arXiv

  30. [36]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787, 2022

    2022

  31. [37]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongx- uan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022

    2022 arXiv

  32. [38]

    Latent consistency models: Synthesizing high- resolution images with few-step inference, 2023

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference, 2023

    2023

  33. [39]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 14297–14306, 2023

    2023

  34. [40]

    Which training methods for gans do actually converge?, 2018

    Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge?, 2018

    2018

  35. [41]

    Softmax bias correction for quantized gener- ative models

    Nilesh Prasad Pandey, Marios Fournarakis, Chirag Patel, and Markus Nagel. Softmax bias correction for quantized gener- ative models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1453–1458, 2023

    2023

  36. [42]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–11, 2023

    2023

  37. [43]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine- Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  38. [44]

    Vari- ational autoencoder

    Lucas Pinheiro Cinelli, Matheus Ara ´ujo Marins, Ed- uardo Ant´unio Barros da Silva, and S´ergio Lima Netto. Vari- ational autoencoder. In Variational Methods for Machine Learning with Applications to Deep Networks , pages 111–

  39. [45]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15932–15942, 2023

    2023

  40. [46]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  41. [47]

    Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters. Proceedings of the 26th ACM SIGKDD International Con- ference on Knowledge Discovery & Data Mining, 2020

    2020

  42. [48]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  43. [50]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. CoRR, abs/2202.00512, 2022

    2022 arXiv

  44. [52]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference on Computer Vision , pages 87–103. Springer, 2025

    2025

  45. [53]

    Post-training quantization on diffusion models

    Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1972–1981, 2023

    1972

  46. [54]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 , 2022

    2022 arXiv

  47. [55]

    Video edit- ing via factorized diffusion distillation, 2024

    Uriel Singer, Amit Zohar, Yuval Kirstain, Shelly Sheynin, Adam Polyak, Devi Parikh, and Yaniv Taigman. Video edit- ing via factorized diffusion distillation, 2024

    2024

  48. [56]

    Improved tech- niques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved tech- niques for training consistency models. arXiv preprint arXiv:2310.14189, 2023

    2023 arXiv

  49. [57]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023

    2023 arXiv

  50. [58]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1921–1930, 2023

    1921

  51. [59]

    Diffusion-gan: Training gans with diffusion

    Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion-gan: Training gans with diffusion. arXiv preprint arXiv:2206.02262, 2022

    2022 arXiv

  52. [60]

    Fairy: Fast parallelized instruction-guided video-to-video synthesis

    Bichen Wu, Ching-Yao Chuang, Xiaoyan Wang, Yichen Jia, Kapil Krishnakumar, Tong Xiao, Feng Liang, Licheng Yu, and Peter Vajda. Fairy: Fast parallelized instruction-guided video-to-video synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  53. [61]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  54. [62]

    Cvpr 2023 text guided video editing competition, 2023

    Jay Zhangjie Wu, Xiuyu Li, Difei Gao, Zhen Dong, Jin- bin Bai, Aishani Singh, Xiaoyu Xiang, Youzeng Li, Zuwei Huang, Yuanxi Sun, Rui He, Feng Hu, Junhua Hu, Hai Huang, Hanyu Zhu, Xu Cheng, Jie Tang, Mike Zheng Shou, Kurt Keutzer, and Forrest Iandola. Cvpr 2023 text guided vide...

    2023

  55. [63]

    Rerender a video: Zero-shot text-guided video-to-video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video-to-video translation. In SIGGRAPH Asia 2023 Conference Papers , pages 1–11, 2023

    2023

  56. [64]

    Fast sampling of dif- fusion models with exponential integrator

    Qinsheng Zhang and Yongxin Chen. Fast sampling of dif- fusion models with exponential integrator. arXiv preprint arXiv:2204.13902, 2022

    2022 arXiv

  57. [65]

    Fastvideoedit: Leveraging consistency models for efficient text-to-video editing

    Youyuan Zhang, Xuan Ju, and James J Clark. Fastvideoedit: Leveraging consistency models for efficient text-to-video editing. arXiv preprint arXiv:2403.06269, 2024

    2024 arXiv

  58. [66]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077, 2023

    2023 arXiv

  59. [68]

    Mobilediffusion: Subsecond text-to-image generation on mobile devices

    Yang Zhao, Yanwu Xu, Zhisheng Xiao, and Tingbo Hou. Mobilediffusion: Subsecond text-to-image generation on mobile devices. arXiv preprint arXiv:2311.16567, 2023

    2023 arXiv

  60. [69]

    Magicvideo: Efficient video generation with latent diffusion models

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022

    2022 arXiv

  61. [70]

    Slimflow: Training smaller one-step diffusion models with rectified flow

    Yuanzhi Zhu, Xingchao Liu, and Qiang Liu. Slimflow: Training smaller one-step diffusion models with rectified flow. In European Conference on Computer Vision , pages 342–359. Springer, 2025. MoViE: Mobile Diffusion for Video Editing Supplementary Material MoViEBaselineBoth Goo...

    2025

  62. [71]

    We ablate this property in Section 4.3

    and using mean = −1, std= 1 gave us the best performance. We ablate this property in Section 4.3

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.