Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FlowMo claims that temporal coherence in text-to-video diffusion models can be improved at inference time by reducing patch-wise temporal variance in appearance-debiased latent predictions, without retraining or external conditioning.

desk verdict Training-free variance guidance for motion coherence that mostly works, with solid human-preference evidence; the main open question is whether the variance signal targets incoherence or just motion magnitude. read the letter →

arxiv 2506.01144 v2 pith:44P6N2CA submitted 2025-06-01 cs.CV

Ariel Shaulov , Itay Hazan , Lior Wolf , Hila Chefer This is my paper
classification cs.CV
keywords text-to-videogenerationtraining-freeguidancetemporalcoherencemotionartifactslatent-spacevarianceflowmatchinginference-timeoptimizationdiffusiontransformers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlowMo asks whether a pre-trained text-to-video diffusion model already contains enough temporal information inside its own latent predictions to fix its motion errors. The paper answers yes: it defines an appearance-debiased temporal representation by taking per-patch distances between consecutive frames, measures patch-wise temporal variance, and uses the maximum-variance patch as a loss to refine the latent during sampling. The method requires no retraining, no external optical flow, and no architectural changes. On Wan2.1-1.3B and CogVideoX-5B, FlowMo improves motion smoothness and overall VBench final score by 6.20% and 5.26% respectively, while human raters prefer it for motion coherence, aesthetic quality, and text alignment. A sympathetic reading is that temporal coherence can be improved by looking inward at the model's own predictions rather than adding external motion priors.

What carries the argument

At the core is the appearance-debiased temporal variance signal. Given the model's predicted velocity $u_{\theta,t}$ for $F$ latent frames, the operator $\Delta$ computes the $\ell^1$-distance between consecutive frames channel-wise, removing shared appearance; then a patch-wise variance tensor $\sigma^2_{w,h,c}$ is computed across the $F-1$ difference frames and averaged over channels to form a per-patch coherence map $s_{w,h}$. The FlowMo loss is the maximum over patches, $L=\max_{w,h} s_{w,h}$, which targets the most dynamically incoherent region rather than averaging over mostly static patches. Gradient descent on this loss updates the input latent $z_{t_i}$, after which the diffusion step is recomputed; it is applied only at early-to-mid timesteps where qualitative visualization shows coarse motion emerging. This mechanism carries the argument by converting the correlation in Section 3.2 into an actionable, training-free guidance step.

What would settle it

Compute the same variance statistic on a larger set of generated videos spanning all coherence ratings and matched for motion magnitude; if high variance also appears in coherent videos or fails to separate moderate incoherence from coherence, the guidance signal would be targeting motion magnitude rather than incoherence. A second check would apply FlowMo to a video whose incoherence is confined to patches with low variance; if artifacts persist, the max-patch variance criterion is not sufficient.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that coherent motion corresponds to low temporal variance in an appearance-debiased latent representation, and that this variance can be used as a guidance signal. The paper observes that predictions of flow-matching video models are appearance-biased, so it computes the $\ell^1$-distance between latent predictions of consecutive frames to cancel shared appearance, then computes variance across frames for each spatial patch and channel. Incoherent videos show consistently higher patch-wise variance than coherent ones, with separation emerging around the fifth denoising step, and coarse motion is established in intermediate steps rather than at the start. FlowMo selects the patch with maximal temporal variance, back-propagates through the prediction to adjust the input latent, recomputes the model prediction, and repeats this refinement at the first twelve timesteps. The paper claims this makes latent transitions smoother and that this maps to smoother pixel-space behavior, significantly boosting motion coherence while preserving or improving visual quality and prompt alignment.

Load-bearing premise

The load-bearing premise is that the size of frame-to-frame changes in the model's internal prediction, after stripping out shared appearance, marks where motion looks incoherent, and that this was verified only on a small user study comparing extreme coherence ratings.

Editorial extensions

If this is right

  • Any pre-trained flow-matching text-to-video model with accessible latent predictions can be guided the same way; the method is architecture-agnostic and needs no fine-tuning.
  • Generated videos gain motion smoothness and fewer artifacts such as extra limbs and objects that appear or disappear, while dynamic degree drops only slightly because spurious motion is removed.
  • The method offers a plug-and-play alternative to retraining or external motion signals for improving temporal fidelity.
  • Applied during the first twelve timesteps, FlowMo also reduces and stabilizes maximal patch-wise variance in later, non-optimized steps, suggesting the coarse motion structure set early controls later coherence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the per-patch variance map that FlowMo uses as a loss could also serve as a cheap automatic localizer of temporal artifacts in generated videos, since it already highlights the regions the optimization targets.
  • Editorial inference: because FlowMo consumes no external inputs, it could plausibly be stacked on top of trajectory-based or optical-flow conditioning methods to clean residual incoherence they leave behind, though the paper does not test this combination.
  • Editorial inference: the same variance objective could be folded into training to give video models a richer temporal prior; the paper itself notes that inference-time optimization is bounded by what the pretrained model already knows how to represent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces FlowMo, a training-free, inference-time guidance method for text-to-video diffusion models. At selected early denoising steps, FlowMo computes an appearance-debiased temporal representation from the model's predicted velocity (the L1 distance between consecutive latent frames, Eq. 2), forms a patch-wise temporal variance map (Eq. 3), averages over channels, and uses the maximum value as a loss to take a gradient step on the input latent. Experiments on Wan2.1-1.3B and CogVideoX-5B report VBench Final Score gains of +6.20% and +5.26%, user-study preferences for motion coherence, aesthetic quality, and text alignment, and ablations of the max-variance objective, the debiasing operator, and the timestep selection.

Significance. If the central mechanism holds, FlowMo is a valuable contribution: it is architecture-agnostic, requires no retraining or external conditioning, and is supported by a relatively large human preference study (640 responses per baseline), per-dimension VBench results, and a comparison to FreeInit. The conceptual claim that patch-wise temporal variance in an appearance-debiased latent prediction tracks perceived incoherence is interesting and actionable. However, the motivating evidence is weakened by a missing control for motion magnitude, and the claim that FlowMo improves quality 'without sacrificing' visual quality or prompt alignment is not fully supported by the reported per-dimension VBench degradations.

major comments (3)
  1. [Sec. 3.2, Eqs. (2)-(3), Fig. 2] The correlation study that motivates the variance objective does not control for motion magnitude. The text states that videos were filtered to have a motion score of at least 3, but it does not match or regress out the motion-magnitude rating between the completely incoherent (1) and completely coherent (5) groups. Since Eq. (3) is a temporal variance, it will increase with the amount of motion regardless of coherence, so the gap in Fig. 2 could reflect motion quantity rather than incoherence. This is load-bearing because the whole method minimizes this variance. Please provide the variance conditioned on the motion score, or a matched-pair comparison with equal motion scores, and report the coherence-vs-variance relationship after controlling for motion. In addition, the aggregate VBench Dynamic Degree drop of less than 1.5% does not rule out larger motion suppression on high-motion prompts; report the per-prompt or distributional effect on Dynamic Degree or on human motion-magnitude ratings.
  2. [Sec. 4.2, Table 3, App. E] The abstract claims FlowMo improves motion coherence 'without sacrificing visual quality or prompt alignment,' but Table 3 shows several per-dimension degradations: CogVideoX Temporal Flickering drops from 99.23% to 96.21%, Human Action drops on both models (Wan2.1: 98.27% to 97.23%; CogVideoX: 97.81% to 95.69%), and Wan2.1 Background Consistency and Color also decline. Since Temporal Flickering and Human Action are directly relevant to temporal coherence and overall quality, the claim of no sacrifice is too strong as stated. Either temper the claim to acknowledge these dimensions, or provide an analysis showing that these drops are not statistically significant or are outweighed in a principled way.
  3. [Sec. 4.3, Fig. 6] The ablation study is presented only as qualitative still frames. The central design choices—max versus mean variance, presence of the debiasing operator, and the set of optimized timesteps—are load-bearing for FlowMo's effectiveness, so they should be evaluated with quantitative metrics (e.g., VBench Motion Smoothness, Final Score, or a user study) in addition to the qualitative examples. Without quantitative ablation results, it is difficult to judge whether the differences shown in Fig. 6 are robust or cherry-picked.
minor comments (6)
  1. [Sec. 3.2] The word 'denoinsing' appears in the first paragraph of Sec. 3.2 and should be corrected to 'denoising'.
  2. [Sec. 3.3] In the paragraph following Eq. (8), the text says 'the loss in Eq. 7' when referring to the FlowMo loss; this should be Eq. (8).
  3. [Algorithm 1, line 8] The notation 'max_{w∼[W],h∼[H]}' reads as sampling from a distribution; use 'max_{w∈[W],h∈[H]}' for clarity.
  4. [Sec. 4, Implementation details] The phrase 'first 12 timesteps of the generation' is not reproducible without knowing the exact scheduler and timestep indices. Please specify the scheduler and the concrete list of timesteps used.
  5. [Sec. 4.2, User study] The text reports '640 unique responses per baseline' but does not state the number of prompts or participants. Adding these numbers would help readers assess the study design.
  6. [App. E] There is a typo in the final sentence: 'produced viseos' should be 'produced videos'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the variance-based guidance is motivated by an independent annotation study and validated on separate human-preference and VBench metrics; only minor non-load-bearing self-citations appear.

full rationale

The paper's derivation chain is self-contained. The proxy for incoherence — patch-wise temporal variance of the appearance-debiased consecutive-frame differences (Eqs. 2–3) — is not defined in terms of the final evaluation metrics. It is motivated by a user study comparing extreme coherence labels in Sec. 3.2, then used to define the FlowMo loss (Eq. 8). The main claims are then validated by separate human preference judgements on the VideoJAM benchmark and by VBench automatic metrics, not by the same data used to motivate the loss. No fitted parameter is renamed as a prediction; the learning rate and timestep range are hand-selected, and the ablation study shows that the maximal-variance and debiasing choices are not immaterial. The only self-referential element is Appendix C, where the paper shows that FlowMo reduces the maximal patch-wise variance it explicitly optimizes; this is a sanity check on the optimizer rather than independent evidence for the coherence claim, and the paper does not rest its central conclusion on it. Citations to the authors' prior work (VideoJAM for the benchmark and appearance-bias observation, Attend-and-Excite for the gradient-guidance design) are present, but the central comparisons are independently benchmarked on VBench and by human raters, so these citations are not load-bearing. There is no uniqueness theorem, no ansatz smuggled in by citation, and no renaming of a known result. The motion-magnitude confound discussed by a skeptic is a correctness or robustness concern, not a circularity, because the claimed coherence gains are measured by external judgments rather than by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method relies on the standard flow-matching and CFG machinery from prior work, plus two domain assumptions about the latent space: that frame differences remove appearance, and that temporal variance correlates with incoherence. The only hand-chosen numbers are the learning rate and the set of guidance timesteps.

free parameters (2)
  • learning rate eta = 0.005
    Hand-chosen for the gradient descent on the input latent (Eq. 9). Sensitivity is not reported.
  • guidance timesteps = first 12 of the denoising schedule
    Selected based on the observation that coarse motion emerges in steps 4-8, but the exact cutoff (12) is not ablated over a range.
assumptions (4)
  • standard math Flow matching formulation: the model predicts velocity v_t = z_0 - z_1, and denoising uses the update z_{t+1} = (1 - σ_t) z_t - σ_t u_θ(z_t, t).
    Used in Sections 3.1 and 3.3; standard in modern video diffusion models.
  • domain assumption Frame-wise ℓ1 distance removes shared appearance information and exposes motion.
    Used in Eq. (2); motivated by the claim that model predictions are appearance-biased, but not proven.
  • domain assumption Temporal coherence corresponds to low patch-wise variance of the appearance-debiased latent differences.
    Central hypothesis validated in Sec. 3.2 on a small, extreme-selected dataset.
  • domain assumption Gradient descent on the input latent to reduce the variance loss improves the final video's coherence without harming content.
    The core mechanism in Algorithm 1 lines 9-12; empirically tested but not theoretically justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation." pith.science (2026). https://pith.science/paper/44P6N2CA

@misc{pith2026250601144,
  author       = {Pith},
  title        = {Pith review of: FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/44P6N2CA}},
  note         = {Machine review of arXiv:2506.01144}
}
read the original abstract

Text-to-video diffusion models are notoriously limited in their ability to model temporal aspects such as motion, physics, and dynamic interactions. Existing approaches address this limitation by retraining the model or introducing external conditioning signals to enforce temporal consistency. In this work, we explore whether a meaningful temporal representation can be extracted directly from the predictions of a pre-trained model without any additional training or auxiliary inputs. We introduce FlowMo, a novel training-free guidance method that enhances motion coherence using only the model's own predictions in each diffusion step. FlowMo first derives an appearance-debiased temporal representation by measuring the distance between latents corresponding to consecutive frames. This highlights the implicit temporal structure predicted by the model. It then estimates motion coherence by measuring the patch-wise variance across the temporal dimension and guides the model to reduce this variance dynamically during sampling. Extensive experiments across multiple text-to-video models demonstrate that FlowMo significantly improves motion coherence without sacrificing visual quality or prompt alignment, offering an effective plug-and-play solution for enhancing the temporal fidelity of pre-trained video diffusion models.

Figures

Figures reproduced from arXiv: 2506.01144 by the authors.

Figure 1
Figure 1. Text-to-video results before and after applying FlowMo on (a) Wan2.1 [1] and CogVideoX-5B [2]. We present FlowMo, an inference-time guidance method to enhance temporal coherence in text-to-video models. Our method mitigates severe temporal artifacts, such as additional limbs (woman, 1st row, 2nd row), objects that appear or disappear (flamingo, 2nd row), and object distortions (woman, dolphin, 1st row), without requ… view at source ↗
Figure 2
Figure 2. Quantitative motivation. We mea￾sure the mean temporal variance of spatial patches for coherent and incoherent videos. Incoherent videos portray higher variance. The separation is visible from step 5 onward. 95%-confidence interval was computed using the seaborn python package. Quantitative motivation. Our central hypothesis is that temporally coherent motion corresponds to a form of local stability in uθ,t. Specifi… view at source ↗
Figure 3
Figure 3. Qualitative motivation. We visual￾ize the model prediction per timestep across the generation. Coarse spatial information is determined in the first steps (0-4), whereas motion is determined at steps 5-8, and refined in later steps. Qualitative motivation. To qualitatively explore the process of motion generation in text-to-video models, we visualize the evolution of the model’s latent space prediction across the ge… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Qualitative results. Text-to-video results before and after applying FlowMo on (a) Wan2.1 [1] and (b) CogVideoX [2]. FlowMo mitigates severe temporal artifacts, e.g., extra limbs (women, 2nd, 3rd row), objects that appear or disappear (2nd, 3rd row), and distortions (4…
Figure 5
Figure 5. Figure 5: User study conducted on Wan2.1-1.3B [1] (left) and CogVideoX-5B [2] (right) using VideoJAM-bench [3], designed specifically to evaluate motion coherence. Our method significantly improves temporal coherence in all models, while maintaining or improving the visual quali…
Figure 6
Figure 6. Figure 6: Ablation study. We ablate the main design choices of FlowMo, i.e., using the maximal variance for the objective (3rd row), using the appearance-debiasing operator (4th row), the selection of the optimization steps (5th row), and show that FlowMo is significantly superi…
Figure 7
Figure 7. Figure 7: Qualitative results. Text-to-video results of FreeInit [59] (1st, 3rd row) and FlowMo (2nd, 4th row) when applied on Wan2.1-1.3B [1]. FlowMo better mitigates severe temporal artifacts, e.g. distortions and object that appear and disappear. Qualitative comparisons betwe…
Figure 8
Figure 8. Figure 8: User studies conducted on Wan2.1-1.3B [1] using VideoJAM-bench [3]. The studies compare three variants of the model: vanilla (Wan2.1-1.3B), with FlowMo, and with FreeInit. Our method significantly outperforms the baselines in both studies. 95%-confidence intervals were…
Figure 9
Figure 9. Figure 9: A visualization of channel 0 (selected arbitrarily, and used in the main paper) of the latent [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: A visualization of channel 7 (selected randomly) of the latent prediction at different [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: FlowMo effect on patch-wise variance. We plot the maximal temporal vari￾ance of spatial patches for videos with and without applying FlowMo guidance, and ob￾serve that our method significantly reduces and stabilizes the variance in the generation steps that impact mot…
Figure 12
Figure 12. Figure 12: Screenshot of the Google Form used in the user study. [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Twins: Learn to Predict Unified Representations with Focal Loss

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Channel-wise concatenation of SigLIP2 and Flux VAE features into one token, trained with a focal-style flow-matching loss, yields a unified representation with 1.59 gFID on ImageNet 256 and VAE-level reconstruction.

Reference graph

Works this paper leans on

68 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Wan: Open and advanced large-scale video generative models

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wa...

    arXiv 2025

  2. [2]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022

    arXiv 2022

  3. [3]

    Videojam: Joint appearance-motion representations for enhanced motion generation in video models

    Hila Chefer, Uriel Singer, Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. Videojam: Joint appearance-motion representations for enhanced motion generation in video models. arXiv preprint arXiv:2502.02492, 2025

    arXiv 2025

  4. [4]

    How far is video generation from world model: A physical law perspective, 2024

    Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: A physical law perspective, 2024

    work page 2024

  5. [5]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024

    2024

  6. [6]

    Motion prompting: Controlling video generation with motion trajectories, 2024

    Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez- Guevara, Carl Doersch, Yusuf Aytar, Michael Rubinstein, Chen Sun, Oliver Wang, Andrew Owens, and Deqing Sun. Motion prompting: Controlling video generation with motion trajectories, 2024

    work page 2024

  7. [7]

    Wan-Duo Kurt Ma, J. P. Lewis, and W. Bastiaan Kleijn. Trailblazer: Trajectory control for diffusion-based video generation, 2023

    work page 2023

  8. [8]

    Physgen: Rigid-body physics- grounded image-to-video generation

    Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics- grounded image-to-video generation. In European Conference on Computer Vision ECCV, 2024

    work page 2024

Show all 68 references
  1. [9]

    Flatten: optical flow-guided attention for consistent text-to-video editing

    Yuren Cong, Mengmeng Xu, Christian Simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez-Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. Flatten: optical flow-guided attention for consistent text-to-video editing. arXiv preprint arXiv:2310.05922, 2023

    arXiv

  2. [10]

    Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz

    S. Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1526–1535, 2017

    2018

  3. [11]

    Video-lavit: Unified video-language pre-training with decoupled visual-motional tokenization

    Yang Jin, Zhicheng Sun, Kun Xu, Kun Xu, Liwei Chen, Hao Jiang, Quzhe Huang, Chengru Song, Yuliang Liu, Di Zhang, Yang Song, Kun Gai, and Yadong Mu. Video-lavit: Unified video-language pre-training with decoupled visual-motional tokenization. ArXiv, abs/2402.03161, 2024

    2024

  4. [12]

    Packing input frame context in next-frame prediction models for video generation

    Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint arXiv:2504.12626, 2025

    2025

  5. [13]

    Unsupervised representation learning with deep convolutional generative adversarial networks

    Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434, 2015

    2015 arXiv

  6. [14]

    Interpreting the latent space of gans for semantic face editing

    Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Interpreting the latent space of gans for semantic face editing. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9240–9249, 2019

    2020

  7. [15]

    Bermano, Gal Chechik, and Daniel Cohen-Or

    Rinon Gal, Or Patashnik, Haggai Maron, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. Stylegan- nada. ACM Transactions on Graphics (TOG), 41:1 – 13, 2021

    2021

  8. [16]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. I...

    2024

  9. [17]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020

    2020

  10. [18]

    Kingma, Ben Poole, Mohammad Norouzi, David J

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022. 11

    arXiv

  11. [19]

    Diffusion models beat GANs on image synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. arXiv preprint arXiv:2105.05233, 2021

    2021 arXiv

  12. [20]

    Zero-shot text-to-image generation, 2021

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021

    2021

  13. [21]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion mod...

    2022

  14. [22]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    2022 arXiv

  15. [23]

    Still-moving: Customized video generation without customized video data

    Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, Omer Tov, Michael Rubinstein, Lior Wolf, Tali Dekel, Tomer Michaeli, and Inbar Mosseri. Still-moving: Customized video generation without customized video data. arXiv preprint arXiv:2407.08674, 2024

    2024 arXiv

  16. [24]

    Emu edit: Precise image editing via recognition and generation tasks

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In CVPR, 2024

    2024

  17. [25]

    Video editing via factorized diffusion distillation

    Uriel Singer, Amit Zohar, Yuval, Shelly Sheynin, Adam Polyak, Devi Parikh, and Yaniv Taigman. Video editing via factorized diffusion distillation. In ECCV, 2024

    2024

  18. [26]

    Prompt-to- prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022

    2022 arXiv

  19. [27]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 18392–18402, 2023

    2023

  20. [28]

    Add-it: Training-free object insertion in images with pretrained diffusion models

    Yoad Tewel, Rinon Gal, Dvir Samuel, Yuval Atzmon, Lior Wolf, and Gal Chechik. Add-it: Training-free object insertion in images with pretrained diffusion models. arXiv preprint arXiv:2411.07232, 2024

    2024 arXiv

  21. [29]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6007–6017, 2023

    2023

  22. [30]

    Geodiffuser: Geometry-based image editing with diffusion models

    Rahul Sajnani, Jeroen Vanbaar, Jie Min, Kapil Katyal, and Srinath Sridhar. Geodiffuser: Geometry-based image editing with diffusion models. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 472–482. IEEE, 2025

    2025

  23. [31]

    Paint by example: Exemplar-based image editing with diffusion models

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18381–18391, 2023

    2023

  24. [32]

    Cascaded diffusion models for high fidelity image generation

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. arXiv preprint arXiv:2106.15282, 2021

    2021 arXiv

  25. [33]

    Lumiere: A space-time diffusion model for video generation

    Omer BarTal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, and Inbar Mosseri. Lumiere: A space-time diffusion model for video generation. arXiv preprint arXiv:2401.1...

    2024 arXiv

  26. [34]

    Latent-Shift: Latent diffusion with temporal shift for efficient text-to-video generation.arXiv preprint arXiv:2304.08477, 2023

    Jie An, Songyang Zhang, Harry Yang, Sonal Gupta, Jia-Bin Huang, Jiebo Luo, and Xi Yin. Latent-Shift: Latent diffusion with temporal shift for efficient text-to-video generation.arXiv preprint arXiv:2304.08477, 2023

    2023 arXiv

  27. [35]

    Cosmos world foundation model platform for physical ai

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025

    2025 arXiv

  28. [36]

    Genie: Generative interactive environments

    Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024

    2024

  29. [37]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 12

    2024 arXiv

  30. [38]

    Hedra: Ai-powered character video generation

    Hedra. Hedra: Ai-powered character video generation. https://www.hedra.com/, 2025. Accessed: May 13, 2025

    2025

  31. [39]

    Sora: Text-to-Video Generation

    OpenAI. Sora: Text-to-Video Generation. https://openai.com/sora/, 2024. Accessed: May 13, 2025

    2024

  32. [40]

    Genie 2: A Large-Scale Foundation World Model

    Google DeepMind. Genie 2: A Large-Scale Foundation World Model. https://deepmind.google/ discover/blog/genie-2-a-large-scale-foundation-world-model/ , 2024. Accessed: May 13, 2025

    2024

  33. [41]

    Generating worlds

    World Labs. Generating worlds. https://www.worldlabs.ai/blog, 2024. Accessed: 2025-05-13

    2024

  34. [42]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023

    2023

  35. [43]

    SDXL: improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  36. [44]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  37. [45]

    LaVie: High-quality video generation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. LaVie: High-quality video generation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023

    2023 arXiv

  38. [46]

    Unipc: A unified predictor-corrector framework for fast sampling of diffusion models

    Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36:49842–49869, 2023

    2023

  39. [47]

    Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models. ACM transactions on Graphics (TOG) , 42(4):1–10, 2023

    2023

  40. [48]

    Be yourself: Bounded attention for multi-subject text-to-image generation

    Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or. Be yourself: Bounded attention for multi-subject text-to-image generation. European Conference on Computer Vision (ECCV), pages 432–448, 2024

    2024

  41. [49]

    Make it count: Text-to-image generation with an accurate number of objects

    Lital Binyamin, Yoad Tewel, Hilit Segev, Eran Hirsch, Royi Rassin, and Gal Chechik. Make it count: Text-to-image generation with an accurate number of objects. arXiv preprint arXiv:2406.10210, 2024

    2024 arXiv

  42. [50]

    Separate-and-enhance: Compositional finetuning for text-to-image diffusion models

    Zhipeng Bao, Yijun Li, Krishna Kumar Singh, Yu-Xiong Wang, and Martial Hebert. Separate-and-enhance: Compositional finetuning for text-to-image diffusion models. In ACM SIGGRAPH 2024 Conference Papers, pages 1–10, 2024

    2024

  43. [51]

    Training-free guidance in text-to-video generation via multimodal planning and structured noise initialization

    Jialu Li, Shoubin Yu, Han Lin, Jaemin Cho, Jaehong Yoon, and Mohit Bansal. Training-free guidance in text-to-video generation via multimodal planning and structured noise initialization. arXiv preprint arXiv:2504.08641, 2025

    2025 arXiv

  44. [52]

    3dv-ton: Textured 3d-guided consistent video try-on via diffusion models

    Min Wei, Chaohui Yu, Jingkai Zhou, and Fan Wang. 3dv-ton: Textured 3d-guided consistent video try-on via diffusion models. arXiv preprint arXiv:2504.17414, 2025

    2025 arXiv

  45. [53]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7310–7320, 2024

    2024

  46. [54]

    Boosting text-to-video generative model with mllms feedback

    Xun Wu, Shaohan Huang, Guolong Wang, Jing Xiong, and Furu Wei. Boosting text-to-video generative model with mllms feedback. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  47. [55]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221, 2022

    2022 arXiv

  48. [56]

    Phenaki: Variable length video generation from open domain textual description

    Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Moham- mad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. arXiv preprint arXiv:2210.02399, 2022

    arXiv

  49. [57]

    Gen-l-video: Multi-text to long video generation via temporal co-denoising

    Fu-Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264, 2023. 13

    2023 arXiv

  50. [58]

    FreeNoise: Tuning-free longer video diffusion via noise rescheduling

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. FreeNoise: Tuning-free longer video diffusion via noise rescheduling. arXiv preprint arXiv:2310.15169, 2023

    2023 arXiv

  51. [59]

    Freeinit: Bridging initialization gap in video diffusion models

    Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initialization gap in video diffusion models. arXiv preprint arXiv:2312.07537, 2023

    2023 arXiv

  52. [60]

    Videoguide: Improving video diffusion models without training through a teacher’s guide

    Dohun Lee, Bryan S Kim, Geon Yeong Park, and Jong Chul Ye. Videoguide: Improving video diffusion models without training through a teacher’s guide. arXiv preprint arXiv:2410.04364, 2024

    2024 arXiv

  53. [61]

    AnimateD- iff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. AnimateD- iff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023

    2023 arXiv

  54. [62]

    FLUX, 2024

    Black Forest Labs. FLUX, 2024

    2024

  55. [63]

    Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Lu...

    2024

  56. [64]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  57. [65]

    Task me anything

    Jieyu Zhang, Weikai Huang, Zixian Ma, Oscar Michel, Dong He, Tanmay Gupta, Wei-Chiu Ma, Ali Farhadi, Aniruddha Kembhavi, and Ranjay Krishna. Task me anything. arXiv preprint arXiv:2406.11775, 2024

    2024 arXiv

  58. [66]

    Consisti2v: Enhancing visual consistency for image-to-video generation

    Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, and Wenhu Chen. Consisti2v: Enhancing visual consistency for image-to-video generation. arXiv preprint arXiv:2402.04324, 2024

    2024 arXiv

  59. [67]

    Id- animator: Zero-shot identity-preserving human video generation

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id- animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275 , 2024

    2024 arXiv

  60. [68]

    Young Adult Male Doing Handstand on the beach

    Jianwen Jiang, Chao Liang, Jiaqi Yang, Gaojie Lin, Tianyun Zhong, and Yanbo Zheng. Loopy: Taming audio-driven portrait avatar with long-term motion dependency. arXiv preprint arXiv:2409.02634, 2024. 14 A Comparison between FlowMo and FreeInit In this section, we compare FlowMo...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.