Pith. sign in

REVIEW 5 major objections 5 minor 126 references

Consistent and Controllable Image Animation with Motion Linear Diffusion Transformers

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MiraMo claims image animation can be made faster and more consistent by replacing quadratic attention with a RoPE-compatible cosine linear attention, learning motion residuals rather than frames, and initializing inference noise with low-fr

desk verdict Solid engineering paper with a useful linear-attention recipe, but the central quality-parity claim lacks a controlled ablation and the synthetic-data section has a clear internal contradiction. read the letter →

arxiv 2508.07246 v1 pith:QDGZXO7X submitted 2025-08-10 cs.CV

Xin Ma , Yaohui Wang , Genyun Jia , Xinyuan Chen , Tien-Tsin Wong , Cunjian Chen This is my paper
classification cs.CV
keywords imageanimationimage-to-videogenerationlinearattentiondiffusiontransformerflowmatchingmotionresiduallearningDCTnoiserefinementdynamicscontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MiraMo is an image-animation model built on a linear-attention diffusion transformer. The paper claims that three design choices let it beat U-Net-based animators on appearance consistency and motion smoothness while running faster: a cosine-kernel temporal attention that keeps O(N) cost, training on latent motion residuals rather than frames themselves, and a DCT-based noise initialization at inference. The authors also report that post-training on synthetic videos works better than on a small curated real dataset, producing higher VBench scores. If these claims hold, transformer-based image animation can be both cheaper and steadier without sacrificing prompt alignment or dynamics.

What carries the argument

The core mechanism is a RoPE-compatible cosine linear attention (Eq. 8–9), where the similarity between query $q_i$ and key $k_j$ is $1 + (q_i/\|q_i\|)(k_j/\|k_j\|)^T$, allowing shared sums over keys/values to be computed once per sequence and reducing attention cost from $O(N^2)$ to $O(N)$. This attention is inserted as a temporal-attention layer in each SANA transformer block to capture frame-to-frame relationships. Two supporting mechanisms carry the consistency claim: motion residual learning, which subtracts the first frame's latent from all subsequent latents so the model predicts only inter-frame differences, and DCTInit, which replaces the low-frequency part of the inference noise wi

What would settle it

Replace MiraMo's temporal linear attention with vanilla quadratic attention under identical training and inference; if the full-attention variant scores strictly better on temporal consistency and motion smoothness at comparable compute on the VBench I2V suite, the paper's core efficiency-quality tradeoff claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that image animation does not need to trade quality for speed. By adapting the SANA linear-attention text-to-image architecture into a text-to-video model with a new temporal linear attention (Eqs. 8–9) that stays compatible with RoPE, then fine-tuning it to predict latent motion residuals (differences from the first frame) rather than full frames, and finally refining the inference-time noise with low-frequency DCT components of the input image, MiraMo produces videos that hold the input image's appearance across frames, move smoothly, and follow the prompt, all at O(N) attention cost with a 0.74B-parameter model. In the reported comparisons it achieves the best or near

Load-bearing premise

The whole efficiency-without-quality-loss claim rests on the premise that the cosine-kernel temporal linear attention captures video temporal relationships as well as full quadratic attention, a comparison the paper asserts but does not quantitatively demonstrate.

Editorial extensions

If this is right

  • Image animation can be built on linear-attention transformers, closing the architectural gap with text-to-video models at a fraction of the compute.
  • Motion residual learning becomes a viable alternative to direct frame prediction for appearance preservation in I2V generation.
  • DCT-based noise initialization can replace FFT-based FreeInit-style injection to avoid color drift and abrupt motion artifacts.
  • Synthetic video data can substitute for carefully curated real data in post-training, lowering the cost of high-quality fine-tuning.
  • The residual representation supports motion transfer and video editing via flow inversion, as the paper's applications demonstrate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The same cosine-kernel attention could be dropped into the spatial attention of existing video diffusion transformers, since RoPE compatibility already exists there, potentially cutting memory for high-resolution and long-video generation.
  • Editorial inference: The paper's residual representation suggests a general recipe for training-free motion transfer: invert motion residuals from any source video and apply them to a modified first frame, a protocol that could extend to other animators beyond MiraMo.
  • Editorial inference: DCTInit's rationale predicts that other energy-compacting transforms, such as wavelets, would show similar or better suppression of abrupt motion; a simple ablation could test whether DCT is optimal or merely sufficient.
  • Editorial inference: If synthetic post-training data routinely beats small curated real data, it suggests that SFT gains depend more on distribution coverage and prompt diversity than on pixel-level realism, a hypothesis the paper does not directly test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. MiraMo is an image-to-video (I2V) framework built by extending the SANA linear-attention text-to-image model into a text-to-video model with a new cosine-kernel temporal linear attention compatible with RoPE, then adapting it to I2V via motion-residual learning (predicting z_i - z_1), DCT-based low-frequency noise initialization during inference, and a dynamics-degree control based on MS-SSIM buckets. The T2V backbone is trained on roughly 2M Vimeo25M videos and post-trained on about 150k synthetic videos; I2V is evaluated on UCF-101/MSR-VTT and VBench against U-Net and DiT baselines, with throughput/memory comparisons against vanilla attention. The paper claims high appearance consistency, smooth controllable motion, and O(N) attention complexity with no quality loss.

Significance. If substantiated, this would be a useful demonstration that linear attention can support video diffusion, with orthogonal contributions (motion-residual learning and DCTInit) that could transfer to other backbones. The paper's strengths include a focused ablation in Table V supporting MRL over direct frame prediction and DCTInit over FFT initialization, a quantitative dynamics-control analysis in Fig. 10, and a user study component. However, the load-bearing claim of linear-attention parity with vanilla attention is not directly tested, and some reported results conflict with the text (Table VI). The authors also state a self-admitted limitation (image-level VAE may cause flickering) that tempers the consistency claim.

major comments (5)
  1. [Sec. III-B, Fig. 3; Table II] The claim that linear attention 'preserves generation quality' is not established by a controlled experiment. Fig. 3 shows throughput/memory and a '+4.76% VBench' number but does not specify the vanilla-attention baseline's architecture, training data, or compute, so it cannot isolate the attention mechanism. Table I compares external T2V models with different data and budgets. Moreover, the statement that frame-dimension temporal attention 'outperforms' the full 3D variant is given without quantitative comparison. Without a same-cost linear-vs-vanilla ablation, the central O(N)-without-quality-loss conclusion is unsupported. Table II further shows Ours FVD 201.90 vs Cinemo 168.16 on UCF-101, so external comparisons do not supply the missing parity evidence.
  2. [Sec. IV-C, Table VI] The synthetic-data claim is contradicted by the table. The text states that the model fine-tuned on synthetic data achieves 'over a 3% improvement' over real data, but Table VI reports Sync data values (45.50, 88.40, 88.40, 35.50) that are all lower than Real data (52.91, 92.00, 90.55, 37.18). If the table rows are swapped or the metrics mislabeled, this must be corrected; as written, Sec. III-F and Sec. IV-C's central claim about post-training data selection has no support.
  3. [Sec. I, Sec. IV-B, Tables II-III] The abstract/introduction claim of 'best appearance consistency and motion smoothness' is stronger than the evidence. In Table II, MiraMo's FVD is worse than Cinemo on UCF-101 (201.90 vs 168.16) and marginally worse on MSR-VTT (93.91 vs 93.51); in Table III, Cinemo has higher Motion Smoothness (98.21 vs 96.93) and Camera Motion (30.68 vs 21.57). The user study (Table IV) only includes SVD and SEINE, not Cinemo or PIA. The defensible claim is 'comparable FVD/CLIPSIM with better aesthetic quality and dynamic degree at lower compute.' The text should be revised to that claim, or additional experiments provided.
  4. [Sec. III-B, Eq. (9)] Eq. (9) as written does not have a non-negative denominator: D_i = n + (q_i/||q_i||) Σ_j (k_j/||k_j||)^T can approach zero because the sum of unit vectors has no lower bound. The paper's justification for rejecting Eq. (7) is that RoPE may make inner products negative, but the same concern applies to the denominator of Eq. (9). No empirical stability analysis is given. Since this attention is load-bearing for the efficiency claim, the authors should either provide a proof or regularizer ensuring D_i is bounded away from zero, or report numerical safeguards and a stability ablation.
  5. [Sec. III-C, Algorithm 1] The motion-residual training formulation is underspecified. The notation z1 is used both for the first-frame latent and for the Gaussian endpoint in the flow-matching interpolation z_t = t z1 + (1-t) z0 (Sec. III-A). In Algorithm 1, M = {z_i - z_1} is computed, then 'noised Mt via the diffusion process' and input X_t = cat([z_1, M_t + z_1]) are defined, but the flow-matching latent variables z_0 and z_1 for the residual distribution are never defined, nor is the velocity target v_t in Eq. (14) expressed in terms of M. This makes the training objective ambiguous and prevents reproducibility. Please define the noising schedule and target for residuals explicitly and reconcile the notation.
minor comments (5)
  1. [Eq. (3)] The text says 'σ is the sigmoid function', but the scaled dot-product attention in Eq. (3) uses softmax, not sigmoid. This typo is confusing in a central equation.
  2. [Sec. III-B] Typo: 'Finlay' should be 'Finally'. Also the phrase 'increase both training and inference efficiency' should likely be 'increase computational cost'.
  3. [Sec. IV-A] The implementation details state 'MiraMo is identical to SANA [9]' but the SANA reference is [38], not [9]. Please correct the citation.
  4. [Sec. IV-A] The inference-speed claim would be stronger with wall-clock generation times per video, not only throughput/memory plots. The current ablation reports no end-to-end latency for the I2V pipeline.
  5. [Sec. IV-C] Table V reports 'Average' and 'Average Consistency' without defining their composition beyond a parenthetical in the text. Please specify exactly which VBench dimensions are averaged and the number of prompts/videos used.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MiraMo's components are validated by fresh ablations and external benchmarks; self-citations are contextual, not load-bearing.

full rationale

I walked the claimed derivation chain and found no step where a prediction or first-principles result is equivalent to its inputs by construction. The linear attention mechanism (Eq. 8-9) is an architectural choice with an analytical O(N) complexity claim, and its quality parity is tested empirically in Fig. 3 against a vanilla attention Transformer (throughput, memory, VBench), not merely asserted as a consequence of its definition. Motion residual learning (Sec. III-C) is a supervised learning objective on frame differences, validated by ablations in Table V (DPF vs MRL) and Fig. 11; the target residuals are computed from training videos and are not fitted to the test outputs. DCTInit (Sec. III-D) is an inference-time noise initialization method compared against FFT and no-refinement baselines, again empirically. Dynamics degree control (Sec. III-E) uses an MS-SSIM-based bucket conditioned via AdaIN and is validated by Fig. 9 and the control experiments in Sec. IV-C. The paper does cite the authors' own prior work [37] as the basis for the extension, but the new experiments (Tables II-V, user study, ablations) are fresh evaluation runs on external benchmarks, so the self-citation is not load-bearing. The main identifiable weaknesses—e.g., the lack of a fully described controlled swap for the linear-vs-vanilla attention comparison and the asserted superiority of frame-only temporal attention over 3D temporal attention—are absent experimental details, not circular reasoning. No equation reduces to its own input, and no fitted parameter is renamed as a prediction. Hence the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The model is assembled from pretrained components (SANA T2I, its VAE) and trained on Vimeo25M plus model-generated videos; none of these are derived in the paper. The genuinely free choices are the dynamics bucketization, the frame-interval sampling, the unspecified DCT filter H, and the inference hyperparameters. The paper introduces no invented entities: the motion-residual representation is a reparameterization of the data (Sec. III-C) and DCTInit is a sampling procedure (Sec. III-D). The most fragile axioms are the expressivity of the cosine-kernel temporal attention (Eq. 9) and the unmeasured premise that a train/inference noise discrepancy motivates DCTInit.

free parameters (5)
  • dynamics degree bucket count = 20 buckets
    Hand-chosen discretization of the Eq. 13 MS-SSIM frame similarity into conditioning buckets b=0..19 (Sec. III-E).
  • training clip frame interval = 3 to 10 frames
    Randomly sampled interval to 'alleviate skewness' of the long-tail dynamics distribution (Sec. III-E); affects the dynamics conditioning label quality.
  • DCT low-pass filter H = not specified
    The cutoff or mask for H in Eqs. 10-11 is never defined; it controls how much input-image low-frequency content is injected at inference (Sec. III-D).
  • inference solver steps and CFG scale = 25 steps, guidance 5.5
    Flow-DPM-Solver steps and classifier-free guidance scale chosen without ablations (Sec. IV-A).
  • synthetic post-training dataset size = 150,000 videos
    Scale of the Wan2.1/HunyuanVideo-generated SFT set (Sec. III-F); the claimed benefit is contradicted by Table VI as printed.
assumptions (5)
  • standard math Flow-matching objective (Eq. 2) with logit-normal timestep schedule (Stable Diffusion 3)
    Invoked in Sec. III-A and used for all training; taken from [42], [43].
  • domain assumption SANA's ReLU linear attention and Mix-FFN blocks, with pretrained weights, transfer to video with only added temporal layers
    Sec. III-B builds MiraMo's T2V backbone on SANA [38]; the base model's quality is inherited, not derived.
  • domain assumption A frame-wise image-level VAE is an adequate video latent encoder
    Sec. III-C compresses each frame with the SANA VAE; the authors themselves flag this causes flickering in Sec. IV-D limitations.
  • standard math RoPE preserves cosine similarity because it does not change vector magnitudes, making Eq. 8 RoPE-compatible
    Stated in Sec. III-B; correct for rotation matrices, but the claim that ReLU linear attention (Eq. 7) becomes unstable under RoPE is asserted without analysis.
  • ad hoc to paper Flow-matching training with logit-normal timesteps creates a train/inference noise discrepancy fixable by injecting input-image low-frequency DCT content
    Sec. III-D motivates DCTInit; the discrepancy is inferred from related DDPM work [34], [111], not measured for this model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Consistent and Controllable Image Animation with Motion Linear Diffusion Transformers." pith.science (2026). https://pith.science/paper/QDGZXO7X

@misc{pith2026250807246,
  author       = {Pith},
  title        = {Pith review of: Consistent and Controllable Image Animation with Motion Linear Diffusion Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDGZXO7X}},
  note         = {Machine review of arXiv:2508.07246}
}
read the original abstract

Image animation has seen significant progress, driven by the powerful generative capabilities of diffusion models. However, maintaining appearance consistency with static input images and mitigating abrupt motion transitions in generated animations remain persistent challenges. While text-to-video (T2V) generation has demonstrated impressive performance with diffusion transformer models, the image animation field still largely relies on U-Net-based diffusion models, which lag behind the latest T2V approaches. Moreover, the quadratic complexity of vanilla self-attention mechanisms in Transformers imposes heavy computational demands, making image animation particularly resource-intensive. To address these issues, we propose MiraMo, a framework designed to enhance efficiency, appearance consistency, and motion smoothness in image animation. Specifically, MiraMo introduces three key elements: (1) A foundational text-to-video architecture replacing vanilla self-attention with efficient linear attention to reduce computational overhead while preserving generation quality; (2) A novel motion residual learning paradigm that focuses on modeling motion dynamics rather than directly predicting frames, improving temporal consistency; and (3) A DCT-based noise refinement strategy during inference to suppress sudden motion artifacts, complemented by a dynamics control module to balance motion smoothness and expressiveness. Extensive experiments against state-of-the-art methods validate the superiority of MiraMo in generating consistent, smooth, and controllable animations with accelerated inference speed. Additionally, we demonstrate the versatility of MiraMo through applications in motion transfer and video editing tasks.

Figures

Figures reproduced from arXiv: 2508.07246 by the authors.

Figure 1
Figure 1. Image animation from our method. Please visit the project page to visualize the animations. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Appearance consistency and motion smoothness. (a) Upper row: appearance changes over time (result from PIA [1]). (b) Lower row: the car takes a sudden turnaround (result from SEINE [2]). directly used in real-world video production such as photog￾raphy, filmmaking, and augmented reality. Recent advancements in large-scale diffusion models have enabled generalization to open domains, demonstrating suc￾cess in image [… view at source ↗
Figure 3
Figure 3. , our MiraMo achieves higher throughput, lower GPU memory consumption, and better performance compared to the vanilla attention-based Transformer model. Thirdly, to enable our linear Transformer to effectively capture temporal relationships in videos, we introduce a new linear attention mechanism that is compatible with RoPE positional embed￾ding. Fourthly, to further improve the final video quality of the linear Tr… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Model pipeline overview. Our model learns the distribution of motion residuals instead of predicting the frames directly. The details of the training procedure can be seen in the algorithm. 1. manual segmentation, enabling the synthesis of more natural movements [70], …
Figure 5
Figure 5. Figure 5: The structure of the SANA block and Mix-FFN. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: The power spectrums of DCT and FFT. leakage due to edge effects since FFT assumes the input signal is periodic, and the finite length of input images results in these edge effects; and (2) poor concentration of low-frequency energy. The complex exponential basis functi…
Figure 7
Figure 7. Figure 7: Qualitative visual comparisons. We compare Cinemo with both closed-source commercial tools and open-source research works. “a ship sailing on the water” is the user prompt. Our videos are cropped for better comparison. Please visit the project page to visualize the com…
Figure 8
Figure 8. Figure 8: (a) vs. (b): The impact of FFT and DCT decomposition. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Dynamics degree control. From top to bottom, the bucket value b increases and effectively increases the speed of the car. Please visit the project page to visualize the animations. Models I2V Average Consistency Average Consistency Dynamic Degree DPF 92.62 93.62 32.68 …
Figure 11
Figure 11. Figure 11: Results by using alternative motion residual rep￾resentation. The generated videos tend to degrade rapidly. The benefit of synthetic video data. As shown in Tab. VI, the synthetic video dataset enhances the quality of the gen￾erated videos. The model trained on synthe…
Figure 10
Figure 10. Figure 10: The performance differences between four different [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 12
Figure 12. Figure 12: Motion control by textual prompts. Our model can effectively respond to textual prompts, leading to visually appealing outcomes. Please visit the project page to visualize the animations. Original video Output video Original video Output video Original first frame Mod…
Figure 13
Figure 13. Figure 13: Motion transfer/Video editing results. Our model can be easily extended to motion transfer or video editing on any given video from any source, not necessarily the ones generated by our model. Our model can also handle different resolutions when applied to other appli…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

126 extracted references · 59 canonical work pages

  1. [1]

    Pia: Your personalized image animator via plug-and-play modules in text-to- image models,

    Y . Zhang, Z. Xing, Y . Zeng, Y . Fang, and K. Chen, “Pia: Your personalized image animator via plug-and-play modules in text-to- image models,” in Computer Vision and Pattern Recognition , 2024

    2024

  2. [2]

    Seine: Short-to-long video diffusion model for generative transition and prediction,

    X. Chen, Y . Wang, L. Zhang, S. Zhuang, X. Ma, J. Yu, Y . Wang, D. Lin, Y . Qiao, and Z. Liu, “Seine: Short-to-long video diffusion model for generative transition and prediction,” in International Conference on Learning Representations, 2023

    2023

  3. [3]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Computer Vision and Pattern Recognition , 2022, pp. 10 684–10 695

    2022

  4. [4]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis,

    J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Y . Wu, Z. Wang, J. Kwok, P. Luo, H. Lu et al., “Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis,” in International Conference on Learning Representations , 2024

    2024

  5. [5]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis,

    D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, and R. Rombach, “Sdxl: Improving latent diffusion models for high-resolution image synthesis,” in International Conference on Learning Representations, 2024

    2024

  6. [6]

    Magic3d: High-resolution text- to-3d content creation,

    C.-H. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang, K. Kreis, S. Fidler, M.-Y . Liu, and T.-Y . Lin, “Magic3d: High-resolution text- to-3d content creation,” in Computer Vision and Pattern Recognition , 2023, pp. 300–309

    2023

  7. [7]

    Dreamfusion: Text- to-3d using 2d diffusion,

    B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, “Dreamfusion: Text- to-3d using 2d diffusion,” in International Conference on Learning Representations, 2023

    2023

  8. [8]

    Instant3d: Fast text-to-3d with sparse- view generation and large reconstruction model,

    J. Li, H. Tan, K. Zhang, Z. Xu, F. Luan, Y . Xu, Y . Hong, K. Sunkavalli, G. Shakhnarovich, and S. Bi, “Instant3d: Fast text-to-3d with sparse- view generation and large reconstruction model,” in International Conference on Learning Representations , 2024

    2024

Show all 126 references
  1. [9]

    Lavie: High-quality video generation with cascaded latent diffusion models,

    Y . Wang, X. Chen, X. Ma, S. Zhou, Z. Huang, Y . Wang, C. Yang, Y . He, J. Yu, P. Yanget al., “Lavie: High-quality video generation with cascaded latent diffusion models,” International Journal of Computer Vision, pp. 1–20, 2024

    2024

  2. [10]

    Latte: Latent diffusion transformer for video generation,

    X. Ma, Y . Wang, X. Chen, G. Jia, Z. Liu, Y .-F. Li, C. Chen, and Y . Qiao, “Latte: Latent diffusion transformer for video generation,” Transactions on Machine Learning Research , 2025

    2025

  3. [11]

    Align your latents: High-resolution video synthesis with latent diffusion models,

    A. Blattmann, R. Rombach, H. Ling, T. Dockhorn, S. W. Kim, S. Fidler, and K. Kreis, “Align your latents: High-resolution video synthesis with latent diffusion models,” in Computer Vision and Pattern Recognition , 2023, pp. 22 563–22 575

    2023

  4. [12]

    Videofusion: Decomposed diffusion models for high-quality video generation,

    Z. Luo, D. Chen, Y . Zhang, Y . Huang, L. Wang, Y . Shen, D. Zhao, J. Zhou, and T. Tan, “Videofusion: Decomposed diffusion models for high-quality video generation,” in Computer Vision and Pattern Recognition, 2023, pp. 10 209–10 218

    2023

  5. [13]

    Make-a-video: Text-to-video generation without text-video data,

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni et al. , “Make-a-video: Text-to-video generation without text-video data,” in International Conference on Learning Representations, 2023

    2023

  6. [14]

    Animatediff: Animate your personalized text-to-image diffusion mod- els without specific tuning,

    Y . Guo, C. Yang, A. Rao, Y . Wang, Y . Qiao, D. Lin, and B. Dai, “Animatediff: Animate your personalized text-to-image diffusion mod- els without specific tuning,” in International Conference on Learning Representations, 2024

    2024

  7. [15]

    Animateanything: Fine-grained open domain image animation with motion guidance,

    Z. Dai, Z. Zhang, Y . Yao, B. Qiu, S. Zhu, L. Qin, and W. Wang, “Animateanything: Fine-grained open domain image animation with motion guidance,” arXiv e-prints, pp. arXiv–2311, 2023

    2023

  8. [16]

    Dynamicrafter: Animating open-domain images with video diffusion priors,

    J. Xing, M. Xia, Y . Zhang, H. Chen, W. Yu, H. Liu, G. Liu, X. Wang, Y . Shan, and T.-T. Wong, “Dynamicrafter: Animating open-domain images with video diffusion priors,” in European Conference on Computer Vision. Springer, 2024, pp. 399–417

    2024

  9. [17]

    I2vgen-xl: High-quality image-to-video synthe- sis via cascaded diffusion models,

    S. Zhang, J. Wang, Y . Zhang, K. Zhao, H. Yuan, Z. Qin, X. Wang, D. Zhao, and J. Zhou, “I2vgen-xl: High-quality image-to-video synthe- sis via cascaded diffusion models,” arXiv preprint arXiv:2311.04145 , 2023

    2023 arXiv

  10. [18]

    Consisti2v: Enhancing visual consistency for image-to-video genera- tion,

    W. Ren, H. Yang, G. Zhang, C. Wei, X. Du, S. Huang, and W. Chen, “Consisti2v: Enhancing visual consistency for image-to-video genera- tion,” Transactions on Machine Learning Research , 2024

    2024

  11. [19]

    Motionstone: Decoupled motion intensity modulation with diffusion transformer for image-to-video generation,

    S. Shi, B. Gong, X. Chen, D. Zheng, S. Tan, Z. Yang, Y . Li, J. He, K. Zheng, J. Chen et al. , “Motionstone: Decoupled motion intensity modulation with diffusion transformer for image-to-video generation,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 22 864–22 874

    2025

  12. [20]

    Through-the-mask: Mask-based motion trajectories for image-to-video generation,

    G. Yariv, Y . Kirstain, A. Zohar, S. Sheynin, Y . Taigman, Y . Adi, S. Benaim, and A. Polyak, “Through-the-mask: Mask-based motion trajectories for image-to-video generation,” in Computer Vision and Pattern Recognition Conference, June 2025, pp. 18 198–18 208. 13

    2025

  13. [21]

    Extrapolating and decoupling image-to-video generation models: Motion modeling is easier than you think,

    J. Tian, X. Qu, Z. Lu, W. Wei, S. Liu, and Y . Cheng, “Extrapolating and decoupling image-to-video generation models: Motion modeling is easier than you think,” in Computer Vision and Pattern Recognition Conference, 2025, pp. 12 512–12 521

    2025

  14. [22]

    I2vguard: Safeguarding images against misuse in diffusion-based image-to-video models,

    D. Gui, X. Guo, W. Zhou, and Y . Lu, “I2vguard: Safeguarding images against misuse in diffusion-based image-to-video models,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 12 595–12 604

    2025

  15. [23]

    Mo- tionpro: A precise motion controller for image-to-video generation,

    Z. Zhang, F. Long, Z. Qiu, Y . Pan, W. Liu, T. Yao, and T. Mei, “Mo- tionpro: A precise motion controller for image-to-video generation,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 27 957–27 967

    2025

  16. [24]

    Levitor: 3d trajectory oriented image-to-video synthesis,

    H. Wang, H. Ouyang, Q. Wang, W. Wang, K. L. Cheng, Q. Chen, Y . Shen, and L. Wang, “Levitor: 3d trajectory oriented image-to-video synthesis,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 12 490–12 500

    2025

  17. [25]

    Osv: One step is enough for high-quality image to video generation,

    X. Mao, Z. Jiang, F.-Y . Wang, J. Zhang, H. Chen, M. Chi, Y . Wang, and W. Luo, “Osv: One step is enough for high-quality image to video generation,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 12 585–12 594

    2025

  18. [26]

    Motif: Making text count in image animation with motion focal loss,

    S. Wang, S. Azadi, R. Girdhar, S. Rambhatla, C. Sun, and X. Yin, “Motif: Making text count in image animation with motion focal loss,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 7773–7783

    2025

  19. [27]

    Hunyuanvideo: A systematic framework for large video generative models,

    W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang et al., “Hunyuanvideo: A systematic framework for large video generative models,” arXiv preprint arXiv:2412.03603 , 2024

    2024 arXiv

  20. [28]

    Wan: Open and advanced large-scale video generative models,

    T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu, H. Zhao, J. Yang et al. , “Wan: Open and advanced large-scale video generative models,” arXiv preprint arXiv:2503.20314 , 2025

    2025 arXiv

  21. [29]

    Cogvideox: Text-to-video diffu- sion models with an expert transformer,

    Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y . Yang, W. Hong, X. Zhang, G. Feng et al., “Cogvideox: Text-to-video diffu- sion models with an expert transformer,” in International Conference on Learning Representations , 2025

    2025

  22. [30]

    Open-sora plan: Open-source large video generation model,

    B. Lin, Y . Ge, X. Cheng, Z. Li, B. Zhu, S. Wang, X. He, Y . Ye, S. Yuan, L. Chen et al. , “Open-sora plan: Open-source large video generation model,” arXiv preprint arXiv:2412.00131 , 2024

    2024 arXiv

  23. [31]

    Open-sora: Democratizing efficient video production for all,

    Z. Zheng, X. Peng, T. Yang, C. Shen, S. Li, H. Liu, Y . Zhou, T. Li, and Y . You, “Open-sora: Democratizing efficient video production for all,” arXiv preprint arXiv:2412.20404 , 2024

    2024 arXiv

  24. [32]

    Ltx-video: Realtime video latent diffusion,

    Y . HaCohen, N. Chiprut, B. Brazowski, D. Shalem, D. Moshe, E. Richardson, E. Levin, G. Shiran, N. Zabari, O. Gordon et al. , “Ltx-video: Realtime video latent diffusion,” arXiv preprint arXiv:2501.00103, 2024

    2024 arXiv

  25. [33]

    Roformer: En- hanced transformer with rotary position embedding,

    J. Su, M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “Roformer: En- hanced transformer with rotary position embedding,” Neurocomputing, vol. 568, p. 127063, 2024

    2024

  26. [34]

    Freeu: Free lunch in diffusion u-net,

    C. Si, Z. Huang, Y . Jiang, and Z. Liu, “Freeu: Free lunch in diffusion u-net,” in Computer Vision and Pattern Recognition , 2024

    2024

  27. [35]

    Freeinit: Bridging initialization gap in video diffusion models,

    T. Wu, C. Si, Y . Jiang, Z. Huang, and Z. Liu, “Freeinit: Bridging initialization gap in video diffusion models,” in European Conference on Computer Vision . Springer, 2024, pp. 378–394

    2024

  28. [36]

    Input perturbation reduces exposure bias in diffusion models,

    M. Ning, E. Sangineto, A. Porrello, S. Calderara, and R. Cucchiara, “Input perturbation reduces exposure bias in diffusion models,” in International Conference on Machine Learning , 2023

    2023

  29. [37]

    Consistent and controllable image animation with motion diffusion models,

    X. Ma, Y . Wang, G. Jia, X. Chen, T.-T. Wong, Y .-F. Li, and C. Chen, “Consistent and controllable image animation with motion diffusion models,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 7288–7298

    2025

  30. [38]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers,

    E. Xie, J. Chen, J. Chen, H. Cai, H. Tang, Y . Lin, Z. Zhang, M. Li, L. Zhu, Y . Lu et al., “Sana: Efficient high-resolution image synthesis with linear diffusion transformers,” in International Conference on Learning Representations, 2025

    2025

  31. [39]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Neural Information Processing Systems, vol. 33, 2020, pp. 6840–6851

    2020

  32. [40]

    Denoising diffusion implicit mod- els,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit mod- els,” in International Conference on Learning Representations , 2021

    2021

  33. [41]

    Score-based generative modeling through stochastic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” in International Conference on Learning Rep- resentations, 2021

    2021

  34. [42]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boesel et al. , “Scaling rectified flow transformers for high-resolution image synthesis,” in International Conference on Machine Learning , 2024

    2024

  35. [43]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” in International Conference on Learning Representations, 2023

    2023

  36. [44]

    Zero-shot text-to-image generation,

    A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. V oss, A. Radford, M. Chen, and I. Sutskever, “Zero-shot text-to-image generation,” in International Conference on Machine Learning . Pmlr, 2021, pp. 8821–8831

    2021

  37. [45]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimanset al., “Photorealistic text-to-image diffusion models with deep language understanding,” Neural Information Processing Systems , vol. 35, pp. 36 479–36...

    2022

  38. [46]

    Scaling autoregressive models for content-rich text-to-image generation,

    J. Yu, Y . Xu, J. Y . Koh, T. Luong, G. Baid, Z. Wang, V . Vasudevan, A. Ku, Y . Yang, B. K. Ayan et al. , “Scaling autoregressive models for content-rich text-to-image generation,” Transactions on Machine Learning Research, vol. 2, no. 3, p. 5, 2022

    2022

  39. [47]

    Hierarchi- cal text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchi- cal text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022

    2022 arXiv

  40. [48]

    Visual autoregres- sive modeling: Scalable image generation via next-scale prediction,

    K. Tian, Y . Jiang, Z. Yuan, B. Peng, and L. Wang, “Visual autoregres- sive modeling: Scalable image generation via next-scale prediction,” Neural Information Processing Systems , vol. 37, pp. 84 839–84 865, 2024

    2024

  41. [49]

    Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis,

    J. Han, J. Liu, Y . Jiang, B. Yan, Y . Zhang, Z. Yuan, B. Peng, and X. Liu, “Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis,” in Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 15 733–15 744

    2025

  42. [50]

    Training-free stylized text-to-image generation with fast inference,

    X. Ma, Y . Wang, X. Chen, T.-T. Wong, and C. Chen, “Training-free stylized text-to-image generation with fast inference,” arXiv preprint arXiv:2505.19063, 2025

    2025 arXiv

  43. [51]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,

    J. Z. Wu, Y . Ge, X. Wang, S. W. Lei, Y . Gu, Y . Shi, W. Hsu, Y . Shan, X. Qie, and M. Z. Shou, “Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,” in International Conference on Computer Vision , 2023, pp. 7623–7633

    2023

  44. [52]

    Preserve your own correlation: A noise prior for video diffusion models,

    S. Ge, S. Nah, G. Liu, T. Poon, A. Tao, B. Catanzaro, D. Jacobs, J.-B. Huang, M.-Y . Liu, and Y . Balaji, “Preserve your own correlation: A noise prior for video diffusion models,” in International Conference on Computer Vision, 2023, pp. 22 930–22 941

    2023

  45. [53]

    Magicvideo: Efficient video generation with latent diffusion models,

    D. Zhou, W. Wang, H. Yan, W. Lv, Y . Zhu, and J. Feng, “Magicvideo: Efficient video generation with latent diffusion models,” arXiv preprint arXiv:2211.11018, 2022

    2022 arXiv

  46. [54]

    Latent video dif- fusion models for high-fidelity long video generation,

    Y . He, T. Yang, Y . Zhang, Y . Shan, and Q. Chen, “Latent video dif- fusion models for high-fidelity long video generation,” arXiv preprint arXiv:2211.13221, 2022

    2022 arXiv

  47. [55]

    Phenaki: Variable length video generation from open domain textual descriptions,

    R. Villegas, M. Babaeizadeh, P.-J. Kindermans, H. Moraldo, H. Zhang, M. T. Saffar, S. Castro, J. Kunze, and D. Erhan, “Phenaki: Variable length video generation from open domain textual descriptions,” in International Conference on Learning Representations , 2022

    2022

  48. [56]

    Movie gen: A cast of media foun- dation models, 2025,

    A. Polyak, A. Zohar, A. Brown, A. Tjandra, A. Sinha, A. Lee, A. Vyas, B. Shi, C. Ma, C. Chuang et al., “Movie gen: A cast of media foun- dation models, 2025,” URL https://arxiv. org/abs/2410.13720 , p. 51, 2024

    2025 arXiv

  49. [57]

    Autoregressive video generation without vector quantiza- tion,

    H. Deng, T. Pan, H. Diao, Z. Luo, Y . Cui, H. Lu, S. Shan, Y . Qi, and X. Wang, “Autoregressive video generation without vector quantiza- tion,” in International Conference on Learning Representations , 2025

    2025

  50. [58]

    Pyramidal flow matching for efficient video generative modeling,

    Y . Jin, Z. Sun, N. Li, K. Xu, H. Jiang, N. Zhuang, Q. Huang, Y . Song, Y . Mu, and Z. Lin, “Pyramidal flow matching for efficient video generative modeling,” in International Conference on Learning Representations, 2025

    2025

  51. [59]

    Spatial- temporal graph mamba for music-guided dance video synthesis,

    H. Tang, L. Shao, Z. Zhang, L. Van Gool, and N. Sebe, “Spatial- temporal graph mamba for music-guided dance video synthesis,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2025

    2025

  52. [60]

    Magictime: Time-lapse video generation models as metamorphic simulators,

    S. Yuan, J. Huang, Y . Shi, Y . Xu, R. Zhu, B. Lin, X. Cheng, L. Yuan, and J. Luo, “Magictime: Time-lapse video generation models as metamorphic simulators,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2025

    2025

  53. [61]

    Cascaded dynamic memory refinement and semantic alignment for exo-to-ego cross-view video generation,

    W. Hu, J. T. Hoe, J. Li, H. Hu, X. Jiang, and Y .-P. Tan, “Cascaded dynamic memory refinement and semantic alignment for exo-to-ego cross-view video generation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2025

    2025

  54. [62]

    Efficient diffusion models: A comprehensive survey from principles to practices,

    Z. Ma, Y . Zhang, G. Jia, L. Zhao, Y . Ma, M. Ma, G. Liu, K. Zhang, N. Ding, J. Li et al. , “Efficient diffusion models: A comprehensive survey from principles to practices,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2025

    2025

  55. [63]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,” Neural Information Processing Systems , vol. 35, pp. 8633–8646, 2022. 14

    2022

  56. [64]

    Imagen video: High definition video generation with diffusion models,

    J. Ho, W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, D. J. Fleet et al. , “Imagen video: High definition video generation with diffusion models,” arXiv preprint arXiv:2210.02303, 2022

    2022 arXiv

  57. [65]

    Cascaded diffusion models for high fidelity image generation,

    J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi, and T. Salimans, “Cascaded diffusion models for high fidelity image generation,” Jour- nal of Machine Learning Research , vol. 23, no. 47, pp. 1–33, 2022

    2022

  58. [66]

    Stable video diffusion: Scaling latent video diffusion models to large datasets,

    A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y . Levi, Z. English, V . V oleti, A. Lettset al., “Stable video diffusion: Scaling latent video diffusion models to large datasets,”arXiv preprint arXiv:2311.15127, 2023

    2023 arXiv

  59. [67]

    Automatic animation of hair blowing in still portrait photos,

    W. Xiao, W. Liu, Y . Wang, B. Ghanem, and B. Li, “Automatic animation of hair blowing in still portrait photos,” in International Conference on Computer Vision , 2023, pp. 22 963–22 975

    2023

  60. [68]

    Warp-guided gans for single-photo facial animation,

    J. Geng, T. Shao, Y . Zheng, Y . Weng, and K. Zhou, “Warp-guided gans for single-photo facial animation,” ACM Transactions on Graphics , vol. 37, no. 6, pp. 1–12, 2018

    2018

  61. [69]

    Imaginator: Conditional spatio-temporal gan for video generation,

    Y . Wang, P. Bilinski, F. Bremond, and A. Dantcheva, “Imaginator: Conditional spatio-temporal gan for video generation,” in Winter Con- ference on Applications of Computer Vision , 2020, pp. 1160–1169

    2020

  62. [70]

    Latent image animator: Learning to animate images via latent space navigation,

    Y . Wang, D. Yang, F. Bremond, and A. Dantcheva, “Latent image animator: Learning to animate images via latent space navigation,” in International Conference on Learning Representations , 2022

    2022

  63. [71]

    Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer,

    J. Cui, H. Li, Y . Zhan, H. Shang, K. Cheng, Y . Ma, S. Mu, H. Zhou, J. Wang, and S. Zhu, “Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 21 086–21 095

    2025

  64. [72]

    Zero-1-to-a: Zero-shot one image to animatable head avatars using video diffusion,

    Z. Zhou, F. Ma, H. Fan, and T.-S. Chua, “Zero-1-to-a: Zero-shot one image to animatable head avatars using video diffusion,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 15 941–15 952

    2025

  65. [73]

    Blowing in the wind: Cyclenet for human cinemagraphs from still images,

    H. Bertiche, N. J. Mitra, K. Kulkarni, C.-H. P. Huang, T. Y . Wang, M. Madadi, S. Escalera, and D. Ceylan, “Blowing in the wind: Cyclenet for human cinemagraphs from still images,” in Computer Vision and Pattern Recognition, 2023, pp. 459–468

    2023

  66. [74]

    Under- standing object dynamics for interactive image-to-video synthesis,

    A. Blattmann, T. Milbich, M. Dorkenwald, and B. Ommer, “Under- standing object dynamics for interactive image-to-video synthesis,” in Computer Vision and Pattern Recognition , 2021, pp. 5171–5181

    2021

  67. [75]

    Motion representations for articulated animation,

    A. Siarohin, O. J. Woodford, J. Ren, M. Chai, and S. Tulyakov, “Motion representations for articulated animation,” in Computer Vision and Pattern Recognition, 2021, pp. 13 653–13 662

    2021

  68. [76]

    Leo: Generative latent image animator for human video synthesis,

    Y . Wang, X. Ma, X. Chen, C. Chen, A. Dantcheva, B. Dai, and Y . Qiao, “Leo: Generative latent image animator for human video synthesis,” International Journal of Computer Vision , pp. 1–13, 2024

    2024

  69. [77]

    Lia: Latent image animator,

    Y . Wang, D. Yang, F. Bremond, and A. Dantcheva, “Lia: Latent image animator,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  70. [78]

    Stableanimator: High-quality identity-preserving human image anima- tion,

    S. Tu, Z. Xing, X. Han, Z.-Q. Cheng, Q. Dai, C. Luo, and Z. Wu, “Stableanimator: High-quality identity-preserving human image anima- tion,” in Computer Vision and Pattern Recognition Conference , 2025, pp. 21 096–21 106

    2025

  71. [79]

    X-dyna: Expressive dynamic human image animation,

    D. Chang, H. Xu, Y . Xie, Y . Gao, Z. Kuang, S. Cai, C. Zhang, G. Song, C. Wang, Y . Shi, Z. Chen, S. Zhou, L. Luo, G. Wetzstein, and M. Soleymani, “X-dyna: Expressive dynamic human image animation,” in Computer Vision and Pattern Recognition Conference , June 2025, pp. 5499–5509

    2025

  72. [80]

    Animating pictures with stochastic motion textures,

    Y .-Y . Chuang, D. B. Goldman, K. C. Zheng, B. Curless, D. H. Salesin, and R. Szeliski, “Animating pictures with stochastic motion textures,” ACM Transactions on Graphics, vol. 24, no. 3, p. 853–860, Jul. 2005

    2005

  73. [81]

    Animating animal motion from still,

    X. Xu, L. Wan, X. Liu, T.-T. Wong, L. Wang, and C.-S. Leung, “Animating animal motion from still,” ACM Transactions on Graphics, vol. 27, no. 5, 2008

    2008

  74. [82]

    Stochastic image-to-video synthesis using cinns,

    M. Dorkenwald, T. Milbich, A. Blattmann, R. Rombach, K. G. Derpa- nis, and B. Ommer, “Stochastic image-to-video synthesis using cinns,” in Computer Vision and Pattern Recognition , 2021, pp. 3742–3753

    2021

  75. [83]

    A phase-based approach for animating images using video examples,

    E. Prashnani, M. Noorkami, D. Vaquero, and P. Sen, “A phase-based approach for animating images using video examples,” in Computer Graphics Forum, vol. 36. Wiley Online Library, 2017, pp. 303–311

    2017

  76. [84]

    F 3a-gan: Facial flow for face animation with generative adversarial networks,

    X. Wu, Q. Zhang, Y . Wu, H. Wang, S. Li, L. Sun, and X. Li, “F 3a-gan: Facial flow for face animation with generative adversarial networks,” IEEE Transactions on Image Processing, vol. 30, pp. 8658–8670, 2021

    2021

  77. [85]

    Animegan: A novel lightweight gan for photo animation,

    J. Chen, G. Liu, and X. Chen, “Animegan: A novel lightweight gan for photo animation,” in Artificial Intelligence Algorithms and Applications. Springer, 2020, pp. 242–256

    2020

  78. [86]

    Ganimation: Anatomically-aware facial animation from a single image,

    A. Pumarola, A. Agudo, A. M. Martinez, A. Sanfeliu, and F. Moreno- Noguer, “Ganimation: Anatomically-aware facial animation from a single image,” in European Conference on Computer Vision, 2018, pp. 818–833

    2018

  79. [87]

    Videocomposer: Compositional video synthesis with motion controllability,

    X. Wang, H. Yuan, S. Zhang, D. Chen, J. Wang, Y . Zhang, Y . Shen, D. Zhao, and J. Zhou, “Videocomposer: Compositional video synthesis with motion controllability,” Neural Information Processing Systems , vol. 36, 2024

    2024

  80. [88]

    Videocrafter1: Open diffusion models for high-quality video generation,

    H. Chen, M. Xia, Y . He, Y . Zhang, X. Cun, S. Yang, J. Xing, Y . Liu, Q. Chen, X. Wang et al. , “Videocrafter1: Open diffusion models for high-quality video generation,” arXiv preprint arXiv:2310.19512, 2023

    2023 arXiv

  81. [89]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning . PMLR, 2021, pp. 8748–8763

    2021

  82. [90]

    Animatelcm: Accelerating the animation of personalized diffusion models and adapters with decoupled consistency learning,

    F.-Y . Wang, Z. Huang, X. Shi, W. Bian, G. Song, Y . Liu, and H. Li, “Animatelcm: Accelerating the animation of personalized diffusion models and adapters with decoupled consistency learning,” in ACM Special Interest Group for Computer Graphics and Interactive Tech- niques Asia, 2024

    2024

  83. [91]

    Dreamvideo: High-fidelity image-to-video generation with image retention and text guidance,

    C. Wang, J. Gu, P. Hu, S. Xu, H. Xu, and X. Liang, “Dreamvideo: High-fidelity image-to-video generation with image retention and text guidance,” arXiv preprint arXiv:2312.03018 , 2023

    2023 arXiv

  84. [92]

    Vdt: General-purpose video diffusion transformers via mask modeling,

    H. Lu, G. Yang, N. Fei, Y . Huo, Z. Lu, P. Luo, and M. Ding, “Vdt: General-purpose video diffusion transformers via mask modeling,” in International Conference on Learning Representations , 2023

    2023

  85. [93]

    I2v-adapter: A general image-to-video adapter for diffusion models,

    X. Guo, M. Zheng, L. Hou, Y . Gao, Y . Deng, P. Wan, D. Zhang, Y . Liu, W. Hu, Z. Zha et al., “I2v-adapter: A general image-to-video adapter for diffusion models,” in ACM Special Interest Group on Computer Graphics, 2024, pp. 1–12

    2024

  86. [94]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations , 2022

    2022

  87. [95]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation,

    L. Hu, “Animate anyone: Consistent and controllable image-to-video synthesis for character animation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 8153–8163

    2024

  88. [96]

    Magicanimate: Temporally consistent human image animation using diffusion model,

    Z. Xu, J. Zhang, J. H. Liew, H. Yan, J.-W. Liu, C. Zhang, J. Feng, and M. Z. Shou, “Magicanimate: Temporally consistent human image animation using diffusion model,” in Computer Vision and Pattern Recognition, 2024

    2024

  89. [97]

    Snapfusion: Text-to-image diffusion model on mobile devices within two seconds,

    Y . Li, H. Wang, Q. Jin, J. Hu, P. Chemerys, Y . Fu, Y . Wang, S. Tulyakov, and J. Ren, “Snapfusion: Text-to-image diffusion model on mobile devices within two seconds,” in Neural Information Pro- cessing Systems, vol. 36, 2023, pp. 20 662–20 678

    2023

  90. [98]

    Mobilediffusion: Instant text-to-image generation on mobile devices,

    Y . Zhao, Y . Xu, Z. Xiao, H. Jia, and T. Hou, “Mobilediffusion: Instant text-to-image generation on mobile devices,” in European Conference on Computer Vision . Springer, 2024, pp. 225–242

    2024

  91. [99]

    Bitsfusion: 1.99 bits weight quantization of diffusion model,

    Y . Sui, Y . Li, A. Kag, Y . Idelbayev, J. Cao, J. Hu, D. Sagar, B. Yuan, S. Tulyakov, and J. Ren, “Bitsfusion: 1.99 bits weight quantization of diffusion model,” in International Conference on Learning Represen- tations, 2024

    2024

  92. [100]

    Linfusion: 1 gpu, 1 minute, 16k image,

    S. Liu, W. Yu, Z. Tan, and X. Wang, “Linfusion: 1 gpu, 1 minute, 16k image,” arXiv preprint arXiv:2409.02097 , 2024

    2024 arXiv

  93. [101]

    Snapgen: Taming high- resolution text-to-image models for mobile devices with efficient ar- chitectures and training,

    J. Chen, D. Hu, X. Huang, H. Coskun, A. Sahni, A. Gupta, A. Goyal, D. Lahiri, R. Singh, Y . Idelbayev et al. , “Snapgen: Taming high- resolution text-to-image models for mobile devices with efficient ar- chitectures and training,” in Computer Vision and Pattern Recognition Con...

    2025

  94. [102]

    Matten: Video generation with mamba-attention,

    Y . Gao, J. Huang, X. Sun, Z. Jie, Y . Zhong, and L. Ma, “Matten: Video generation with mamba-attention,” arXiv preprint arXiv:2405.03025 , 2024

    2024 arXiv

  95. [103]

    Lingen: Towards high-resolution minute-length text-to-video generation with linear computational com- plexity,

    H. Wang, C.-Y . Ma, Y .-C. Liu, J. Hou, T. Xu, J. Wang, F. Juefei- Xu, Y . Luo, P. Zhang, T. Hou et al., “Lingen: Towards high-resolution minute-length text-to-video generation with linear computational com- plexity,” in Computer Vision and Pattern Recognition Conference , 202...

    2025

  96. [104]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” in Conference on Language Modeling , 2024

    2024

  97. [105]

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality,

    T. Dao and A. Gu, “Transformers are ssms: Generalized models and efficient algorithms through structured state space duality,” in International Conference on Machine Learning , 2024

    2024

  98. [106]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in International Conference on Computer Vision, 2023, pp. 4195–4205

    2023

  99. [107]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,”arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  100. [108]

    An introduction to variational autoencoders,

    D. P. Kingma, M. Welling et al. , “An introduction to variational autoencoders,” Foundations and Trends® in Machine Learning, vol. 12, no. 4, pp. 307–392, 2019. 15

    2019

  101. [109]

    Non-local neural networks,

    X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Computer Vision and Pattern Recognition , 2018, pp. 7794–7803

    2018

  102. [110]

    S. Jianlin. (2020, Jul) Exploring linear attention: Does attention really need a softmax? [Online]. Available: https://kexue.fm/archives/7546

    2020

  103. [111]

    Common diffusion noise schedules and sample steps are flawed,

    S. Lin, B. Liu, J. Li, and X. Yang, “Common diffusion noise schedules and sample steps are flawed,” in Winter Conference on Applications of Computer Vision, 2024, pp. 5404–5411

    2024

  104. [112]

    Videofusion: Decomposed diffusion models for high-quality video generation,

    Z. Luo, D. Chen, Y . Zhang, Y . Huang, L. Wang, Y . Shen, D. Zhao, J. Zhou, and T. Tan, “Videofusion: Decomposed diffusion models for high-quality video generation,” in Computer Vision and Pattern Recognition, June 2023

    2023

  105. [113]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models,

    H. Chen, Y . Zhang, X. Cun, M. Xia, X. Wang, C. Weng, and Y . Shan, “Videocrafter2: Overcoming data limitations for high-quality video diffusion models,” in Computer Vision and Pattern Recognition , 2024, pp. 7310–7320

    2024

  106. [114]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers,

    W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang, “Cogvideo: Large-scale pretraining for text-to-video generation via transformers,” in International Conference on Learning Representations , 2022

    2022

  107. [115]

    Hierarchical spatio-temporal decoupling for text-to-video generation,

    Z. Qing, S. Zhang, J. Wang, X. Wang, Y . Wei, Y . Zhang, C. Gao, and N. Sang, “Hierarchical spatio-temporal decoupling for text-to-video generation,” in Computer Vision and Pattern Recognition , 2024

    2024

  108. [116]

    Vbench: Comprehensive benchmark suite for video generative models,

    Z. Huang, Y . He, J. Yu, F. Zhang, C. Si, Y . Jiang, Y . Zhang, T. Wu, Q. Jin, N. Chanpaisit et al. , “Vbench: Comprehensive benchmark suite for video generative models,” in Computer Vision and Pattern Recognition, 2024, pp. 21 807–21 818

    2024

  109. [117]

    Follow-your-click: Open-domain regional image animation via short prompts,

    Y . Ma, Y . He, H. Wang, A. Wang, C. Qi, C. Cai, X. Li, Z. Li, H.- Y . Shum, W. Liu et al. , “Follow-your-click: Open-domain regional image animation via short prompts,” in AAAI Conference on Artificial Intelligence, 2025

    2025

  110. [118]

    Msr-vtt: A large video description dataset for bridging video and language,

    J. Xu, T. Mei, T. Yao, and Y . Rui, “Msr-vtt: A large video description dataset for bridging video and language,” in Computer Vision and Pattern Recognition, 2016, pp. 5288–5296

    2016

  111. [119]

    A dataset of 101 human action classes from videos in the wild,

    K. Soomro, A. R. Zamir, and M. Shah, “A dataset of 101 human action classes from videos in the wild,” Center for Research in Computer Vision, vol. 2, no. 11, pp. 1–7, 2012

    2012

  112. [120]

    Fvd: A new metric for video generation,

    T. Unterthiner, S. van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly, “Fvd: A new metric for video generation,” inInternational Conference on Learning Representations Workshop , 2019

    2019

  113. [121]

    Temporal generative adversarial nets with singular value clipping,

    M. Saito, E. Matsumoto, and S. Saito, “Temporal generative adversarial nets with singular value clipping,” in International Conference on Computer Vision, 2017, pp. 2830–2839

    2017

  114. [122]

    On buggy resizing li- braries and surprising subtleties in fid calculation,

    G. Parmar, R. Zhang, and J.-Y . Zhu, “On buggy resizing li- braries and surprising subtleties in fid calculation,” arXiv preprint arXiv:2104.11222, vol. 5, p. 14, 2021

    2021 arXiv

  115. [123]

    Godiva: Generating open-domain videos from natural descriptions,

    C. Wu, L. Huang, Q. Zhang, B. Li, L. Ji, F. Yang, G. Sapiro, and N. Duan, “Godiva: Generating open-domain videos from natural descriptions,” arXiv preprint arXiv:2104.14806 , 2021

    2021 arXiv

  116. [124]

    Raft: Recurrent all-pairs field transforms for optical flow,

    Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” in European Conference on Computer Vision. Springer, 2020, pp. 402–419

    2020

  117. [125]

    Stable flow: Vital layers for training- free image editing,

    O. Avrahami, O. Patashnik, O. Fried, E. Nemchinov, K. Aberman, D. Lischinski, and D. Cohen-Or, “Stable flow: Vital layers for training- free image editing,” in Computer Vision and Pattern Recognition Conference, 2025, pp. 7877–7888

    2025

  118. [126]

    Plug-and-play diffu- sion features for text-driven image-to-image translation,

    N. Tumanyan, M. Geyer, S. Bagon, and T. Dekel, “Plug-and-play diffu- sion features for text-driven image-to-image translation,” in Computer Vision and Pattern Recognition , 2023, pp. 1921–1930

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.