Pith. sign in

REVIEW 3 major objections 7 minor 75 references

MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MotionShot claims that motion from a reference video can be transferred to an arbitrary target object, even with very different appearance and structure, without training, by aligning semantic keypoints, warping shapes, and steering a…

desk verdict A plausible training-free motion-transfer pipeline whose two-level alignment idea is worth taking seriously, but whose evidence is too thin to support the claimed margin and whose guidance step has an unaddressed spatial-alignment assumption. read the letter →

arxiv 2507.16310 v1 pith:Y2RTKD7T submitted 2025-07-22 cs.CV

Yanchen Liu , Yanan Sun , Zhening Xing , Junyao Gao , Kai Chen , Wenjie Pei This is my paper
classification cs.CV
keywords motiontransfertext-to-videogenerationtraining-freetemporalattentionguidancesemanticcorrespondencethinplatesplinediffusionmodelsretargeting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MotionShot sets out to show that motion from a reference video can be transferred to a target object with very different appearance and body structure without any training of the video model. The method's bet is that the needed correspondence can be recovered from one text prompt and one reference clip: first synthesize a fake target that shares the reference's initial pose, then match structure-aware keypoints using combined Stable Diffusion and DINO features, and finally warp the reference's shape onto the target with thin plate splines. Those warped frames produce temporal attention guidance that steers a frozen text-to-video diffusion model, so the generated video reproduces the reference motion while preserving the target's appearance. If the paper is right, motion transfer becomes a correspondence-plus-guidance problem rather than a per-object trained descriptor problem, and a single reference clip can animate objects as different as a dancing character and a cartoon bear.

What carries the argument

The load-bearing mechanism is the two-level motion alignment: semantic alignment establishes which parts of the reference correspond to which parts of the target, and morphological alignment reshapes the reference's motion onto the target's geometry. The concrete objects are (i) the structure-aware keypoint set $K_{\mathrm{ref}}$ and its matched counterpart $K_{\mathrm{tar}}$, (ii) the TPS warp defined by $T^t(p) = A^t[p;1] + \sum_i w_{t,i} U(\|K^t_{\mathrm{tar},i} - p\|_2)$, and (iii) the temporal attention guidance $\hat{\epsilon}_\theta = \epsilon_\theta(z_t, \mathrm{text}, t) - \lambda \nabla_{z_t} g$. The warp is what makes motion compatible with target structure; the attention guidance is what injects that motion into a frozen diffusion sampler. The fake target generated with ControlNet and a deliberately degraded segmentation map is the bridge that lets semantic matching work despite the fact that the real target does not exist yet.

What would settle it

Run MotionShot on a reference-target pair deliberately chosen to share no semantic parts, such as a waving flag as reference and a wooden chair as target, and inspect whether keypoint matches and warped frames remain coherent; the paper's own limitation section predicts failure. A second check targets the fake-target bridge: vary the ControlNet conditioning weight in fake-target generation from 0 to 1 and measure whether final motion fidelity tracks keypoint match precision; a collapse at weight 0 would confirm that the initial pose injected through the degraded segmentation map is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that fine-grained motion transfer between arbitrary objects is achievable without training by splitting motion into two levels. High-level semantic motion alignment matches keypoints sampled on the reference's contour and interior to the target through fused Stable Diffusion and DINOv2 features; low-level morphological motion alignment tracks those keypoints through the reference video with CoTracker3, builds a target keypoint sequence with global rotation and translation plus local polar-coordinate adjustments, and deforms reference frames into the target's shape with a TPS warp. The warped frames are then used to extract a temporal attention map, sparsified by top-k selection, and generation is guided by minimizing the energy $g = \|M \cdot (A_{\mathrm{ref}} - A_{\mathrm{gen}})\|_2^2$ during sampling. If correct, the method means a text prompt plus one reference video is enough to make an arbitrary target character perform the reference motion, even when the two objects share almost no shape vocabulary.

Load-bearing premise

The whole pipeline assumes that reliable semantic correspondences between reference and target can be read off from a generated fake-target image; if that fake target's pose or structure drifts from the intended target, or if reference and target share no recognizable semantic parts, the keypoint matching that every later stage depends on breaks, and the authors themselves note the result can be unpredictable.

Editorial extensions

If this is right

  • A text prompt plus a single reference clip is sufficient input, with no training, fine-tuning, or per-object landmark definitions needed for motion transfer.
  • Objects whose appearances and structures differ widely, such as a horse and a panda or an anime character and a cartoon bear, can share motion because correspondence is semantic rather than skeletal.
  • The quality of transfer is governed by the number and placement of sampled keypoints, and the paper's ablations settle on $m = 30$ to balance TPS deformation smoothness against overfitting.
  • Because guidance is applied only during early denoising steps with top-k sparsified temporal attention, the method can be used as a post-hoc control on an existing frozen text-to-video model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's framing suggests motion transfer is now mainly bounded by semantic correspondence quality, so better open-world matching models should improve transfer directly since the video generator itself is never changed.
  • Because the fake target is generated with a small ControlNet conditioning weight, the method inherits the text-to-image model's sensitivity, and generating several fake targets and aggregating their keypoint matches is a natural, untested way to reduce variance.
  • The early-step-only guidance implies the method could generalize to longer or more complex motion by applying the same attention constraint at later timesteps, but the paper does not explore this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. MotionShot is a training-free framework for transferring the motion of a reference video to a target object described by a text prompt. The pipeline has three stages: (i) semantic motion alignment, where a 'fake target' image is generated with ControlNet from a degraded segmentation map of the reference first frame, keypoints are sampled on the reference and fake target using SAM, and correspondences are established by nearest-neighbor matching in combined Stable Diffusion and DINOv2 feature spaces (Eq. 1); (ii) morphological motion alignment, where the reference keypoints are tracked with CoTracker3, a target keypoint sequence is constructed via global rotation/translation and local polar-coordinate adjustments (Eq. 2), and TPS warping maps the reference frames to the target shape (Eqs. 3-5); and (iii) attention-guided generation, where the temporal attention maps of the warped reference are used to guide the sampling of an AnimateDiff model by penalizing the difference with the generated video's temporal attention (Eqs. 6-7). The paper reports qualitative comparisons and a quantitative table (Table 1) with CLIP scores and a 20-volunteer user study.

Significance. If the method works as claimed, it would be a noteworthy training-free contribution to text-to-video motion transfer, combining semantic (SD+DINO) and geometric (TPS) alignment in a way that avoids per-instance training. The use of off-the-shelf components and the explicit two-level alignment strategy are conceptually appealing, and the paper shows qualitative examples of cross-object transfer that are more convincing than the baselines. However, the evaluation is currently too weak to establish the claims: the objective metrics show only marginal improvements, the user study is small and unreported in terms of variance and significance, and the spatial anchoring between the warped reference and the generated video is not guaranteed. The paper also does not release code, which limits reproducibility.

major comments (3)
  1. [Sec. 3.4, Eq. (6)] The energy function g penalizes the difference between the warped reference's temporal attention map A_ref and the generated video's A_gen at the same spatial locations p, but the generated video is never anchored to the coordinate frame of the fake target. The fake target is used only to construct correspondences and warps; the AnimateDiff generation is initialized from noise and driven by text, with no conditioning or initialization from the fake target or its layout. If the generated target object appears at a different position, scale, or initial pose than the fake target, the pixel-wise guidance supervises the wrong spatial locations, which can cause artifacts or appearance leakage rather than clean motion transfer. The paper should either provide an explicit spatial alignment mechanism (e.g., first-frame conditioning, attention injection, or a post-hoc alignment step) or demonstrate empirically that the generated object consistently aligns with the fake target across a diverse set of prompts. This is distinct from the acknowledged semantic-similarity limitation in Sec. 5.
  2. [Table 1 and Sec. 4.4] The quantitative claims are not supported by the reported numbers. The user study has only 20 volunteers, reports no standard deviations, confidence intervals, or significance tests, and gives MotionShot near-ceiling scores (4.90-4.95) while baselines receive 2.40-3.00, a gap that is implausibly large for a perceptual study. The objective metrics, which do have a defined scale, show only marginal differences (e.g., CLIP text alignment 26.95 vs 26.77 for VMC; temporal consistency 97.81 vs 97.72), and without error bars these differences may be within noise. The claim of significant outperformance requires per-video results, variance estimates, and statistical tests, as well as a description of the user-study protocol (e.g., how many videos each volunteer rated, whether sessions were timed, whether pairwise comparisons were used).
  3. [Sec. 3.2] The fake-target generation is load-bearing for the entire pipeline, but its key parameters are not specified or ablated. The paper states that a 'degraded segmentation map' of the reference is used and the ControlNet condition weight is set to 0.6, yet the degradation procedure is not described, and no sensitivity analysis is reported. Since the semantic correspondence, keypoint construction, and TPS warping all assume that the fake target has the intended target structure and a pose close to the reference, the robustness of the method to this step is unquantified. The authors should describe the degradation, vary the condition weight and degradation level, and report how these choices affect the final transfer quality.
minor comments (7)
  1. [Sec. 1] The phrase 'an unique two-level motion alignment strategy' should be 'a unique two-level motion alignment strategy'.
  2. [Sec. 3.2] There is a duplicate citation in 'Previous studies [19, 35, 53, 69, 69]'; the second '[69]' should be removed.
  3. [Sec. 3.4] The sentence 'This to the construction of a sparse control mask' is incomplete; it should read 'This leads to the construction of a sparse control mask'.
  4. [Sec. 4.2] The user-study criterion 'appearance diversity between input and generated videos' is not a standard metric and its meaning should be clarified; a high score could simply indicate that the generated subject differs from the reference, not that it matches the intended target appearance.
  5. [Title and Sec. 5] The title and abstract claim 'arbitrary objects', but Sec. 5 states that the method fails when reference and target share no semantic similarities; the scope limitation should be reflected in the terminology used (e.g., 'semantically related objects').
  6. [Sec. 4.1] The hyperparameters (control weight, number of sampling steps, guidance timestep, keypoint count) are fixed without sensitivity analysis; the paper would benefit from an ablation table for the most sensitive ones, particularly m and the guidance strength lambda.
  7. [Project page] The project page does not appear to link to code or checkpoints, which limits reproducibility; the authors should consider releasing the implementation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MotionShot is a training-free inference pipeline whose output is not forced by fitted parameters or by self-citation.

full rationale

MotionShot's derivation chain is a modular, training-free inference pipeline, not a fit-then-predict scheme. The intermediate fake target is generated by ControlNet from the prompt and a degraded reference segmentation (Sec. 3.2); keypoint correspondences come from SAM-based structure-aware sampling plus SD+DINO feature matching (Eq. 1); target trajectories are built with CoTracker3 and rigid rotation/shift operations (Eq. 2); and shape warping is obtained by solving the TPS bending-energy problem (Eqs. 3-5). The final video is produced by guiding a frozen AnimateDiff model through the temporal-attention energy function in Eqs. 6-7. No parameter is fitted to the evaluation data, and no output quantity is defined in terms of the input such that the claimed transfer is forced by construction. The paper's self-citations (StyleShot, FaceShot) appear only in related work and are not load-bearing: the central attention-guidance recipe explicitly follows the external MotionClone [34], and the semantic-correspondence premise is grounded in external SD/DINO feature-matching literature [69]. The acknowledged Sec. 5 limitation that semantically dissimilar pairs may yield unpredictable results is an honest boundary condition, not a disguised circularity. The reviewer's concern about the absence of an explicit spatial anchor between the fake-target coordinate frame and the generated video is a real robustness and correctness risk, but it is an assumption about spatial alignment, not an equivalence between inputs and outputs; Eqs. 6-7 define the guidance mechanism rather than deriving a prediction from fitted data. Overall, no circular step was found.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the reliability of the semantic/morphological correspondence and on the transferability of temporal attention guidance. While each component is drawn from prior work, the paper introduces the fake-target bootstrap and the two-level alignment as novel constructs; these are unvalidated assumptions that carry the load. Free parameters are numerous hand-tuned values with limited ablation.

free parameters (8)
  • keypoint count m = 30
    Number of sampled keypoints for semantic matching and TPS; chosen after ablation on a single qualitative example (Fig. 6).
  • contour sampling interval d = 200
    Uniform spacing of contour keypoints; set in Sec. 4.1 without ablation.
  • ControlNet condition weight = 0.6
    Weight balancing segmentation hint vs text in fake target generation (Sec. 4.1).
  • attention guidance timestep tau = 400
    Timestep at which temporal attention is extracted, following MotionClone [34] (Sec. 4.1).
  • top-k selection k = 1
    Number of sparse attention entries kept in the control mask, following [34] (Sec. 4.1).
  • guidance strength lambda = not reported
    Scaling of the attention-guidance gradient in Eq. (7); implementation details omit its value, which is critical for reproduction.
  • number of sampling steps and guidance schedule = 300 steps, guidance first 180
    DDIM sampling schedule (Sec. 4.1).
  • PCA retained dimension = unspecified
    The dimension reduction in semantic feature matching (Sec. 3.2) is not specified.
assumptions (6)
  • domain assumption Temporal attention maps in a frozen text-to-video diffusion model encode transferable motion information.
    Adopted from MotionClone [34] and used in Sec. 3.4 to define the guiding energy g; the paper does not independently validate this encoding for cross-appearance targets.
  • domain assumption Stable Diffusion features and DINOv2 features, when concatenated and PCA-reduced, provide reliable semantic correspondence between reference and target images.
    Borrowed from 'A tale of two features' [69] and used in Sec. 3.2 for keypoint matching; reliability is assumed for the arbitrary pairs targeted by the paper.
  • ad hoc to paper The ControlNet-generated 'fake target' approximates the intended target's structure well enough to anchor correspondences.
    The fake target is generated from a degraded segmentation map and the text prompt; if it diverges from the true target structure, keypoint correspondences will be wrong. The paper introduces this step specifically for its pipeline without a separate validation.
  • domain assumption CoTracker3 tracks sampled keypoints correctly across the reference video.
    Used in Sec. 3.3 to construct the reference keypoint sequence K_ref; tracking errors would propagate to the target sequence and warped frames.
  • standard math The thin-plate-spline bending-energy minimization yields a warp that preserves the reference motion while matching the target shape.
    Standard TPS mathematics; the paper relies on this interpolation to deform reference frames in Sec. 3.3.
  • domain assumption SAM segmentation masks are accurate enough for keypoint sampling.
    The structure-aware keypoint sampling in Sec. 3.2 depends on precise object masks from SAM for both reference and fake target images.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation." pith.science (2026). https://pith.science/paper/Y2RTKD7T

@misc{pith2026250716310,
  author       = {Pith},
  title        = {Pith review of: MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y2RTKD7T}},
  note         = {Machine review of arXiv:2507.16310}
}
read the original abstract

Existing text-to-video methods struggle to transfer motion smoothly from a reference object to a target object with significant differences in appearance or structure between them. To address this challenge, we introduce MotionShot, a training-free framework capable of parsing reference-target correspondences in a fine-grained manner, thereby achieving high-fidelity motion transfer while preserving coherence in appearance. To be specific, MotionShot first performs semantic feature matching to ensure high-level alignments between the reference and target objects. It then further establishes low-level morphological alignments through reference-to-target shape retargeting. By encoding motion with temporal attention, our MotionShot can coherently transfer motion across objects, even in the presence of significant appearance and structure disparities, demonstrated by extensive experiments. The project page is available at: https://motionshot.github.io/.

Figures

Figures reproduced from arXiv: 2507.16310 by the authors.

Figure 1
Figure 1. Visualization results of our MotionShot. Given any reference video, MotionShot can effectively transfer motion across arbitrary objects in text-to-video generation. Notably, when the reference and target objects have distinct appearances and structures—such as an anime boy and a Winnie bear—MotionShot demonstrates remarkable adaptive motion transfer capabilities. Abstract Existing text-to-video methods struggle to t… view at source ↗
Figure 2
Figure 2. The architecture of MotionShot, a training-free motion transfer method capable of handling reference-target object pairs with substantial appearance difference. A novel two-level motion alignment strategy, high-level semantic motion alignment as well as low-level morphological motion alignment, is introduced to establish the adaptive temporal attention guidance, leading to effective motion transfer. erence video whi… view at source ↗
Figure 3
Figure 3. Structure-aware keypoint sampling consisting of uni￾form contour sampling and Poisson disk internal sampling. fine-grained control over object motion. In practical scenar￾ios where users need precise control over object movement, they often provide a reference video that demonstrates the desired motion. This process, which involves transferring the motion depicted in the reference video to the target gen￾erated obje… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: TPS-based shape warping transfers the motion of the reference object while preserving the structure of the target object. Structure-aware keypoint sampling. After obtaining a fake target object, we establish semantic correspondence between the reference and target imag…
Figure 5
Figure 5. Figure 5: Visual comparison with baseline methods. MotionShot demonstrates strong semantic alignment and excellent morphological accuracy, whereas baseline methods are influenced by the shape of the reference object, resulting in poor morphological outcomes (e.g., the horse’s ne…
Figure 6
Figure 6. Figure 6: Ablation of number of sampled keypoints m. Only SD feature X-pose Only DINO feature Fuse SD & DINO feature Ref Points Tar Points Reshaped Frame Result Frame [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Influence of different keypoints matching methods. Our proposed method (Fuse SD & DINO feature) achieve best high-level semantic motion transfer result. Reference Frame Temporal Attn. Guidance Result Frame Original Resized TPS-warped [PITH_FULL_IMAGE:figures/full_fig_…
Figure 8
Figure 8. Figure 8: Influence of different shape retargeting methods. Our method produces motion that is well-aligned with the target sub￾ject, resulting in more harmonious visual outcomes. internal points) to m = 60 (48 contour points, 12 internal points). When m is small (e.g.,m = 10), …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 50 canonical work pages

  1. [1]

    Skeleton- aware networks for deep motion retargeting

    Kfir Aberman, Peizhuo Li, Dani Lischinski, Olga Sorkine- Hornung, Daniel Cohen-Or, and Baoquan Chen. Skeleton- aware networks for deep motion retargeting. TOG, 2020. 3

    work page 2020

  2. [2]

    Deep vit features as dense visual descriptors

    Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors. arXiv preprint arXiv:2112.05814, 2021. 5

    arXiv 2021

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2

    arXiv 2023

  4. [4]

    Bookstein

    Fred L. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. TPAMI, 1989. 5

    work page 1989

  5. [5]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. TOG,

    work page

  6. [6]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023. 2

    arXiv

  7. [7]

    Control-a-video: Controllable text-to-video generation with diffusion models

    Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv e-prints, pages arXiv–2305, 2023. 2, 3, 7

    work page 2023

  8. [8]

    Online motion retar- getting

    Kwang-Jin Choi and Hyeongseok Ko. Online motion retar- getting. Comput. Animat. Virtual Worlds, 2000. 3

    work page 2000

Show all 75 references
  1. [9]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. NeurIPS, 2021. 1, 7

    2021

  2. [10]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In ICCV, 2023. 2, 3, 7

    2023

  3. [11]

    Similarity distribution based member- ship inference attack on person re-identification

    Junyao Gao, Xinyang Jiang, Huishuai Zhang, Yifan Yang, Shuguang Dou, Dongsheng Li, Duoqian Miao, Cheng Deng, and Cairong Zhao. Similarity distribution based member- ship inference attack on person re-identification. InProceed- ings of the AAAI conference on artificial intellig...

    2023

  4. [12]

    Styleshot: A snap- shot on any style

    Junyao Gao, Yanchen Liu, Yanan Sun, Yinhao Tang, Yan- hong Zeng, Kai Chen, and Cairong Zhao. Styleshot: A snap- shot on any style. arXiv preprint arXiv:2407.01414, 2024. 2

    2024 arXiv

  5. [13]

    Faceshot: Bring any character into life

    Junyao Gao, Yanan Sun, Fei Shen, Xin Jiang, Zhening Xing, Kai Chen, and Cairong Zhao. Faceshot: Bring any character into life. arXiv preprint arXiv:2503.00740, 2025. 3

    2025 arXiv

  6. [14]

    Retargetting motion to new characters

    Michael Gleicher. Retargetting motion to new characters. SIGGRAPH, 1998. 3

    1998

  7. [15]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 1, 2, 7

    2023 arXiv

  8. [16]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In ECCV, 2024. 2, 3

    2024

  9. [17]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and Jos ´e Lezama. Photorealistic video generation with diffusion models. In ECCV, 2024. 2

    2024

  10. [18]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221 ,

    arXiv

  11. [19]

    Unsupervised keypoints from pretrained diffusion models

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Xingzhe He, Hossam Isack, Abhishek Kar, Helge Rhodin, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised keypoints from pretrained diffusion models. In CVPR, 2024. 3, 4

    2024

  12. [20]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 1

    2020

  13. [21]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. NeurIPS, 2022. 2

    2022

  14. [22]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. In CVPR, 2024. 2, 3

    2024

  15. [23]

    Pose-aware attention network for flexible mo- tion retargeting by body part

    Lei Hu, Zihao Zhang, Chongyang Zhong, Boyuan Jiang, and Shi hong Xia. Pose-aware attention network for flexible mo- tion retargeting by body part. TVCG, 2023. 3

    2023

  16. [24]

    Vmc: Video motion customization using temporal attention adap- tion for text-to-video diffusion models

    Hyeonho Jeong, Geon Yeong Park, and Jong Chul Ye. Vmc: Video motion customization using temporal attention adap- tion for text-to-video diffusion models. In CVPR, 2024. 3, 7

    2024

  17. [25]

    Text2performer: Text- driven human video generation

    Yuming Jiang, Shuai Yang, Tong Liang Koh, Wayne Wu, Chen Change Loy, and Ziwei Liu. Text2performer: Text- driven human video generation. In ICCV, 2023. 2

    2023

  18. [26]

    Co- tracker3: Simpler and better point tracking by pseudo- labelling real videos

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker3: Simpler and better point tracking by pseudo- labelling real videos. arXiv preprint arXiv:2410.11831 ,

    arXiv

  19. [27]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In ICCV, 2023. 2

    2023

  20. [28]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross B

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chlo´e Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross B. Girshick. Segment anything. ICCV, 2023. 4

    2023

  21. [29]

    Hunyuanvideo: A systematic framework for large video generative models, 2025

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang 9 Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai,...

    2025

  22. [30]

    A hierarchical approach to interactive motion editing for human-like figures

    Jehee Lee and Sung yong Shin. A hierarchical approach to interactive motion editing for human-like figures. SIG- GRAPH, 1999. 3

    1999

  23. [31]

    Animateanything: Consistent and con- trollable animation for video generation

    Guojun Lei, Chi Wang, Hong Li, Rong Zhang, Yikai Wang, and Weiwei Xu. Animateanything: Consistent and con- trollable animation for video generation. arXiv preprint arXiv:2411.10836, 2024. 2, 3

    2024 arXiv

  24. [32]

    Videogen: A reference-guided latent diffusion ap- proach for high definition text-to-video generation

    Xin Li, Wenqing Chu, Ye Wu, Weihang Yuan, Fanglong Liu, Qi Zhang, Fu Li, Haocheng Feng, Errui Ding, and Jingdong Wang. Videogen: A reference-guided latent diffusion ap- proach for high definition text-to-video generation. arXiv preprint arXiv:2309.00398, 2023. 1, 2

    2023 arXiv

  25. [33]

    Pmnet: Learning of disentangled pose and movement for unsuper- vised motion retargeting

    Jongin Lim, Hyung Jin Chang, and Jin Young Choi. Pmnet: Learning of disentangled pose and movement for unsuper- vised motion retargeting. In BMVC, 2019. 3

    2019

  26. [34]

    Motionclone: Training-free motion cloning for controllable video generation

    Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, and Yi Jin. Motionclone: Training-free motion cloning for controllable video generation. arXiv preprint arXiv:2406.05338, 2024. 3, 6, 7

    2024 arXiv

  27. [35]

    Diffusion hyperfeatures: Search- ing through time and space for semantic correspondence

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holyn- ski, and Trevor Darrell. Diffusion hyperfeatures: Search- ing through time and space for semantic correspondence. NeurIPS, 2023. 3, 4

    2023

  28. [36]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 2

    2024 arXiv

  29. [37]

    Follow your pose: Pose- guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose- guided text-to-video generation using pose-free videos. In AAAI, 2024. 2, 3

    2024

  30. [38]

    Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation

    Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, et al. Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation. In SIGGRAPH Asia, 2024. 2, 3

    2024

  31. [39]

    Principal components analysis (pca)

    Andrzej Ma ´ckiewicz and Waldemar Ratajczak. Principal components analysis (pca). Computers & Geosciences ,

  32. [40]

    Conform: Contrast is all you need for high- fidelity text-to-image diffusion models

    Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. Conform: Contrast is all you need for high- fidelity text-to-image diffusion models. In CVPR, 2024. 3

    2024

  33. [41]

    Motionflow: Attention-driven mo- tion transfer in video diffusion models

    Tuna Han Salih Meral, Hidir Yesiltepe, Connor Dunlop, and Pinar Yanardag. Motionflow: Attention-driven mo- tion transfer in video diffusion models. arXiv preprint arXiv:2412.05275, 2024. 3

    2024 arXiv

  34. [42]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  35. [43]

    Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model

    Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, and Yinqiang Zheng. Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. In ECCV, 2024. 2, 3

    2024

  36. [44]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 5

    2023 arXiv

  37. [45]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 2

    2023

  38. [46]

    The 2017 davis challenge on video object segmentation, 2018

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation, 2018. 7

    2017

  39. [47]

    Zoran Popovic and Andrew P. Witkin. Physically based mo- tion transformation. SIGGRAPH, 1999. 3

    1999

  40. [48]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125,

    arXiv

  41. [49]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2, 5

    2022

  42. [50]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015. 2

    2015

  43. [51]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  44. [52]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 1, 7

    2010 arXiv

  45. [53]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. NeurIPS, 2023. 3, 4

    2023

  46. [54]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. In CVPR, 2023. 3

    2023

  47. [55]

    Neural kinematic networks for unsupervised motion retargetting

    Ruben Villegas, Jimei Yang, Duygu Ceylan, and Honglak Lee. Neural kinematic networks for unsupervised motion retargetting. CVPR, 2018. 3

    2018

  48. [56]

    Contact-aware retargeting of skinned motion

    Ruben Villegas, Duygu Ceylan, Aaron Hertzmann, Jimei Yang, and Jun Saito. Contact-aware retargeting of skinned motion. ICCV, 2021. 3

    2021

  49. [57]

    Motion inversion for video customization

    Luozhou Wang, Ziyang Mai, Guibao Shen, Yixun Liang, Xin Tao, Pengfei Wan, Di Zhang, Yijun Li, and Yingcong Chen. Motion inversion for video customization. arXiv preprint arXiv:2403.20193, 2024. 3

    2024 arXiv

  50. [58]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. NeurIPS, 2023. 2, 3, 7 10

    2023

  51. [59]

    Lavie: High-quality video generation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. IJCV, 2024. 2

    2024

  52. [60]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In SIGGRAPH, 2024. 2, 3

    2024

  53. [61]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, 2023. 2, 7

    2023

  54. [62]

    Make-your-video: Cus- tomized video generation using textual and structural guid- ance

    Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xi- aodong Cun, Xintao Wang, et al. Make-your-video: Cus- tomized video generation using textual and structural guid- ance. TVCG, 2024. 2, 3

    2024

  55. [63]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In ECCV,

  56. [64]

    X- pose: Detecting any keypoints

    Jie Yang, Ailing Zeng, Ruimao Zhang, and Lei Zhang. X- pose: Detecting any keypoints. In ECCV, 2024. 8

    2024

  57. [65]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 2

    2024 arXiv

  58. [66]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023. 2, 3

    2023 arXiv

  59. [67]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation.arXiv preprint arXiv:2303.12346, 2023

    Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, et al. Nuwa-xl: Diffusion over diffusion for extremely long video generation.arXiv preprint arXiv:2303.12346, 2023. 1, 2

    2023 arXiv

  60. [68]

    Show-1: Marrying pixel and latent diffusion models for text-to-video generation

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent diffusion models for text-to-video generation. IJCV, 2024. 2

    2024

  61. [69]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. NeurIPS, 2023. 3, 4, 5

    2023

  62. [70]

    Skinned motion retargeting with residual per- ception of motion semantics & geometry

    Jiaxu Zhang, Junwu Weng, Di Kang, Fang Zhao, Shaoli Huang, Xuefei Zhe, Linchao Bao, Ying Shan, Jue Wang, and Zhigang Tu. Skinned motion retargeting with residual per- ception of motion semantics & geometry. CVPR, 2023. 3

    2023

  63. [71]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 4

    2023

  64. [72]

    Training-free motion-guided video generation with en- hanced temporal consistency using motion consistency loss

    Xinyu Zhang, Zicheng Duan, Dong Gong, and Lingqiao Liu. Training-free motion-guided video generation with en- hanced temporal consistency using motion consistency loss. arXiv preprint arXiv:2501.07563, 2025. 3

    2025 arXiv

  65. [73]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Jun- hao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In ECCV, 2024. 3

    2024

  66. [74]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 2

    2024 arXiv

  67. [75]

    Magicvideo: Efficient video generation with latent diffusion models

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022. 2 11

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.