Pith. sign in

REVIEW 3 major objections 3 minor 2 cited by

Motion Modes: What Could Happen Next?

T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Motion Modes shows that a training-free scheme can probe a pre-trained flow-based image-to-video generator to discover several distinct, plausible motions for a selected object in a static image, while suppressing camera and background…

desk verdict Useful training-free method for diverse object motion discovery; the evaluation leans too heavily on self-confirming metrics and one overclaimed human comparison, but the core idea is sound and worth refereeing. read the letter →

arxiv 2412.00148 v1 pith:WOPK5BAO submitted 2024-11-29 cs.CV

Karran Pandey , Matheus Gadelha , Yannick Hold-Geoffroy , Karan Singh , Niloy J. Mitra , Paul Guerrero This is my paper · ORCID
classification cs.CV
keywords motionpredictionimage-to-videogenerationdiffusionguidancetraining-freediversitysamplingfieldsobjectanimationdrag-basedediting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a pre-trained image-to-video generator can be probed at inference time, without any training or fine-tuning, to discover several distinct, plausible motions for a selected object in a static image. It proposes Motion Modes, which steers the denoising process of a flow-based generator with energy terms that keep the camera static, move the masked object, push each new motion away from previously found ones, and smooth the motion in time. The authors report that on 28 scenes with articulated objects, animals, vehicles, waves, and flags, this guided sampling beats random-noise, prompt-based, arrow-based, and ControlNet baselines on diversity and focus, and that human raters frequently prefer its motions for plausibility, diversity, and expectation. If correct, the result matters because it turns a black-box video prior into an exploration tool for cinematic ideation and drag-based image editing without needing task-specific training data.

What carries the argument

The load-bearing objects are the flow generator and four guidance energies. The flow generator models motion as a per-pixel 2D offset field over frames and predicts it separately from appearance, so object and camera motion are partly separated from lighting and shadow changes before guidance begins. The energies are: static-camera $E_c$, which penalizes the average motion magnitude outside the object mask; object-motion $E_o$, a soft-inverse activation applied to the difference between average motion inside and outside the mask; diversity $E_d$, a repulsion from previously found motions measured by a masked angular-and-magnitude distance with weights $w_{mag}=0.25$, $w_{angle}=0.75$; and smoothness $E_s$, which penalizes large frame-to-frame changes inside the mask with weights $w_{mag}=0.75$, $w_{angle}=0.25$. At inference, the gradient of the combined energy is applied to the predicted noise-free motion in each denoising step; iterative sampling discards motions with final energy above $\rho=5.0$ and stops after two consecutive discards.

What would settle it

Run the same backbone with 1000 unguided random noise draws on a fixed set of masked-object scenes, and check whether Motion Modes ever returns a motion that is essentially absent among those random samples; if not, the diversity guidance only selects from already-sampleable motions rather than discovering new modes, and if the returned motions are judged implausible by human raters, the plausibility claim is contradicted.

Watch

Extended reading notes

Core claim

The central discovery is that a flow-based image-to-video generator's latent distribution already contains enough diverse, plausible object motions that a composition of simple energy functions can extract them, disentangled from camera motion and other scene changes. Motion Modes represents a motion as a time-dependent 2D vector field, uses a pre-trained flow generator that predicts this field separately from appearance, and during each denoising step perturbs the predicted clean motion in the direction that lowers the weighted sum of four energies. Iterating this guided sampling builds a set of motions; a threshold on the final guidance energy discards implausible samples and stops when the scene seems to have no more new modes. The paper reports that the resulting motions are more focused and more diverse than four baselines at equal sample budget, and that in a user study 96% of generated motions were judged plausible, 92% of participants' expected motions were produced, and 19% were plausible but outside expectation.

Load-bearing premise

The method assumes the pre-trained generator's latent distribution already contains diverse, plausible motions for the given object and scene; if the prior cannot express a motion, Motion Modes cannot discover it.

Editorial extensions

If this is right

  • Because the method is training-free, it can be applied to any pre-trained flow-based image-to-video generator that generates motion separately from appearance, so future improvements to the backbone likewise improve motion discovery without retraining the guidance.
  • Up to six distinct motions are typically sampled per object, and the stopping criterion automatically gives fewer motions for scenes that admit fewer plausible futures.
  • The discovered motions can be converted into detailed drag-arrow sets for drag-based editors and motion-to-video generators, replacing a single ambiguous arrow with a complete, plausible motion and avoiding artifacts such as a floating train or a squashed drawer.
  • On the paper's evaluations, the guided motions outperform random noise, random arrows, ControlNet-restricted motion, farthest-point-sampled noise, and LLM-generated prompts on both diversity and focus, and are judged more plausible, diverse, and expected by users.
  • The same motions can condition multiple videos: the paper shows that different random noises with the same motion follow the motion accurately while differing in small details.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the energy recipe transfers across flow-based generators, the practical contribution is an inference-time control layer rather than a new model, meaning a generator upgrade would immediately boost motion discovery without re-deriving guidance.
  • Sequential repulsion samples modes one at a time, so continuous motion spaces (a laptop sliding anywhere on a desk) are represented by discrete exemplars; clustering guided samples or interpolating between them in latent space could give continuous motion control.
  • The dependence on the prior's coverage implies a measurable bias gap: for a fixed scene category, comparing Motion Modes' discovered distributions with human-annotated plausible motions would quantify how much of the missing motion diversity is the generator's data bias rather than the guidance scheme.
  • The arrow-completion application suggests a direct causal test: if replacing a user's single drag arrow with the closest retrieved motion consistently reduces editing artifacts across many users and scenes, that would characterize when detailed motion completion helps versus when it over-constrains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. Motion Modes proposes a training-free, inference-time guidance method for image-to-video generation. Given a static image and an object mask, the method samples the motion prior of a pre-trained flow-based generator (Motion-I2V) while minimizing energy functions that encourage static camera, focused object motion, diversity across sampled motions, and temporal smoothness. The sampled motions are then rendered into videos. The paper evaluates the method on 28 images against several baselines (prompt-based, ControlNet, random arrows, random noise, FPS noise) using energy-based metrics (Tables 1 and 2), two user studies, and qualitative examples, and also demonstrates an arrow-completion application.

Significance. If the central claim holds, the paper offers a practical, training-free way to discover diverse and plausible object motions from a single image, with clear applications in animation and video editing. The method is well-motivated and the user studies provide non-circular evidence that human judges prefer our motions to baselines on plausibility, diversity, and expectedness. The main weakness is that the headline quantitative metrics (Tables 1 and 2) are exactly the energy functions being optimized during guided denoising, making those numbers partly tautological; the abstract's claim of 'surpassing ... human predictions' is not supported by the experimental design. The limitations are honestly stated, including the reliance on the generator's prior and the discrete sampling of continuous motion spaces.

major comments (3)
  1. [Section 4, Table 1, Eq. (4)] The 'diverse' metric is defined as the average diversity guidance energy E_d, and the 'focused' metric is 0.5(E_o + E_c), which are precisely the terms minimized by the guided denoising step in Eq. (4): x'_t = x_t - ∇_{x_t} E(x0_θ, m, X) with E = λ_d E_d + λ_c E_c + λ_o E_o + λ_s E_s. Consequently, the result that Motion Modes outperforms all baselines on Table 1 is largely a restatement that the optimization achieved its objective; the metric does not independently validate the method. Please add non-circular quantitative metrics (e.g., optical-flow-based plausibility or external motion-quality scores), or present the user studies as the primary quantitative evidence and relegate the energy-based numbers to a sanity check.
  2. [Abstract and Section 4 (User Study II)] The abstract's claim of 'surpassing ... human predictions regarding plausibility and diversity' is not supported by the experimental design. Study 2 asks participants to list expected motions and then measures the fraction of those expectations that our motions cover (92%); it does not compare our motions against human-generated motion predictions in a head-to-head evaluation. Please either remove this claim or add an experiment where humans produce motion predictions for the same images and are compared against our outputs on plausibility and diversity.
  3. [Section 4, Table 2] The ablation study uses the same energy-based metrics as Table 1, so the conclusions are partly confounded. For example, removing E_d and then measuring E_d will trivially show a worse diversity score, since the metric is the very term removed; removing E_c and E_o similarly directly increases the focused metric E_f. The qualitative statement that each component improves the diversity-focus tradeoff is thus not supported by independent evidence. Please evaluate the ablations with the user-study protocol or another external metric, or clearly state that the energy-based ablation numbers are expected by construction.
minor comments (3)
  1. [Section 3.2] Typo: 'Similiar' should be 'Similar' in the sentence beginning 'Similiar to classifier-free guidance'.
  2. [Section 4, Table 1 and Table 2] The tables report no error bars or significance tests. Since the numbers are dominated by the optimization objective, adding variance or confidence intervals would help readers assess stability across the 28 images.
  3. [Section 4, User Studies] The number of images differs between the quantitative evaluation (28), User Study I (27), and User Study II (10). Please clarify the reason for these differences and whether they affect comparability of the reported percentages.

Circularity Check

1 steps flagged · score 6.0 of 10

Table 1's 'diverse' and 'focused' metrics are the same energy terms the method explicitly minimizes, so the headline quantitative comparison is partly circular.

  1. self definitional [Section 3.2 (Guided Inference) and Section 4 (Quantitative comparison, Table 1)]
    "First, the diversity of motions in a set X can be measured with the average diversity guidance energy ¯Ed(m, X ) := P x∈X Ed(x, m, X )/|X |. Second, the focus of motions on only the selected object can be measured based on the average object motion and static camera guidance energies ¯Ef := 0.5( ¯Eo + ¯Ec), with ¯Eo and ¯Ec computed analogous to ¯Ed, but scaled by a factor of 0.01 and 0.1, respectively, to account for scaling differences."

    The proposed method generates motions by minimizing E = λdEd + λcEc + λoEo + λsEs during denoising (Eq. 4), and the success metrics in Table 1 are then defined as ¯Ed (the diversity guidance energy) and ¯Ef = 0.5(¯Eo + ¯Ec) (the object and static-camera guidance energies). Because Motion Modes is the only method that explicitly optimizes these energies while the baselines do not, its lower (better) scores on 'diverse' and 'focused' are largely guaranteed by construction: the metric restates the optimized objective. Tables 2 and 3 use the same self-referential metrics for ablation claims. The independent user studies (Fig. 4, Fig. 8) provide non-circular evidence, but the headline quantitative comparison is not an independent test of diversity or focus.

full rationale

The paper's method itself is training-free and self-contained: it uses Motion-I2V and particle guidance as external building blocks and does not depend on author-specific uniqueness theorems or self-citations. The load-bearing circularity lies in the evaluation. The 'diverse' and 'focused' quantitative metrics reported in Table 1 are the average diversity guidance energy E_d and the average of the object/camera guidance energies E_o and E_c, exactly the terms minimized by the guided denoising update of Eq. 4. Consequently, the result that Motion Modes scores best is, in part, a restatement that the optimizer minimized its own objective, rather than an independent measure of diversity or focus. The same issue propagates to the ablation tables. This circularity is partial because the human user studies (User Study I and the extended study in Figure 8) independently evaluate plausibility, diversity, and expectedness through pairwise comparisons, and the qualitative results do not rely on the self-defined energies. The abstract's stronger phrasing about surpassing human predictions is not directly supported by Study 2, which measures coverage of participants' stated expectations rather than a head-to-head comparison with human-generated predictions, but that is an evaluation-design gap rather than a further circular step.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method relies on several hand-set hyperparameters and assumptions about the pre-trained generator's prior. No new physical or conceptual entities are postulated. The free parameters are all manually chosen and affect the central results.

free parameters (6)
  • Guidance energy weights = lambda_d=3.0, lambda_c=0.2, lambda_o=0.025, lambda_s=0.1
    Hand-set weights balancing diversity, camera, object, and smoothness energies; not derived from data or a principled criterion.
  • Tau thresholds = tau=40 (object motion), tau=1 (diversity)
    Paper states tau is empirically set; these thresholds define when guidance is considered satisfactory.
  • Distance metric weights = w_mag=0.25, w_angle=0.75 (diversity); w_mag=0.75, w_angle=0.25 (smoothness)
    Chosen to emphasize direction in diversity and magnitude stability in smoothness; hand-tuned.
  • Stopping threshold rho = 5.0
    Motions with guidance energy above rho are discarded, and two consecutive discards stop sampling. Chosen by hand.
  • Guided timesteps = 20 of 25 denoising timesteps
    Guidance is applied only to the first 20 timesteps; this choice affects the behavior but is not justified by an independent criterion.
  • Number of sampled motions = up to 6
    The sampling budget for the motion set is set to 6 in experiments; this is an arbitrary cap.
assumptions (5)
  • domain assumption The pre-trained Motion-I2V flow generator provides a valid motion prior and separates motion from appearance.
    Section 3.1 states the approach applies to any pre-trained diffusion-based image-to-video model that generates motion and appearance independently. The whole method inherits this prior.
  • domain assumption Gradient-guided denoising steers samples toward lower-energy motions while staying on the natural motion manifold.
    Section 3.2 modifies the denoising step with energy gradients, following classifier-guidance intuition. This assumes the gradients do not push samples off the manifold in a harmful way.
  • domain assumption The object mask is given and correctly identifies the target object.
    All energies weight pixels by mask m; an incorrect mask would guide the wrong region and invalidate the disentanglement claim.
  • ad hoc to paper Pairwise repulsion with distance d produces a diverse set of distinct motions without collapsing to implausible samples.
    Section 3.2 (iii) borrows particle guidance intuition, but no proof is given that this energy separates modes in the motion prior.
  • ad hoc to paper The energy-based stopping criterion identifies when no new plausible motions remain.
    Section 3.3 discards samples above rho and stops after two consecutive discards; this heuristic is not independently validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Motion Modes: What Could Happen Next?." pith.science (2026). https://pith.science/paper/WOPK5BAO

@misc{pith2026241200148,
  author       = {Pith},
  title        = {Pith review of: Motion Modes: What Could Happen Next?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WOPK5BAO}},
  note         = {Machine review of arXiv:2412.00148}
}
read the original abstract

Predicting diverse object motions from a single static image remains challenging, as current video generation models often entangle object movement with camera motion and other scene changes. While recent methods can predict specific motions from motion arrow input, they rely on synthetic data and predefined motions, limiting their application to complex scenes. We introduce Motion Modes, a training-free approach that explores a pre-trained image-to-video generator's latent distribution to discover various distinct and plausible motions focused on selected objects in static images. We achieve this by employing a flow generator guided by energy functions designed to disentangle object and camera motion. Additionally, we use an energy inspired by particle guidance to diversify the generated motions, without requiring explicit training data. Experimental results demonstrate that Motion Modes generates realistic and varied object animations, surpassing previous methods and even human predictions regarding plausibility and diversity. Project Webpage: https://motionmodes.github.io/

Figures

Figures reproduced from arXiv: 2412.00148 by the authors.

Figure 1
Figure 1. Could you imagine how the scene evolves in each case? [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Motion Modes creates multiple distinct and plausible motions for a given object, disentangled from the motion of other objects, camera and other scene changes. We show three distinct object motions for each of [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Method Overview. We generate a motion x using a guided denoising approach, where guidance energies encourage smooth object motions that are disentangled from camera motions and distinct from previously generated motions. Iterative sampling gives us a set of diverse motions X . vector field defines the trajectory of each pixel as per-frame 2D offsets from its initial position. We generate motions by sampling an exist… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: User Study I. We compare the plausible, diverse, and expected nature of our motions to four baselines. Each pair of bars shows the percentage of comparisons in which our method or a baseline was judged favorably with 95% confidence intervals. the-art text-to-image gene…
Figure 5
Figure 5. Figure 5: Qualitative comparison. Each column shows the first three motions for the masked object in the input (left). Object trajectories have red endpoints, background trajectories (usually due to camera motion) are purple. Motion is additionally visualized by overlaying ghost…
Figure 6
Figure 6. Figure 6: Motion Completion. We can use our set of motions X to complete rough motion hints (single red arrows) given by the user as conditional input to either drag-based image editors like DragonDiffusion or Drag-A-Part, or motion-to-video generators like Motion-I2V. Using the…
Figure 7
Figure 7. Figure 7: Limitations. (Top) The video prior can limit quality (bent clock handles, two cat tails). (Bottom) Continuous motion spaces can only be sampled discretely. Future work. Motion Modes produces videos with negli￾gible camera motion. Extending our approach to generate obje…
Figure 8
Figure 8. Figure 8: Extended user study. We compare the plausible, diverse, and expected nature of our motions to five baselines, including the Random Arrows baseline. Each pair of bars shows the percentage of comparisons in which our method or a baseline was judged favorably with 95% con…
Figure 9
Figure 9. Figure 9: Multiple videos from one motion. We generate multiple videos from the same motion x. They differ in small details, but overall follow the motion accurately. D. Multiple Videos Generated for One Motion All videos in our experiments are obtained by first generating a mot…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A single video-diffusion framework composites both static images and dynamic footage along user-defined trajectories by transporting canonical foreground latents directly into the background latent sequence.

  2. VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation

    cs.CV 2025-02 conditional novelty 6.0 of 10

    VidCRAFT3 is a single image-to-video diffusion system that accepts camera, object, and lighting direction controls separately or jointly, trained in three stages with a new synthetic lighting dataset.

Reference graph

Works this paper leans on

28 extracted references · 19 canonical work pages · cited by 2 Pith papers

  1. [1]

    Diffuhaul: A training-free method for object dragging in images

    Omri Avrahami, Rinon Gal, Gal Chechik, Ohad Fried, Dani Lischinski, Arash Vahdat, and Weili Nie. Diffuhaul: A training-free method for object dragging in images. arXiv preprint arXiv:2406.01594, 2024. 3

    arXiv 2024

  2. [2]

    Mitra, Kuldeep Kulkarni, Chun- Hao Paul Huang, Tuanfeng Y

    Hugo Bertiche, Niloy J. Mitra, Kuldeep Kulkarni, Chun- Hao Paul Huang, Tuanfeng Y . Wang, Meysam Madadi, Sergio Escalera, and Duygu Ceylan. Blowing in the wind: Cyclenet for human cinemagraphs from still images. In CVPR, 2023. 1, 3

    work page 2023

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023. 1

    2023

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2

    arXiv 2023

  5. [5]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. 2

    2024

  6. [6]

    Jaakkola

    Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzi- lay, and Tommi S. Jaakkola. Particle guidance: non-i.i.d. diverse sampling with diffusion models. In ICLR, 2024. 1, 3

    work page 2024

  7. [7]

    Animateanything: Fine- grained open domain image animation with motion guidance,

    Zuozhuo Dai, Zhenghao Zhang, Yao Yao, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Animateanything: Fine- grained open domain image animation with motion guidance,

    work page

  8. [8]

    Interactive dynamic video

    Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham J Mysore, Fredo Durand, and William T Freeman. Interactive dynamic video. ACM TOG (SIGGRAPH), 34(4):1–9, 2015. 2

    work page 2015

Show all 28 references
  1. [9]

    Efros, and Aleksander Holynski

    Dave Epstein, Allan Jabri, Ben Poole, Alexei A. Efros, and Aleksander Holynski. Diffusion self-guidance for controllable image generation, 2023. 4

    2023

  2. [10]

    Ganspace: Discovering interpretable gan controls

    Erik H¨ark¨onen, Aaron Hertzmann, Jaakko Lehtinen, and Syl- vain Paris. Ganspace: Discovering interpretable gan controls. In NeurIPS, pages 9841–9850. Curran Associates, Inc., 2020. 2

    2020

  3. [11]

    Classifier-free diffusion guidance, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. 4

    2022

  4. [12]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 2

    2022

  5. [13]

    Learning to predict part mobility from a single static snapshot

    Ruizhen Hu, Wenchao Li, Oliver Van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang. Learning to predict part mobility from a single static snapshot. ACM TOG (SIGGRAPH), 36 (6), 2017. 2

    2017

  6. [14]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 4

    2013 arXiv

  7. [15]

    Dragapart: Learning a part-level motion prior for articulated objects

    Ruining Li, Chuanxia Zheng, Christian Rupprecht, and An- drea Vedaldi. Dragapart: Learning a part-level motion prior for articulated objects. In ECCV, 2024. 1, 3, 6, 11

    2024

  8. [16]

    Generative image dynamics

    Zhengqi Li, Richard Tucker, Noah Snavely, and Aleksander Holynski. Generative image dynamics. In CVPR, 2024. 1, 2

    2024

  9. [17]

    Mitra, Yong-Liang Yang, Dong-Ming Yan, Wilmot Li, and Maneesh Agrawala

    Niloy J. Mitra, Yong-Liang Yang, Dong-Ming Yan, Wilmot Li, and Maneesh Agrawala. Illustrating how mechanical assemblies work. ACM TOG (SIGGRAPH), 29(3):58:1–58:12,

  10. [18]

    Dragondiffusion: Enabling drag-style manipula- tion on diffusion models

    Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. Dragondiffusion: Enabling drag-style manipula- tion on diffusion models. In ICLR, 2024. 3, 6, 11

    2024

  11. [19]

    Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model

    Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, and Yinqiang Zheng. Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. ECCV, 2024. 10, 11

    2024

  12. [20]

    Drag your gan: Interactive point-based manipulation on the generative image manifold

    Xingang Pan, Ayush Tewari, Thomas Leimk ¨uhler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your gan: Interactive point-based manipulation on the generative image manifold. In ACM TOG (SIGGRAPH), page 1–11, 2023. 3

    2023

  13. [21]

    Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, and Niloy J. Mitra. Diffusion handles: Enabling 3d edits for diffusion models by lifting activations to 3d. 2024. 3, 4

    2024

  14. [22]

    Movie gen: A cast of media foundation models

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720,

    arXiv

  15. [23]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH 2024 Conference Papers , p...

    2024

  16. [24]

    Yujun Shi, Jun Hao Liew, Hanshu Yan, Vincent Y . F. Tan, and Jiashi Feng. Lightningdrag: Lightning fast and accurate drag-based image editing emerging from videos, 2024. 3

    2024

  17. [25]

    Yujun Shi, Chuhui Xue, Jun Hao Liew, Jiachun Pan, Han- shu Yan, Wenqing Zhang, Vincent Y . F. Tan, and Song Bai. Dragdiffusion: Harnessing diffusion models for interactive point-based image editing. In CVPR, pages 8839–8849, 2024. 3

    2024

  18. [26]

    Boximator: Generating rich and controllable motions for video synthesis, 2024

    Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guoqiang Wei, Liping Yuan, and Hang Li. Boximator: Generating rich and controllable motions for video synthesis, 2024. 3

    2024

  19. [27]

    Draganything: Motion control for any- thing using entity representation, 2024

    Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for any- thing using entity representation, 2024. 3

    2024

  20. [28]

    video of a basketball swishing through the hoop after a jump shot

    Zeqi Xiao, Yifan Zhou, Shuai Yang, and Xingang Pan. Video diffusion models are training-free motion interpreter and con- troller. arXiv preprint arXiv:2405.14864, 2024. 2, 3 9 0% 100% ExpectedPlausibile Diverse Prompt Gen. ControlNet FPSNoise OursRand. NoiseRand. Arrows Figure...

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.