REVIEW 6 cited by
MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present MOFA-Video, an advanced controllable image animation method that generates video from the given image using various additional controllable signals (such as human landmarks reference, manual trajectories, and another even provided video) or their combinations. This is different from previous methods which only can work on a specific motion domain or show weak control abilities with diffusion prior. To achieve our goal, we design several domain-aware motion field adapters (\ie, MOFA-Adapters) to control the generated motions in the video generation pipeline. For MOFA-Adapters, we consider the temporal motion consistency of the video and generate the dense motion flow from the given sparse control conditions first, and then, the multi-scale features of the given image are wrapped as a guided feature for stable video diffusion generation. We naively train two motion adapters for the manual trajectories and the human landmarks individually since they both contain sparse information about the control. After training, the MOFA-Adapters in different domains can also work together for more controllable video generation. Project Page: https://myniuuu.github.io/MOFA_Video/
Forward citations
Cited by 6 Pith papers
-
Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry
Vid-CamEdit re-synthesizes monocular videos along user-defined camera paths by conditioning a video diffusion model on 2D flows derived from estimated 3D geometry, without training on multi-view video data.
-
GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control
GeoDrive conditions a frozen video diffusion model on a 3D-rendered version of the requested ego trajectory, cutting trajectory-following error by 42% versus Vista while using 99.7% less training data.
-
Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss
A training-free method that matches sparse-point inter-frame feature correlations to transfer reference motion to generated videos with improved temporal consistency.
-
TransPixeler: Advancing Text-to-Video Generation with Transparency
A LoRA-based adaptation of DiT video generators that jointly outputs aligned RGB and alpha channels via extra tokens, shared positions, and attention masking.
-
ATI: Any Trajectory Instruction for Controllable Video Generation
ATI injects user-drawn point trajectories as soft Gaussian feature masks into a pretrained image-to-video diffusion model, enabling unified camera, object, and local motion control.
-
PhysAnimator: Physics-Guided Generative Cartoon Animation
PhysAnimator combines 2D deformable-body physics simulation with a sketch-guided video diffusion model to animate static anime illustrations with controllable, physically plausible motion.
Discussion (0). Continue with ORCID to comment.