REVIEW 5 cited by
Video Diffusion Models are Training-free Motion Interpreter and Controller
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Video generation primarily aims to model authentic and customized motion across frames, making understanding and controlling the motion a crucial topic. Most diffusion-based studies on video motion focus on motion customization with training-based paradigms, which, however, demands substantial training resources and necessitates retraining for diverse models. Crucially, these approaches do not explore how video diffusion models encode cross-frame motion information in their features, lacking interpretability and transparency in their effectiveness. To answer this question, this paper introduces a novel perspective to understand, localize, and manipulate motion-aware features in video diffusion models. Through analysis using Principal Component Analysis (PCA), our work discloses that robust motion-aware feature already exists in video diffusion models. We present a new MOtion FeaTure (MOFT) by eliminating content correlation information and filtering motion channels. MOFT provides a distinct set of benefits, including the ability to encode comprehensive motion information with clear interpretability, extraction without the need for training, and generalizability across diverse architectures. Leveraging MOFT, we propose a novel training-free video motion control framework. Our method demonstrates competitive performance in generating natural and faithful motion, providing architecture-agnostic insights and applicability in a variety of downstream tasks.
Forward citations
Cited by 5 Pith papers
-
EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation
EmoWorld adds three training-free steering operators to a frozen video diffusion transformer that separately control atmosphere, affect-bearing cues, and temporal emotion transitions in generated videos.
-
PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attention
PostCam generates new videos from a reference video along user-specified camera trajectories using a query-shared cross-attention that fuses pose data and rendered frames, improving control precision and detail preservation.
-
AnyI2V: Animating Any Conditional Image with Motion Control
AnyI2V animates arbitrary conditional images with user-defined trajectories by injecting debiased diffusion features and aligning attention queries across frames, without training.
-
LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer
LMP transfers motion from a reference video to newly generated videos in text-to-video and image-to-video settings without training, using attention maps in a frozen diffusion transformer.
-
Causal-Entity Reflected Egocentric Traffic Accident Video Synthesis
Driver gaze and accident-reason text are used to train a video diffusion model that can edit and generate egocentric crash videos with the correct causal participants, with a new large gaze dataset for accidents.
Discussion (0). Sign in to comment.