REVIEW 27 cited by
Boximator: Generating Rich and Controllable Motions for Video Synthesis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Generating rich and controllable motion is a pivotal challenge in video synthesis. We propose Boximator, a new approach for fine-grained motion control. Boximator introduces two constraint types: hard box and soft box. Users select objects in the conditional frame using hard boxes and then use either type of boxes to roughly or rigorously define the object's position, shape, or motion path in future frames. Boximator functions as a plug-in for existing video diffusion models. Its training process preserves the base model's knowledge by freezing the original weights and training only the control module. To address training challenges, we introduce a novel self-tracking technique that greatly simplifies the learning of box-object correlations. Empirically, Boximator achieves state-of-the-art video quality (FVD) scores, improving on two base models, and further enhanced after incorporating box constraints. Its robust motion controllability is validated by drastic increases in the bounding box alignment metric. Human evaluation also shows that users favor Boximator generation results over the base model.
Forward citations
Cited by 27 Pith papers
-
ActionParty: Multi-Subject Action Binding in Generative Video Games
ActionParty binds discrete actions to individual subjects in a single generated video by jointly modeling subject state tokens and video latents, controlling up to seven players across 46 Melting Pot games.
-
Video Creation by Demonstration
A self-supervised diffusion approach, δ-Diffusion, transfers action concepts from a demonstration video to a new context image using appearance-bottlenecked action latents.
-
FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control
A single video-diffusion framework composites both static images and dynamic footage along user-defined trajectories by transporting canonical foreground latents directly into the background latent sequence.
-
LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models
Fine-tuning CogVideoX with autoregressive context management and bidirectional alignment enables a single model to perform event-based video reconstruction, prediction, and zero-shot interpolation with superior tempor...
-
Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation
UniCaMo builds 3D-grounded motion-consistent input noise from sparse 3D tracks and sphere-sampled noise so pretrained video diffusion models jointly control object and camera motion without architectural changes.
-
SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation
SymphoMotion jointly controls camera trajectories and depth-aware object dynamics inside one video diffusion model, supported by the new RealCOD-25K real-world paired-motion dataset.
-
Precise Action-to-Video Generation Through Visual Action Prompts
Skeleton-based visual action prompts give precise, cross-domain action control for video generation of human and robot interactions.
-
AnyI2V: Animating Any Conditional Image with Motion Control
AnyI2V animates arbitrary conditional images with user-defined trajectories by injecting debiased diffusion features and aligning attention queries across frames, without training.
-
MOVi: Training-free Text-conditioned Multi-Object Video Generation
MOVi improves multi-object video generation without retraining by using LLM-planned trajectories to reinitialize the diffusion noise and by reweighting attention to stop objects from mixing together.
-
IKMo: Image-Keyframed Motion Generation with Trajectory-Pose Conditioned Motion Diffusion Model
A motion diffusion model with decoupled trajectory and keyframe-pose control, wrapped in an MLLM agent system, produces more controllable 3D human motion from images and text.
-
MotionPro: A Precise Motion Controller for Image-to-Video Generation
MotionPro uses region-wise trajectories and a motion mask to control object and camera motion in image-to-video generation, reporting improved trajectory alignment over prior methods.
-
AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Models
A diffusion model animates a character into arbitrary dynamic backgrounds by conditioning on a rendered 3D-avatar video, reframing open-domain animation as a restoration problem.
-
Hybrid Neural-MPM for Interactive Fluid Simulations in Real-Time
A hybrid neural-MPM solver with a chaos-triggered fallback and a diffusion-based sketch controller enables real-time interactive fluid simulation with user control.
-
3D-Fixup: Advancing Photo Editing with 3D Priors
3D-Fixup performs 3D-aware photo editing, such as object rotation and translation, by conditioning a diffusion model on guidance rendered from a reconstructed 3D mesh.
-
VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation
VidCRAFT3 is a single image-to-video diffusion system that accepts camera, object, and lighting direction controls separately or jointly, trained in three stages with a new synthetic lighting dataset.
-
Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss
A training-free method that matches sparse-point inter-frame feature correlations to transfer reference motion to generated videos with improved temporal consistency.
-
Llama Learns to Direct: DirectorLLM for Human-Centric Video Generation
A fine-tuned Llama 3 LLM writes discrete human pose tokens from a text prompt, and a pose-conditioned diffusion video renderer turns them into videos, improving human motion fidelity.
-
MotionBridge: Dynamic Video Inbetweening with Flexible Controls
MotionBridge generates interpolated video frames between two images while following user-supplied trajectory, mask, keyframe, guide-pixel, and text controls.
-
OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation
A drag-style motion control method for 360 degree image-to-video generation, built on spherical trajectory estimation and joint fine-tuning of a pretrained video diffusion model.
-
ObjCtrl-2.5D: Training-free Object Control with Camera Poses
A training-free method that lifts 2D object trajectories into camera poses with depth and uses a frozen camera-control video model to achieve more accurate and 3D-aware object motion, including rotation.
-
Mind the Time: Temporally-Controlled Multi-Event Video Generation
MinT generates multi-event videos where each event's timing is controlled by the user, using a fine-tuned video diffusion transformer with a time-aware rotary position embedding.
-
AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers
AC3D improves camera control in video diffusion transformers by conditioning only early denoising steps and the first 8 of 32 blocks, and by adding 20K static-camera dynamic videos to training.
-
ROICtrl: Boosting Instance Control for Visual Generation
ROICtrl uses a new ROI-Unpool operation to inject per-instance captions and boxes into diffusion models, achieving more accurate multi-instance generation more efficiently than prior adapters.
-
Is Energy Guidance All You Need? Training-Free Norm Injection for Driving World Models
Sampling-time energy guidance steers a frozen rectified-flow driving world model's ego trajectory to a braking target, but the generated video does not follow under current joint self-attention.
-
ATI: Any Trajectory Instruction for Controllable Video Generation
ATI injects user-drawn point trajectories as soft Gaussian feature masks into a pretrained image-to-video diffusion model, enabling unified camera, object, and local motion control.
-
From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
Physical intelligence needs an embodied brain that reasons over interventions and emits capability requests, grounded by a physical harness and shared experience contracts rather than direct actuator policies.
-
Video Diffusion Transformers are In-Context Learners
Concatenating multiple video clips into one input and fine-tuning a LoRA adapter lets a pretrained video diffusion transformer produce consistent multi-scene videos from a single prompt.
Discussion (0). Continue with ORCID to comment.