REVIEW 16 cited by
FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Diffusion model has demonstrated remarkable capability in video generation, which further sparks interest in introducing trajectory control into the generation process. While existing works mainly focus on training-based methods (e.g., conditional adapter), we argue that diffusion model itself allows decent control over the generated content without requiring any training. In this study, we introduce a tuning-free framework to achieve trajectory-controllable video generation, by imposing guidance on both noise construction and attention computation. Specifically, 1) we first show several instructive phenomenons and analyze how initial noises influence the motion trajectory of generated content. 2) Subsequently, we propose FreeTraj, a tuning-free approach that enables trajectory control by modifying noise sampling and attention mechanisms. 3) Furthermore, we extend FreeTraj to facilitate longer and larger video generation with controllable trajectories. Equipped with these designs, users have the flexibility to provide trajectories manually or opt for trajectories automatically generated by the LLM trajectory planner. Extensive experiments validate the efficacy of our approach in enhancing the trajectory controllability of video diffusion models.
Forward citations
Cited by 16 Pith papers
-
QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers
QWERTY enables training-free motion control in pretrained image-to-video DiTs by warping the frame-invariant semantic subspace of queries in 3D full attention and using the predicted noise as self-guidance for latent ...
-
TrajLoc: Trajectory-Attention Localization for Multi-Object Motion Control
TrajLoc enforces per-object trajectory constraints in I2V generation via attention-layer Gaussian heatmap substitution, yielding +4.3 dB PSNR and 51% lower endpoint error on datasets with up to 20 objects across two b...
-
CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration
CoMoGen generates controllable interactive video from mask sequences and images by encoding masks into MMDiT via MaskAdapter and LoRA on motion layers, claiming SOTA motion fidelity.
-
Robot-Factored World Models via Robot Rendering
Conditioning a video world model on rendered nominal robot trajectories (URDF mesh + depth) instead of raw actions or logged future states improves action-following and enables zero-shot embodiment change.
-
Go-with-the-Track: Video Compositing and Motion Control with Point Tracking
Go-with-the-Track unifies multi-reference compositing and point-track motion control in video diffusion by using spatially-aware point-track embeddings injected via an adapter.
-
NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment
NTRK uses a whitening operator to tilt the noise term in diffusion reverse kernels for reward guidance, outperforming baselines with 20x fewer steps on aesthetic tasks.
-
NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment
NTRK is a reward-guided diffusion sampler that uses a whitening operator to bias the noise term toward high-reward outcomes, outperforming baselines with up to 20x fewer sampling steps on aesthetic tasks.
-
EverAnimate: Minute-Scale Human Animation via Latent Flow Restoration
EverAnimate restores drifted latent flow trajectories in chunked video generation via persistent latent propagation and restorative flow matching, achieving measurable gains in PSNR, SSIM, LPIPS, and FID over prior lo...
-
SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation
SymphoMotion jointly controls camera trajectories and depth-aware object dynamics inside one video diffusion model, supported by the new RealCOD-25K real-world paired-motion dataset.
-
HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesis
HVG-3D uses a 3D-aware diffusion architecture with ControlNet to synthesize high-fidelity hand-object interaction videos from 3D control signals, achieving state-of-the-art spatial fidelity and temporal coherence on t...
-
Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints
Sparse 3D hand joints plus an occlusion-aware control module produce higher-fidelity, 3D-consistent egocentric hand-object videos than dense-2D or implicit-pose baselines.
-
Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints
A new occlusion-aware control module generates high-fidelity egocentric videos from sparse 3D hand joints, supported by a million-clip dataset and cross-embodiment benchmark.
-
MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization
MotionAdapter transfers reference video motions into target videos inside DiT diffusion models by isolating attention-derived motion fields and refining them via DINO-guided semantic alignment.
-
Streaming Drag-Oriented Interactive Video Manipulation: Drag Anything, Anytime!
DragStream enables real-time drag, deform, and rotate edits on autoregressively generated videos without retraining, by correcting latent drift and selectively filtering context features.
-
From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
Physical intelligence needs an embodied brain that reasons over interventions and emits capability requests, grounded by a physical harness and shared experience contracts rather than direct actuator policies.
-
Evolution of Video Generative Foundations
This survey traces video generation technology from GANs to diffusion models and then to autoregressive and multimodal approaches while analyzing principles, strengths, and future trends.
Discussion (0). Sign in to comment.