Pith. sign in

REVIEW 6 cited by

TrackGo: A Flexible and Efficient Method for Controllable Video Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2408.11475 v3 pith:M6JD5YRZ submitted 2024-08-21 cs.CV

Haitao Zhou , Chuang Wang , Rui Nie , Jinlin Liu , Dongdong Yu , Qian Yu , Changhu Wang This is my paper · ORCID
classification cs.CV
keywords videogenerationapproachcontrolcontrollableefficientflexiblelayers
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Recent years have seen substantial progress in diffusion-based controllable video generation. However, achieving precise control in complex scenarios, including fine-grained object parts, sophisticated motion trajectories, and coherent background movement, remains a challenge. In this paper, we introduce TrackGo, a novel approach that leverages free-form masks and arrows for conditional video generation. This method offers users with a flexible and precise mechanism for manipulating video content. We also propose the TrackAdapter for control implementation, an efficient and lightweight adapter designed to be seamlessly integrated into the temporal self-attention layers of a pretrained video generation model. This design leverages our observation that the attention map of these layers can accurately activate regions corresponding to motion in videos. Our experimental results demonstrate that our new approach, enhanced by the TrackAdapter, achieves state-of-the-art performance on key metrics such as FVD, FID, and ObjMC scores.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AnyI2V: Animating Any Conditional Image with Motion Control

    cs.CV 2025-07 conditional novelty 6.0 of 10

    AnyI2V animates arbitrary conditional images with user-defined trajectories by injecting debiased diffusion features and aligning attention queries across frames, without training.

  2. VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation

    cs.CV 2025-02 conditional novelty 6.0 of 10

    VidCRAFT3 is a single image-to-video diffusion system that accepts camera, object, and lighting direction controls separately or jointly, trained in three stages with a new synthetic lighting dataset.

  3. LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis

    cs.CV 2024-12 conditional novelty 6.0 of 10

    LeviTor controls 3D object trajectories in generated videos by feeding K-means clustered mask points with estimated depth into a video diffusion model.

  4. AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers

    cs.CV 2024-11 conditional novelty 6.0 of 10

    AC3D improves camera control in video diffusion transformers by conditioning only early denoising steps and the first 8 of 32 blocks, and by adding 20K static-camera dynamic videos to training.

  5. Motion Prompting: Controlling Video Generation with Motion Trajectories

    cs.CV 2024-12 conditional novelty 5.0 of 10

    A single-stage ControlNet on the Lumiere video model, conditioned only on dense point tracks, generalizes to sparse and dense trajectory control for object, camera, and transferred motions.

  6. CPA: Camera-pose-awareness Diffusion Transformer for Video Generation

    cs.CV 2024-12 conditional novelty 5.0 of 10

    CPA adds camera pose control to OpenSora-style diffusion video models by encoding a sparse Plücker motion field into a latent and injecting it into temporal attention layers.

Pith tools