Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

One model can steer both camera paths and 3D object motion so generated videos stay spatially coherent.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 12:25 UTC pith:665YNAMY

load-bearing objection Solid systems paper: real paired-motion dataset + practical dual 3D-aware control that beats the usual 2D baselines; the Box-IoU headline is softer than it looks because labels and scoring share the same monocular estimators. the 2 major comments →

arxiv 2604.03723 v4 pith:665YNAMY submitted 2026-04-04 cs.CV

SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation

Guiyu Zhang , Yabo Chen , Xunzhi Xiang , Junchao Huang , Zhongyu Wang , Li Jiang This is my paper
classification cs.CV
keywords video generationcamera controlobject dynamics3D trajectoriesdiffusion modelsmotion controllabilityRealCOD-25K
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Video generators usually control either the camera or the objects, and when they try both they often mix true object motion with camera-induced parallax on the 2D image plane. SymphoMotion claims that a single diffusion model can jointly govern both if camera paths are injected with geometry-aware point-cloud cues and object motion is driven by both 2D visual anchors and 3D trajectory embeddings. The authors also release RealCOD-25K, a real-world collection of 25K clips that pair camera poses with object-level 3D trajectories, filling a data gap that previously forced methods onto synthetic or single-motion datasets. Experiments and user studies report higher visual fidelity, tighter camera accuracy, and roughly double the object-trajectory Box-IoU of prior joint controllers. A sympathetic reader cares because coherent joint control is what filmmakers and interactive tools actually need: synchronized viewpoint changes and object paths that remain consistent under depth and parallax.

Core claim

SymphoMotion shows that camera trajectories and object dynamics can be jointly controlled inside one video diffusion model by pairing explicit camera-path conditioning with geometry-aware renderings for the camera branch and combining projected 2D boxes with 3D trajectory embeddings for the object branch, trained on real paired annotations rather than synthetic 6-DoF data.

What carries the argument

Two complementary modules—Camera Trajectory Control (CTC), which fuses Plücker camera encodings with rendered point-cloud frames via a Viewpoint Control Module, and Object Dynamics Control (ODC), which injects 3D trajectory embeddings plus 2D visual boxes through an Object Motion Module—together supply disentangled, depth-aware motion signals to a frozen base image-to-video diffusion model.

Load-bearing premise

The automatically recovered 3D trajectories and point-cloud geometry used for both training labels and inference must be accurate enough that the model learns true 3D motion rather than residual projection or depth noise.

What would settle it

On held-out real videos whose camera poses and object 3D paths are measured by an independent high-accuracy tracker or multi-view capture, measure whether SymphoMotion’s Box-IoU and camera-error metrics stay near the reported levels; a large drop under cleaner geometry would falsify the claim that the controller follows true 3D motion.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes SymphoMotion, a unified diffusion framework (built on frozen Wan-I2V) for joint control of camera trajectories and object dynamics in image-to-video generation. Camera Trajectory Control (CTC) injects Plücker embeddings of target poses together with geometry-aware point-cloud renderings (from Depth-Pro) via a Viewpoint Control Module implemented as ControlNet. Object Dynamics Control (ODC) supplies 2D visual guidance by projecting 3D trajectories to per-frame boxes overlaid on the renders, plus 3D trajectory embeddings fused with entity text and injected by cross-attention in an Object Motion Module. To train and evaluate, the authors construct RealCOD-25K (~25K real clips) with paired camera poses and object-level 3D trajectories obtained via a pipeline of SegAnyMo, MegaSAM + Depth-Anything V2 + UniDepth V2, SpatialTrackerV2, and Qwen-2.5-VL captions. On a self-curated 100-video test set they report superior FID/FVD/CLIPSIM, lower CamTransErr/CamRotErr, higher Box-IoU (61.88 vs MotionCtrl 31.42), and better user-study scores, with ablations confirming the value of point-cloud priors, 2D boxes, and 3D trajectories.

Significance. If the claims hold, the work is a solid incremental advance for controllable video generation: it cleanly disentangles camera-induced parallax from object motion via explicit 3D cues, supplies a large real-world paired dataset that fills a clear gap left by static-camera or synthetic-only resources, and releases code and data. The two-stage training recipe, interactive 3D editing interface, and consistent quantitative/qualitative gains over strong camera-only and MotionCtrl baselines make the result useful to the community even if absolute 3D fidelity remains imperfect. Public availability of RealCOD-25K is itself a lasting contribution.

major comments (2)
  1. Section 5.1 and Table 1: The headline object-motion claim rests on Box-IoU (61.88 vs 31.42). Training trajectories (Sec. 4.2) are recovered by the same family of monocular depth + tracking estimators (MegaSAM, Depth-Anything V2, UniDepth V2, SpatialTrackerV2) later used, via SAM2, to produce evaluation boxes. No independent multi-view, stereo, or MoCap 3D ground truth is reported for either the labels or the generated videos. Consequently the large IoU gap may partly measure improved imitation of the annotation pipeline’s own depth/parallax artifacts rather than genuine depth-aware 3D fidelity. This is load-bearing for the abstract’s “depth-aware, spatially coherent object manipulation” claim; an error analysis of the annotation pipeline or evaluation on a small held-out set with higher-quality 3D GT is needed.
  2. Section 5.2 and Table 1: Joint-control comparisons are limited to MotionCtrl. Related-work methods that also attempt simultaneous camera+object control (FMC, Perception-as-Control, ATI, VidCraft3) are discussed but never run head-to-head, even though the authors criticize their 2D or synthetic limitations. Without those numbers the claim that SymphoMotion “significantly outperforms existing methods” in the unified setting is only partially supported.
minor comments (4)
  1. Section 3.4 / Fig. 2: The two-stage schedule freezes CTC before training ODC; a short joint fine-tuning ablation (or statement that it was tried and failed) would clarify whether residual interference remains.
  2. Section 5.1: The 100-video test set is self-curated; release of the list (or statistics on scene diversity, motion amplitude, depth range) would improve reproducibility and allow others to check selection bias.
  3. Fig. 7 and Table 3: Ablation rows are clear, but the qualitative examples would be more convincing if the same camera trajectory and object path were shown side-by-side for every ablated variant rather than different scenes.
  4. Notation: N_p (points per object) and the exact temporal down-sampling factor inside the trajectory encoder are never given numerical values; a short hyper-parameter table would help.

Circularity Check

0 steps flagged

No circularity: empirical method paper whose performance claims rest on held-out metrics external to the training loss, not on by-construction reductions or load-bearing self-citations.

full rationale

SymphoMotion is a standard supervised video-diffusion architecture paper. Camera Trajectory Control and Object Dynamics Control are defined by explicit conditioning modules (Plücker + point-cloud renderings into VCM; 2D boxes + 3D trajectory embeddings into OMM) whose training objectives (Eqs. 4 and 7) simply minimize velocity prediction error under those conditionings. RealCOD-25K supplies the paired labels via an off-the-shelf annotation pipeline; the model is trained to match them and is then scored on a separately curated 100-video set with FID/FVD/CLIPSIM, CamTransErr/CamRotErr, Box-IoU (via SAM2 from first-frame masks), and user studies. None of these metrics is algebraically forced by the loss or by a fitted free parameter; none of the central claims reduces to a self-citation uniqueness theorem or an ansatz imported from the authors’ prior work. The skeptic’s concern that monocular depth/tracking noise is shared between training labels and Box-IoU is a validity/correctness issue about label quality, not a circular derivation step of the kind the analyzer is charged to detect. Consequently the derivation chain is self-contained and non-circular.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

As an empirical systems paper the load-bearing content is architectural choices, training schedule, and the assumption that off-the-shelf 3D estimators supply usable supervision. Free parameters are the usual optimizer and schedule knobs; axioms are standard diffusion and geometry-estimation assumptions; invented entities are the named modules and the dataset.

free parameters (4)
  • learning rate = 1e-5
    Warmed to 1e-5 then held constant; chosen by authors, not derived.
  • two-stage training schedule = CTC then ODC
    CTC trained first, then frozen while ODC is trained; order and freeze decision are design choices that affect final controllability.
  • video length / resolution = 81 frames, 832x480
    81 frames at 832×480 fixed for all training; arbitrary but load-bearing for reported metrics.
  • number of sampled 3D points per object Np
    Trajectory representation dimension chosen by authors; not ablated exhaustively.
axioms (4)
  • domain assumption Pretrained Wan-I2V DiT + Flow Matching is a sufficiently strong frozen backbone for adding motion control adapters.
    Section 3.1 and 3.4; entire method freezes the base model.
  • domain assumption Monocular depth and tracking pipelines (Depth-Pro, MegaSAM, SpatialTrackerV2, Depth-Anything V2, UniDepth V2) produce 3D trajectories accurate enough to serve as ground-truth supervision and inference conditioning.
    Sections 3.2, 3.5, 4.2; if the recovered geometry is systematically biased the controller learns the bias.
  • ad hoc to paper Projecting 3D trajectories to 2D boxes and rendering them onto point-cloud frames supplies unambiguous visual guidance that disentangles object motion from camera parallax.
    Section 3.3; core design claim of ODC.
  • domain assumption Manual aesthetic/OCR/motion filtering plus 120 person-hours of researcher review yields a clean, unbiased RealCOD-25K distribution.
    Section 4.1 curation pipeline.
invented entities (3)
  • Camera Trajectory Control (CTC) + Viewpoint Control Module (VCM) no independent evidence
    purpose: Inject Plücker camera embeddings fused with point-cloud renderings via a ControlNet-style module.
    Named mechanism introduced in Section 3.2; no independent existence outside the paper.
  • Object Dynamics Control (ODC) + Object Motion Module (OMM) no independent evidence
    purpose: Fuse 2D rendered boxes with 3D trajectory embeddings via cross-attention for depth-aware object motion.
    Named mechanism introduced in Section 3.3.
  • RealCOD-25K dataset no independent evidence
    purpose: Provide the first large real-world paired camera-pose + object-level 3D trajectory annotations for training and benchmarking.
    Constructed in Section 4; claimed public but its quality is internal to the paper’s pipeline.

pith-pipeline@v1.1.0-grok45 · 21126 in / 3329 out tokens · 37693 ms · 2026-07-13T12:25:53.894662+00:00 · methodology

0 comments
read the original abstract

Controlling both camera motion and object dynamics is essential for coherent and expressive video generation, yet current methods typically handle only one motion type or rely on ambiguous 2D cues that entangle camera-induced parallax with true object movement. We present SymphoMotion, a unified motion-control framework that jointly governs camera trajectories and object dynamics within a single model. SymphoMotion features a Camera Trajectory Control mechanism that integrates explicit camera paths with geometry-aware cues to ensure stable, structurally consistent viewpoint transitions, and an Object Dynamics Control mechanism that combines 2D visual guidance with 3D trajectory embeddings to enable depth-aware, spatially coherent object manipulation. To support large-scale training and evaluation, we further construct RealCOD-25K, a comprehensive real-world dataset containing paired camera poses and object-level 3D trajectories across diverse indoor and outdoor scenes, addressing a key data gap in unified motion control. Extensive experiments and user studies show that SymphoMotion significantly outperforms existing methods in visual fidelity, camera controllability, and object-motion accuracy, establishing a new benchmark for unified motion control in video generation. Codes and data are publicly available at https://grenoble-zhang.github.io/SymphoMotion/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling

    cs.CV 2026-07 conditional novelty 6.0

    Inference-time manipulation of RoPE, attention masks, per-shot conditioning, and VAE decoding lets frozen text-to-video models produce reference-controlled multi-shot long videos.

Reference graph

Works this paper leans on

62 extracted references · 30 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Vd3d: Taming large video diffu- sion transformers for 3d camera control.arXiv preprint arXiv:2407.12781, 2024

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siaro- hin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffu- sion transformers for 3d camera control.arXiv preprint arXiv:2407.12781, 2024. 2, 3

    Pith/arXiv arXiv 2024

  2. [2]

    Ac3d: Analyzing and improving 3d camera control in video diffusion trans- formers

    Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Ali- aksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion trans- formers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 4

    2025

  3. [3]

    Ac3d: Analyzing and improving 3d camera control in video diffusion trans- formers

    Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Ali- aksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion trans- formers. InProceedings of the Computer Vision and Pattern Recognition Conference, 2025. 2, 3

    2025

  4. [4]

    Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints.arXiv preprint arXiv:2412.07760, 2024

    Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Xiao Fu, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di Zhang. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints.arXiv preprint arXiv:2412.07760, 2024. 3

    Pith/arXiv arXiv 2024

  5. [5]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 6, 7

    Pith/arXiv arXiv 2025

  6. [6]

    Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024

    Aleksei Bochkovskii, AmaãG , l Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024. 4, 6

    Pith/arXiv arXiv 2024

  7. [7]

    Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation

    Chenjie Cao, Jingkai Zhou, Shikai Li, Jingyun Liang, Chao- hui Yu, Fan Wang, Xiangyang Xue, and Yanwei Fu. Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation. InProceedings of the SIG- GRAPH Asia 2025 Conference Papers, pages 1–12, 2025. 3, 7

    2025

  8. [8]

    Perception-as-control: Fine-grained control- lable image animation with 3d-aware motion representation

    Yingjie Chen, Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Perception-as-control: Fine-grained control- lable image animation with 3d-aware motion representation. arXiv preprint arXiv:2501.05020, 2025. 2, 3

    Pith/arXiv arXiv 2025

  9. [9]

    Wan-move: Motion- controllable video generation via latent trajectory guidance

    Ruihang Chu, Yefei He, Zhekai Chen, Shiwei Zhang, Xi- aogang Xu, Bin Xia, Dingdong Wang, Hongwei Yi, Xi- hui Liu, Hengshuang Zhao, et al. Wan-move: Motion- controllable video generation via latent trajectory guidance. arXiv preprint arXiv:2512.08765, 2025. 2

    arXiv 2025

  10. [10]

    Unimax: Fairer and more effective language sampling for large-scale multilingual pretraining.arXiv preprint arXiv:2304.09151,

    Hyung Won Chung, Noah Constant, Xavier Garcia, Adam Roberts, Yi Tay, Sharan Narang, and Orhan Firat. Unimax: Fairer and more effective language sampling for large-scale multilingual pretraining.arXiv preprint arXiv:2304.09151,

    Pith/arXiv arXiv

  11. [11]

    I2vcontrol-camera: Precise video camera control with adjustable motion strength.arXiv preprint arXiv:2411.06525, 2024

    Wanquan Feng, Jiawei Liu, Pengqi Tu, Tianhao Qi, Mingzhen Sun, Tianxiang Ma, Songtao Zhao, Siyu Zhou, and Qian He. I2vcontrol-camera: Precise video camera control with adjustable motion strength.arXiv preprint arXiv:2411.06525, 2024. 2, 3

    Pith/arXiv arXiv 2024

  12. [12]

    3dtrajmaster: Mastering 3d trajectory for multi-entity motion in video generation.arXiv preprint arXiv:2412.07759, 2024

    Xiao Fu, Xian Liu, Xintao Wang, Sida Peng, Menghan Xia, Xiaoyu Shi, Ziyang Yuan, Pengfei Wan, Di Zhang, and Dahua Lin. 3dtrajmaster: Mastering 3d trajectory for multi-entity motion in video generation.arXiv preprint arXiv:2412.07759, 2024. 2, 3, 7

    Pith/arXiv arXiv 2024

  13. [13]

    Motion prompting: Controlling video generation with motion trajec- tories

    Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Yusuf Aytar, Michael Rubinstein, Chen Sun, et al. Motion prompting: Controlling video generation with motion trajec- tories. InProceedings of the Computer Vision and Pattern Recognition Conference, 2025. 2, 3

    2025

  14. [14]

    Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024. 2, 3, 7

    Pith/arXiv arXiv 2024

  15. [15]

    Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models.arXiv preprint arXiv:2503.10592, 2025

    Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models.arXiv preprint arXiv:2503.10592, 2025. 3

    Pith/arXiv arXiv 2025

  16. [16]

    Training-free camera control for video generation.arXiv preprint arXiv:2406.10126, 2024

    Chen Hou and Zhibo Chen. Training-free camera control for video generation.arXiv preprint arXiv:2406.10126, 2024. 2, 3

    Pith/arXiv arXiv 2024

  17. [17]

    Mo- tionmaster: Training-free camera motion transfer for video generation.arXiv preprint arXiv:2404.15789, 2024

    Teng Hu, Jiangning Zhang, Ran Yi, Yating Wang, Hongrui Huang, Jieyu Weng, Yabiao Wang, and Lizhuang Ma. Mo- tionmaster: Training-free camera motion transfer for video generation.arXiv preprint arXiv:2404.15789, 2024. 2

    Pith/arXiv arXiv 2024

  18. [18]

    Live: Long- horizon interactive video world modeling.arXiv preprint arXiv:2602.03747, 2026

    Junchao Huang, Ziyang Ye, Xinting Hu, Tianyu He, Guiyu Zhang, Shaoshuai Shi, Jiang Bian, and Li Jiang. Live: Long- horizon interactive video world modeling.arXiv preprint arXiv:2602.03747, 2026. 2

    arXiv 2026

  19. [19]

    Segment any motion in videos

    Nan Huang, Wenzhao Zheng, Chenfeng Xu, Kurt Keutzer, Shanghang Zhang, Angjoo Kanazawa, and Qianqian Wang. Segment any motion in videos. InProceedings of the Com- puter Vision and Pattern Recognition Conference, 2025. 6

    2025

  20. [20]

    Peekaboo: Interactive video generation via masked- diffusion

    Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peekaboo: Interactive video generation via masked- diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2, 3

    2024

  21. [21]

    Collab- orative video diffusion: Consistent multi-video generation with camera control.Advances in Neural Information Pro- cessing Systems, 2024

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hong- sheng Li, Leonidas J Guibas, and Gordon Wetzstein. Collab- orative video diffusion: Consistent multi-video generation with camera control.Advances in Neural Information Pro- cessing Systems, 2024. 3

    2024

  22. [22]

    Magicmotion: Controllable video genera- tion with dense-to-sparse trajectory guidance.arXiv preprint arXiv:2503.16421, 2025

    Quanhao Li, Zhen Xing, Rui Wang, Hui Zhang, Qi Dai, and Zuxuan Wu. Magicmotion: Controllable video genera- tion with dense-to-sparse trajectory guidance.arXiv preprint arXiv:2503.16421, 2025. 2, 3, 7

    arXiv 2025

  23. [23]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023. 3

    2023

  24. [24]

    Image conductor: Precision control for interactive video syn- thesis

    Yaowei Li, Xintao Wang, Zhaoyang Zhang, Zhouxia Wang, Ziyang Yuan, Liangbin Xie, Ying Shan, and Yuexian Zou. Image conductor: Precision control for interactive video syn- thesis. InProceedings of the AAAI Conference on Artificial Intelligence, 2025. 3

    2025

  25. [25]

    Toward a practical perceptual video quality metric, 2016.Dostupno na: http://techblog

    Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. Toward a practical perceptual video quality metric, 2016.Dostupno na: http://techblog. netflix. com/2016/06/toward-practical-perceptual-video. html [16.8. 2022.], 2016. 6

    2016

  26. [26]

    Megasam: Accurate, fast and ro- bust structure and motion from casual dynamic videos

    Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holyn- ski, and Noah Snavely. Megasam: Accurate, fast and ro- bust structure and motion from casual dynamic videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10486–10496, 2025. 6

    2025

  27. [27]

    Wonderland: Nav- igating 3d scenes from a single image.arXiv preprint arXiv:2412.12091, 2024

    Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos N Pla- taniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Nav- igating 3d scenes from a single image.arXiv preprint arXiv:2412.12091, 2024. 4

    Pith/arXiv arXiv 2024

  28. [28]

    Real-time scene text detection with differentiable binarization and adaptive scale fusion.IEEE transactions on pattern analysis and machine intelligence, 2022

    Minghui Liao, Zhisheng Zou, Zhaoyi Wan, Cong Yao, and Xiang Bai. Real-time scene text detection with differentiable binarization and adaptive scale fusion.IEEE transactions on pattern analysis and machine intelligence, 2022. 6

    2022

  29. [29]

    Motionclone: Training-free motion cloning for controllable video generation.arXiv preprint arXiv:2406.05338, 2024

    Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, and Yi Jin. Motionclone: Training-free motion cloning for controllable video generation.arXiv preprint arXiv:2406.05338, 2024. 2

    Pith/arXiv arXiv 2024

  30. [30]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 4

    Pith/arXiv arXiv 2022

  31. [31]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 6

    Pith/arXiv arXiv 2017

  32. [32]

    Snap video: Scaled spatiotemporal transformers for text-to-video synthesis

    Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang, Aleksei Stoliar, Elisa Ricci, Jian Ren, et al. Snap video: Scaled spatiotemporal transformers for text-to-video synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 3

    2024

  33. [33]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 4195–4205,

  34. [34]

    Unidepthv2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110, 2025

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mat- tia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. Unidepthv2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110, 2025. 7

    Pith/arXiv arXiv 2025

  35. [35]

    Freetraj: Tuning-free tra- jectory control in video diffusion models.arXiv preprint arXiv:2406.16863, 2024

    Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia, and Ziwei Liu. Freetraj: Tuning-free tra- jectory control in video diffusion models.arXiv preprint arXiv:2406.16863, 2024. 3

    Pith/arXiv arXiv 2024

  36. [36]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational conference on machine learning. PmLR, 2021. 4

    2021

  37. [37]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024. 6, 7

    Pith/arXiv arXiv 2024

  38. [38]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 4

    2022

  39. [39]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III

    2015

  40. [40]

    pytorch-fid: Fid score for pytorch

    Maximilian Seitzer. pytorch-fid: Fid score for pytorch. https://github.com/ mseitzer/pytorch-fid, 2020. 7

    2020

  41. [41]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. InACM SIGGRAPH 2024 Conference Papers,

    2024

  42. [42]

    Free-form motion control: Controlling the 6d poses of camera and objects in video gen- eration

    Xincheng Shuai, Henghui Ding, Zhenyuan Qin, Hao Luo, Xingjun Ma, and Dacheng Tao. Free-form motion control: Controlling the 6d poses of camera and objects in video gen- eration. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, 2025. 2, 3, 7

    2025

  43. [43]

    Light field networks: Neural scene representations with single-evaluation rendering.Ad- vances in Neural Information Processing Systems, 2021

    Vincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum, and Fredo Durand. Light field networks: Neural scene representations with single-evaluation rendering.Ad- vances in Neural Information Processing Systems, 2021. 3

    2021

  44. [44]

    To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018. 7

    Pith/arXiv arXiv 2018

  45. [45]

    Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025. 4, 5, 6

    Pith/arXiv arXiv 2025

  46. [46]

    Ati: Any trajectory in- struction for controllable video generation.arXiv preprint arXiv:2505.22944, 2025

    Angtian Wang, Haibin Huang, Jacob Zhiyuan Fang, Yid- ing Yang, and Chongyang Ma. Ati: Any trajectory in- struction for controllable video generation.arXiv preprint arXiv:2505.22944, 2025. 2, 3

    Pith/arXiv arXiv 2025

  47. [47]

    Levitor: 3d trajectory oriented image-to-video syn- thesis

    Hanlin Wang, Hao Ouyang, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Qifeng Chen, Yujun Shen, and Limin Wang. Levitor: 3d trajectory oriented image-to-video syn- thesis. InProceedings of the Computer Vision and Pattern Recognition Conference, 2025. 3

    2025

  48. [48]

    Boximator: Generat- ing rich and controllable motions for video synthesis.arXiv preprint arXiv:2402.01566, 2024

    Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guo- qiang Wei, Liping Yuan, and Hang Li. Boximator: Generat- ing rich and controllable motions for video synthesis.arXiv preprint arXiv:2402.01566, 2024. 3

    Pith/arXiv arXiv 2024

  49. [49]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Pa- pers, 2024. 2, 3, 6, 7

    2024

  50. [50]

    Godiva: Gen- erating open-domain videos from natural descriptions.arXiv preprint arXiv:2104.14806, 2021

    Chenfei Wu, Lun Huang, Qianxi Zhang, Binyang Li, Lei Ji, Fan Yang, Guillermo Sapiro, and Nan Duan. Godiva: Gen- erating open-domain videos from natural descriptions.arXiv preprint arXiv:2104.14806, 2021. 7

    Pith/arXiv arXiv 2021

  51. [51]

    Development of an image data set of construction machines for deep learning object detection.Journal of Computing in Civil Engineering, 2021

    Bo Xiao and Shih-Chung Kang. Development of an image data set of construction machines for deep learning object detection.Journal of Computing in Civil Engineering, 2021. 2

    2021

  52. [52]

    Spatialtrackerv2: 3d point tracking made easy.arXiv preprint arXiv:2507.12462, 2025

    Yuxi Xiao, Jianyuan Wang, Nan Xue, Nikita Karaev, Yuri Makarov, Bingyi Kang, Xing Zhu, Hujun Bao, Yujun Shen, and Xiaowei Zhou. Spatialtrackerv2: 3d point tracking made easy.arXiv preprint arXiv:2507.12462, 2025. 7

    Pith/arXiv arXiv 2025

  53. [53]

    Camco: Camera- controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. Camco: Camera- controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024. 2, 3

    Pith/arXiv arXiv 2024

  54. [54]

    Depth any- thing v2.Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2.Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2024. 7

    2024

  55. [55]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory.arXiv preprint arXiv:2308.08089, 2023

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory.arXiv preprint arXiv:2308.08089, 2023. 2, 3

    Pith/arXiv arXiv 2023

  56. [56]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024. 4, 7

    Pith/arXiv arXiv 2024

  57. [57]

    Proteus-id: Id- consistent and motion-coherent video customization

    Guiyu Zhang, Chen Shi, Zijian Jiang, Xunzhi Xiang, Jingjing Qian, Shaoshuai Shi, and Li Jiang. Proteus-id: Id- consistent and motion-coherent video customization. InPro- ceedings of the SIGGRAPH Asia 2025 Conference Papers, pages 1–11, 2025. 2

    2025

  58. [58]

    Tora: Trajectory-oriented diffusion transformer for video genera- tion

    Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video genera- tion. InProceedings of the Computer Vision and Pattern Recognition Conference, 2025. 2, 3

    2025

  59. [59]

    Vidcraft3: Camera, object, and lighting control for image-to-video generation

    Sixiao Zheng, Zimian Peng, Yanpeng Zhou, Yi Zhu, Hang Xu, Xiangru Huang, and Yanwei Fu. Vidcraft3: Camera, object, and lighting control for image-to-video generation. arXiv preprint arXiv:2502.07531, 2025. 3

    Pith/arXiv arXiv 2025

  60. [60]

    Trackgo: A flexible and efficient method for controllable video generation

    Haitao Zhou, Chuang Wang, Rui Nie, Jinlin Liu, Dongdong Yu, Qian Yu, and Changhu Wang. Trackgo: A flexible and efficient method for controllable video generation. InPro- ceedings of the AAAI Conference on Artificial Intelligence,

  61. [61]

    Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018. 2

    Pith/arXiv arXiv 2018

  62. [62]

    Omniworld: A multi-domain and multi-modal dataset for 4d world modeling.arXiv preprint arXiv:2509.12201, 2025

    Yang Zhou, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Haoyu Guo, Zizun Li, Kaijing Ma, Xinyue Li, Yating Wang, Haoyi Zhu, et al. Omniworld: A multi-domain and multi-modal dataset for 4d world modeling.arXiv preprint arXiv:2509.12201, 2025. 3

    arXiv 2025