Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Single-image 4D synthesis is reframed as joint prediction of geometry and dense point trajectories in one diffusion model.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:26 UTC pith:7JQ7MSFE

load-bearing objection Plausible systems paper, but the geometry claim outruns the evidence—the trajectories are never checked against real 3D, and the metrics don't measure trajectory accuracy. the 3 major comments →

arxiv 2512.05044 v2 pith:7JQ7MSFE submitted 2025-12-04 cs.CV

Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation

Yanran Zhang , Ziyi Wang , Wenzhao Zheng , Zheng Zhu , Jie Zhou , Jiwen Lu This is my paper
classification cs.CV
keywords 4D generationpoint trajectoriesdiffusion modelsingle-image 4Dnovel view synthesisdepth-guided normalizationmotion perceptionTrajScene-60K
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the right way to turn a single photograph into an interactive, dynamic 3D scene is to model the scene's future as dense 4D point trajectories, and to predict those trajectories jointly with the initial geometry in a single diffusion model, rather than first generating video and then reconstructing 4D, or first reconstructing a static 3D shape and then animating it. The authors build a dataset of 60,000 real videos annotated with dense, occlusion-aware point trajectories, and train a diffusion transformer that, conditioned on the input image and depth, produces a cloud of points moving coherently over time. A depth-guided normalization makes the motion representation scale-invariant, and a motion-perception module injects semantic priors about where motion is plausible. The resulting 4D representation can be rendered from arbitrary camera paths. The claim is that this coupled approach yields better spatiotemporal coherence and novel-view consistency than existing decoupled pipelines.

Core claim

The central discovery is that geometry and motion should not be treated as two sequential stages. The paper proposes representing a dynamic scene as a set of per-pixel 3D trajectories over time, and training a diffusion model to generate these trajectories conditioned on the input image and an initial depth map. Starting from a backprojected point cloud, the model predicts relative motion in a normalized coordinate frame, then the motion is de-normalized and fused to form a 4D point cloud sequence. The authors report that this joint formulation outperforms both generate-then-reconstruct and reconstruct-then-generate baselines on video-quality metrics and on a VLM-based 4D-consistency evaluat

What carries the argument

The central object is the dense 4D point trajectory field: for every pixel in the first frame, a sequence of 3D positions (plus occlusion flags) over T frames. The main mechanisms are: (1) Depth-Guided Motion Normalization, which divides each point's motion by the viewing-frustum size at its depth, making motion scale-invariant and easier to learn; (2) 4D-STraG, a Diffusion Transformer fine-tuned from a video-generation backbone, which predicts relative trajectories in a flow-matching objective with image and depth latents concatenated; (3) the Motion Perception Module (MPM), which uses a pretrained motion feature extractor to produce token-wise adaptive normalization that tells the model wh

Load-bearing premise

The training labels for 4D trajectories—extracted with a dense tracking model and monocular depth—are treated as ground truth; if those estimates are systematically biased, the model learns and propagates those biases, and the reported quality metrics do not directly measure trajectory accuracy.

What would settle it

Render the model's generated 4D point trajectories from several viewpoints in a synthetic scene with known ground-truth 3D motion and compute 3D endpoint error or average Jaccard; if the joint model does not beat a straightforward sequential baseline on these direct trajectory metrics, the central claim of superior motion-geometry coupling is not supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, any static image could be turned into a viewable, animated 4D scene with user-controlled camera movement.
  • The joint diffusion formulation removes the error-accumulation failure mode of two-stage pipelines, which should improve reliability for content creation.
  • TrajScene-60K becomes a reusable resource for training other trajectory-based 4D models.
  • The depth-guided normalization offers a principled way to handle scale in generative trajectory prediction, potentially applicable to other point-tracking tasks.
  • The method's 49-frame 512x368 output at about six minutes suggests a practical path to faster-than-optimization 4D generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The pseudo-labeling of trajectories with monocular depth and a tracking model is the most fragile link; if those labels contain systematic depth errors, the model's geometry-consistency claims may reflect the tracker's biases rather than true 3D structure.
  • The evaluation in the paper relies on video-quality and VLM-based consistency scores, not direct trajectory accuracy; a skeptic would want synthetic ground-truth 3D tests before concluding the trajectories are physically correct.
  • The joint modeling idea could extend beyond image-to-4D: the same depth-normalized trajectory field might improve video-based 3D tracking or dynamic scene reconstruction from sparse views.
  • One could test the contribution of the coupling directly by ablating to a sequential pipeline with the same components; the paper does this qualitatively, but a quantitative trajectory error would strengthen the claim.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MoRe4D, a framework for synthesizing dynamic 4D scenes from a single image. The key idea is to represent a scene as dense 4D point trajectories and to train a diffusion model (4D-STraG) that jointly predicts geometry and motion from an input image and caption, rather than decoupling reconstruction and generation. The authors construct TrajScene-60K, a dataset of 60,000 WebVid-derived videos with dense 4D trajectories extracted by DELTA and monocular depth, filtered by a set of heuristic quality checks. 4D-STraG uses depth-guided motion normalization, a Motion Perception Module (MPM) with token-wise MAdaNorm, and depth latents. A second module, 4D-ViSM, renders the predicted point trajectories into videos from arbitrary camera paths by finetuning Wan2.1 with LoRA for masked video completion. Experiments compare against 4Real, GenXD, DimensionX, Gen3C, and Free4D using VBench metrics, VLM-based perceptual ratings, ablations, and qualitative visualizations. The central claim is that the joint modeling of geometry and motion yields superior geometry-aware consistency and temporal coherence in 4D generation.

Significance. If the central claim is established, the paper makes a useful contribution: TrajScene-60K is a large-scale resource for 4D scene modeling, and the joint trajectory-generation formulation is a principled alternative to generate-then-reconstruct and vanilla reconstruct-then-generate pipelines. The depth-guided normalization and motion-aware conditioning are sensible, and the ablations (Table 2, Table C, and Figure 6) support the local contribution of each module. The reported runtime (6 minutes for 49 frames at 512×368) is competitive. However, the evidence for the paper's headline claim—geometry-aware consistency of the generated 4D trajectories—is indirect. The main quantitative metrics (VBench and VLM ratings) measure 2D video appearance and subjective perception, not geometric or trajectory accuracy, and the training pseudo-labels are never validated against independent 3D data. These gaps are load-bearing because the method's novelty rests on the 4D point-track representation being genuinely coherent in 3D. The paper also includes a candid discussion of dataset bias and an explicit acknowledgment that the image-to-4D task is ill-posed for direct error metrics, which is fair; nev

major comments (3)
  1. The training trajectories for TrajScene-60K are pseudo-labels produced by DELTA with monocular depth. The filtering pipeline (App. D.2) removes samples with invalid depth, high depth variance, and scale-inconsistent renderings, but these are internal-consistency checks. In particular, the scale-consistency check is invariant to uniform scaling by construction and cannot detect systematic depth-scale errors or motion-depth coupling artifacts. Since 4D-STraG is trained to regress exactly these labels (Sec. 4.2), any systematic bias in DELTA is learned and propagated into the generated 4D representation. Please add a validation study: run DELTA on a benchmark with independent 3D ground truth (e.g., TAPVid-3D or a synthetic dataset such as Kubric/FlyingThings3D) and report AJ/APD3D/EPE on the extracted 4D tracks, or compare a subset of TrajScene-60K trajectories against an independent multi-
  2. The quantitative evaluation never measures the accuracy or geometric consistency of the 4D point trajectories that constitute the paper's core representation. VBench scores are 2D video-quality statistics, and the VLM assessment (Table A) is a subjective perceptual rating. App. A.1 correctly argues that final-output metrics like AJ/APD3D/EPE presuppose a single ground-truth trajectory and are ill-posed for image-to-4D generation, but this does not justify omitting any direct evaluation of the intermediate trajectory representation. At minimum, report the prediction error of 4D-STraG on a held-out subset of TrajScene-60K (predicted vs. pseudo-GT relative motion) and a multi-view reprojection consistency statistic on the rendered videos. As written, the paper's headline claim is not directly tested by any reported metric.
  3. The comparison with 4Real in Group I is not controlled: 4Real is evaluated on its own official demonstration videos, while MoRe4D is evaluated on the test inputs. The numerical advantage over 4Real in Table 1 is therefore not a valid comparison and should be removed or re-run with both methods on the same inputs. Additionally, no error bars or significance tests are provided anywhere; differences as small as 0.02 (e.g., Subject Consistency 0.8241 vs. 0.8042 in Group II) are treated as decisive. Please report per-sample standard errors, confidence intervals, or multiple-seed results, especially for the ablation and baseline comparisons.
minor comments (5)
  1. The manuscript title in the full text is 'Joint 3D Geometry Reconstruction and Motion Generation for 4D Synthesis from a Single Image', while the arXiv listing and abstract refer to 'Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation' and the abstract also introduces 'MoGe4D' before switching to 'MoRe4D'. Please unify the naming.
  2. In the depth-guided normalization equations, the notation z=P(z)_0 is introduced but not fully defined; please explicitly state that z is the initial depth of the point in the first frame. The formula for ∆~z_t is written as ∆z_t/z, which is fine once z is defined.
  3. The main text says 'we follow the Wan2.1 mask processing strategy, setting the mask value to 0.5 for regions without projected points', but the actual conditioning mechanism (concatenation of z_t, z_rendered, and m_latent) is only described in App. F.2. Move a brief version of this to Sec. 4.3 for readability.
  4. The VLM-based scores are used as a primary consistency measure without any calibration against human judgments. Reporting inter-annotator agreement or a small human-study correlation would substantially strengthen this evaluation.
  5. The main text refers to 'Figure B' for qualitative results, but Figure B appears only in the appendix. Please renumber or cross-reference consistently.

Circularity Check

0 steps flagged

No significant circularity: supervised pseudo-label training and external benchmark comparisons; no step reduces to its inputs by construction.

full rationale

MoRe4D is an empirical learning pipeline rather than an analytic derivation, so the classical circularity patterns do not apply. The trajectory labels in TrajScene-60K are produced by DELTA plus monocular depth (Sec. 3, Fig. 2), and the model is trained to regress those labels; this is supervised training with pseudo-labels, not a fitted parameter being renamed as a prediction or a quantity defined in terms of the output. At inference, UniDepthv2 is deliberately chosen to match the DELTA depth setup (Sec. 4.2, F.3), but the reported VBench and VLM comparisons are against external methods on held-out WebVid samples, so the central comparison is not forced by construction. Appendix A.1 explicitly declines AJ/APD3D/EPE as ill-posed for image-to-4D; that is an acknowledged validation gap (trajectory accuracy is not independently measured), but it is an evidence limitation, not circularity. No load-bearing self-citation chain is present: the backbone (Wan2.1), tracker (DELTA), depth estimator (UniDepthv2), motion extractor (OmniMAE), and VLM judge are all external, pre-existing systems. The mild self-referential element—pseudo-label training targets and an inference depth model chosen for consistency with DELTA—affects external validity but does not make any output equal to an input by definition. Therefore no significant circularity is identified.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on pseudo-labeled data and proxy metrics rather than an analytical derivation. No new physical or conceptual entities are introduced; 4D-STraG, MPM, and 4D-ViSM are software modules without a falsifiable external handle.

free parameters (2)
  • Trained weights of 4D-STraG and 4D-ViSM (billions of parameters) = not reported
    Fitted to TrajScene-60K; the central output depends on them, but no independent benchmark verifies their trajectory accuracy.
  • MAdaNorm global gating coefficients gamma1, gamma2 = not reported
    Learned during fine-tuning (Eq. 4-6); they modulate how strongly motion features affect each DiT block.
axioms (4)
  • domain assumption DELTA's estimated trajectories are accurate enough to serve as training ground truth.
    Dataset Section 3 relies on DELTA's depth and tracking to build TrajScene-60K; no independent 3D ground truth is used.
  • domain assumption UniDepthv2 monocular depth at inference is compatible with the depth distribution used to normalize training labels.
    F.3 chooses UniDepthv2 'ensuring consistency' with DELTA; if depth distributions differ, Eq. (1) normalization and de-normalization are miscalibrated.
  • domain assumption VBench and Qwen2.5-VL ratings are valid proxies for 4D geometric consistency.
    Section 5.2 and A.1 use these instead of direct 4D trajectory metrics, which the paper declares prohibitive or ill-posed.
  • domain assumption Wan2.1's video-diffusion prior transfers to the trajectory-as-color domain after fine-tuning.
    Section 4.2 builds on Wan2.1 and acknowledges a core conflict; the success of transfer is only validated by the paper's own ablations.

pith-pipeline@v1.3.0-alltime-deepseek · 18091 in / 13701 out tokens · 126665 ms · 2026-08-03T18:26:30.424363+00:00 · methodology

0 comments
read the original abstract

Generating interactive and dynamic 4D scenes from a single static image remains a core challenge. Most existing generate-then-reconstruct and reconstruct-then-generate methods decouple geometry from motion, causing spatiotemporal inconsistencies and poor generalization. To address these, we present MoGe4D (Motion and Geometry-Aware image-to-4D Synthesis), a geometry-conditioned framework for single-image 4D synthesis that models a scene as dense 4D point trajectories. Instead of treating geometry and dynamics as two disconnected stages, our method starts from an initial geometric prior inferred from the input image and predicts future time-varying trajectories in a diffusion process, improving spatiotemporal coherence while preserving structural stability. To support this task, we first introduce TrajScene-60K, a large-scale dataset of 60,000 video samples with dense 4D point trajectories, addressing the scarcity of high-quality training data for scene-level 4D generation. Built on this, our diffusion-based 4D Scene Trajectory Generator (4D-STraG) predicts geometry-consistent and motion-plausible trajectory fields conditioned on the input image, with a depth-guided motion normalization strategy to reduce scale ambiguity and a Motion Perception Module (MPM) to inject motion-aware priors. We further propose a 4D View Synthesis Module (4D-ViSM) to render the generated 4D representation into videos under arbitrary camera trajectories. Experiments show that MoGe4D produces high-quality 4D scenes with strong temporal coherence, favorable geometry-aware consistency, and compelling novel-view synthesis from a single image. Code: https://github.com/Zhangyr2022/MoGe4D.

Figures

Figures reproduced from arXiv: 2512.05044 by Jie Zhou, Jiwen Lu, Wenzhao Zheng, Yanran Zhang, Zheng Zhu, Ziyi Wang.

Figure 1
Figure 1. Figure 1: MoRe4D for 4D synthesis from a single image. Most existing paradigms either suffer from geometric inconsistencies (generate￾then-reconstruct) or are constrained by animating a pre-determined static geometry (vanilla reconstruct-then-generate). Our MoRe4D advances by tightly coupling geometric modeling and motion generation, effectively achieving consistent 4D motion and geometry. Abstract Generating intera… view at source ↗
Figure 2
Figure 2. Figure 2: TrajScene-60K curation pipeline. We curate videos from WebVid-10M, filtered via VLMs for structured motion and countable entities. Dense 4D point tracks are extracted and refined via depth filtering and Gaussian Splatting, producing 60K high-quality 4D scenes. data, particularly for scene-level videos with significant mo￾tion and complex dynamics, we introduce TrajScene-60K, a comprehensive dataset designe… view at source ↗
Figure 3
Figure 3. Figure 3: Pipeline of MoRe4D. Top: The 4D Scene Trajectory Generator (Sec. 4.2), a Diffusion Transformer, jointly generates geometry and motion. Bottom-Left: The Motion Perception Module (MPM) identifies potential motion regions and semantic structure from the input image. Bottom-Right: The 4D View Synthesis Module (Sec. 4.3) renders the output into novel-view videos. Point Trajectory Initialization and Normalizatio… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results of our model. The first row shows the 4D point cloud generated by our 4D-STraG. The second and third rows show the videos rendered by our 4D-ViSM under two distinct, user-defined camera trajectories. our comparisons into three groups based on model avail￾ability and technical constraints, with each group evaluated under appropriate camera trajectories. In Group I, we com￾pared against t… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison with baseline methods. For each sample, the first row shows the baseline results while the second row presents our MoRe4D results. The first column displays the input image and text prompt. ing upward, forward, leftward, rightward, and downward movements, to assess performance under challenging cam￾era motions. For Groups II and III, we used 200 held-out samples from WebVid-10M that … view at source ↗
Figure 6
Figure 6. Figure 6: Ablation studies on normalization methods and module components. (Rows 1-2) Depth-guided motion normalization stabilizes 4D point cloud generation. (Rows 3-6) Removing the MPM module reduces motion magnitude while excluding depth guidance breaks structural motion consistency, validating our design choices [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D Reconstruction

    cs.CV 2026-01 unverdicted novelty 7.0

    FreeOrbit4D recovers a foreground-complete 4D proxy via decoupled background and object-centric reconstruction to provide geometric guidance for large-angle camera redirection in monocular videos using conditional vid...

  2. MMDiff: Extending Diffusion Transformers for Multi-Modal Generation

    cs.CV 2026-06 conditional novelty 6.0

    Frozen DiT features fused across multiple denoising timesteps with concept attention and optional DINO yield competitive multi-modal dense predictions and strong synthetic training data.

Reference graph

Works this paper leans on

66 extracted references · 22 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Wan2.1-Fun-14B-Control.https:// huggingface.co/alibaba- pai/Wan2.1- Fun- 14B-Control, 2024

    Alibaba PAI Team. Wan2.1-Fun-14B-Control.https:// huggingface.co/alibaba- pai/Wan2.1- Fun- 14B-Control, 2024. 5

    2024

  2. [2]

    Wan2.1-Fun-14B-InP.https:// huggingface.co/alibaba- pai/Wan2.1- Fun- 14B-InP, 2024

    Alibaba PAI Team. Wan2.1-Fun-14B-InP.https:// huggingface.co/alibaba- pai/Wan2.1- Fun- 14B-InP, 2024. 5

    2024

  3. [3]

    Fitvid: Overfitting in pixel-level video prediction.arXiv preprint arXiv:2106.13195, 2021

    Mohammad Babaeizadeh, Mohammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, and Dumitru Erhan. Fitvid: Overfitting in pixel-level video prediction.arXiv preprint arXiv:2106.13195, 2021. 2

    Pith/arXiv arXiv 2021

  4. [4]

    4D-fy: Text-to-4d generation using hybrid score distillation sampling

    Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B Lindell. 4D-fy: Text-to-4d generation using hybrid score distillation sampling. InCVPR, pages 7996–8006, 2024. 2

    2024

  5. [5]

    Recammaster: Camera-controlled generative rendering from a single video.arXiv preprint arXiv:2503.11647, 2025

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lian- rui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlled generative rendering from a single video.arXiv preprint arXiv:2503.11647, 2025. 2

    Pith/arXiv arXiv 2025

  6. [6]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 2

    Pith/arXiv arXiv 2025

  7. [7]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G ¨ul Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. InICCV, pages 1728–1738, 2021. 3, 5, 7

    2021

  8. [8]

    Gs-dit: Advancing video generation with pseudo 4d gaussian fields through efficient dense 3d point tracking.arXiv preprint arXiv:2501.02690,

    Weikang Bian, Zhaoyang Huang, Xiaoyu Shi, Yijin Li, Fu- Yun Wang, and Hongsheng Li. Gs-dit: Advancing video generation with pseudo 4d gaussian fields through efficient dense 3d point tracking.arXiv preprint arXiv:2501.02690,

    Pith/arXiv arXiv

  9. [9]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. InCVPR, pages 19457–19467, 2024. 2

    2024

  10. [10]

    Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023. 2

    Pith/arXiv arXiv 2023

  11. [11]

    4dnex: Feed-forward 4d generative modeling made easy.arXiv preprint arXiv:2508.13154, 2025

    Zhaoxi Chen, Tianqi Liu, Long Zhuo, Jiawei Ren, Zeng Tao, He Zhu, Fangzhou Hong, Liang Pan, and Ziwei Liu. 4dnex: Feed-forward 4d generative modeling made easy.arXiv preprint arXiv:2508.13154, 2025. 2

    Pith/arXiv arXiv 2025

  12. [12]

    Adver- sarial video generation on complex datasets.arXiv preprint arXiv:1907.06571, 2019

    Aidan Clark, Jeff Donahue, and Karen Simonyan. Adver- sarial video generation on complex datasets.arXiv preprint arXiv:1907.06571, 2019. 2

    Pith/arXiv arXiv 1907

  13. [13]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 7

    2017

  14. [14]

    Autoregressive video generation with- out vector quantization.arXiv preprint arXiv:2412.14169,

    Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation with- out vector quantization.arXiv preprint arXiv:2412.14169,

    Pith/arXiv arXiv

  15. [15]

    Stochastic video generation with a learned prior

    Emily Denton and Rob Fergus. Stochastic video generation with a learned prior. InICML, pages 1174–1183. PMLR,

  16. [16]

    Tap-vid: A benchmark for tracking any point in a video.NeurIPS, 35:13610–13626, 2022

    Carl Doersch, Ankush Gupta, Larisa Markeeva, Adria Re- casens, Lucas Smaira, Yusuf Aytar, Joao Carreira, Andrew Zisserman, and Yi Yang. Tap-vid: A benchmark for tracking any point in a video.NeurIPS, 35:13610–13626, 2022. 7

    2022

  17. [17]

    Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024. 2

    Pith/arXiv arXiv 2024

  18. [18]

    Omnimae: Single model masked pretraining on images and videos

    Rohit Girdhar, Alaaeldin El-Nouby, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Omnimae: Single model masked pretraining on images and videos. InCVPR, pages 10406–10417, 2023. 5

    2023

  19. [19]

    Kubric: A scalable dataset generator

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapra- gasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. InCVPR, pages 3749–3761,

  20. [20]

    Vid2avatar: 3d avatar reconstruction from videos in the wild via self-supervised scene decomposition

    Chen Guo, Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges. Vid2avatar: 3d avatar reconstruction from videos in the wild via self-supervised scene decomposition. InCVPR, pages 12858–12868, 2023. 2

    2023

  21. [21]

    Probabilistic video generation using holis- tic attribute control

    Jiawei He, Andreas Lehrmann, Joseph Marino, Greg Mori, and Leonid Sigal. Probabilistic video generation using holis- tic attribute control. InECCV, pages 452–467, 2018. 2

    2018

  22. [22]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, pages 770–778, 2016. 8

    2016

  23. [23]

    Video dif- fusion models.NeurIPS, 35:8633–8646, 2022

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models.NeurIPS, 35:8633–8646, 2022. 2

    2022

  24. [24]

    Cogvlm2: Visual language mod- els for image and video understanding.arXiv preprint arXiv:2408.16500, 2024

    Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Jun- hui Ji, Zhao Xue, et al. Cogvlm2: Visual language mod- els for image and video understanding.arXiv preprint arXiv:2408.16500, 2024. 3, 5

    Pith/arXiv arXiv 2024

  25. [25]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. InCVPR, pages 21807–21818, 2024. 5, 8

    2024

  26. [26]

    Animate3d: Animating any 3d model with multi-view video diffusion.NeurIPS, 37:125879– 125906, 2024

    Yanqin Jiang, Chaohui Yu, Chenjie Cao, Fan Wang, Weim- ing Hu, and Jin Gao. Animate3d: Animating any 3d model with multi-view video diffusion.NeurIPS, 37:125879– 125906, 2024. 2

    2024

  27. [27]

    Geo4d: Leveraging video genera- tors for geometric 4d scene reconstruction.arXiv preprint arXiv:2504.07961, 2025

    Zeren Jiang, Chuanxia Zheng, Iro Laina, Diane Larlus, and Andrea Vedaldi. Geo4d: Leveraging video genera- tors for geometric 4d scene reconstruction.arXiv preprint arXiv:2504.07961, 2025. 8

    Pith/arXiv arXiv 2025

  28. [28]

    Optimizing 4d gaussians for dynamic scene video from single landscape images

    In-Hwan Jin, Haesoo Choo, Seong-Hun Jeong, Park Heemoon, Junghwan Kim, Oh-joon Kwon, and Kyeongbo Kong. Optimizing 4d gaussians for dynamic scene video from single landscape images. InICLR, 2025. 2

    2025

  29. [29]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1,

  30. [30]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In ICCV, pages 15954–15964, 2023. 2

    2023

  31. [31]

    Tapvid-3d: A benchmark for tracking any point in 3d.NeurIPS, 37:82149–82165, 2024

    Skanda Koppula, Ignacio Rocco, Yi Yang, Joe Heyward, Joao Carreira, Andrew Zisserman, Gabriel Brostow, and Carl Doersch. Tapvid-3d: A benchmark for tracking any point in 3d.NeurIPS, 37:82149–82165, 2024. 7

    2024

  32. [32]

    Flow matching for generative mod- eling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling. InICLR, 2023. 5

    2023

  33. [33]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. 3, 5

    Pith/arXiv arXiv 2024

  34. [34]

    Re- conx: Reconstruct any scene from sparse views with video diffusion model.arXiv preprint arXiv:2408.16767, 2024

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Re- conx: Reconstruct any scene from sparse views with video diffusion model.arXiv preprint arXiv:2408.16767, 2024. 2

    Pith/arXiv arXiv 2024

  35. [35]

    Free4d: Tuning-free 4d scene gen- eration with spatial-temporal consistency.arXiv preprint arXiv:2503.20785, 2025

    Tianqi Liu, Zihao Huang, Zhaoxi Chen, Guangcong Wang, Shoukang Hu, Liao Shen, Huiqiang Sun, Zhiguo Cao, Wei Li, and Ziwei Liu. Free4d: Tuning-free 4d scene gen- eration with spatial-temporal consistency.arXiv preprint arXiv:2503.20785, 2025. 2, 5, 6, 3, 7

    Pith/arXiv arXiv 2025

  36. [36]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR. 5

  37. [37]

    Follow your pose: Pose- guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose- guided text-to-video generation using pose-free videos. In AAAI, pages 4117–4125, 2024. 2

    2024

  38. [38]

    A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation

    Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. InCVPR, pages 4040–4048, 2016. 7

    2016

  39. [39]

    Pla4d: Pixel-level alignments for text-to-4d gaussian splat- ting.arXiv preprint arXiv:2405.19957, 2024

    Qiaowei Miao, Jinsheng Quan, Kehan Li, and Yawei Luo. Pla4d: Pixel-level alignments for text-to-4d gaussian splat- ting.arXiv preprint arXiv:2405.19957, 2024. 2

    Pith/arXiv arXiv 2024

  40. [40]

    Advances in 4d generation: A survey.arXiv preprint arXiv:2503.14501, 2025

    Qiaowei Miao, Kehan Li, Jinsheng Quan, Zhiyuan Min, Shaojie Ma, Yichao Xu, Yi Yang, Ping Liu, and Yawei Luo. Advances in 4d generation: A survey.arXiv preprint arXiv:2503.14501, 2025. 1

    Pith/arXiv arXiv 2025

  41. [41]

    Multidiff: Consistent novel view synthesis from a single image

    Norman M ¨uller, Katja Schwarz, Barbara R ¨ossle, Lorenzo Porzi, Samuel Rota Bulo, Matthias Nießner, and Peter Kontschieder. Multidiff: Consistent novel view synthesis from a single image. InCVPR, pages 10258–10268, 2024. 2

    2024

  42. [42]

    Delta: Dense efficient long-range 3d track- ing for any video

    Tuan Duc Ngo, Peiye Zhuang, Evangelos Kalogerakis, Chuang Gan, Sergey Tulyakov, Hsin-Ying Lee, and Chaoyang Wang. Delta: Dense efficient long-range 3d track- ing for any video. InICLR, 2025. 3, 4

    2025

  43. [43]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 4195–4205,

  44. [44]

    Unidepthv2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110, 2025

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mat- tia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. Unidepthv2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110, 2025. 4, 8

    Pith/arXiv arXiv 2025

  45. [45]

    L4gm: Large 4d gaussian reconstruction model.Advances in Neural Information Processing Systems, 37:56828–56858, 2025

    Jiawei Ren, Cheng Xie, Ashkan Mirzaei, Karsten Kreis, Zi- wei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, Huan Ling, et al. L4gm: Large 4d gaussian reconstruction model.Advances in Neural Information Processing Systems, 37:56828–56858, 2025. 2

    2025

  46. [46]

    Gen3c: 3d-informed world-consistent video generation with precise camera con- trol

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas M ¨uller, Alexan- der Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world-consistent video generation with precise camera con- trol. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 6121–6132, 2025. 2, 6, 7

    2025

  47. [47]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. InACM SIGGRAPH, pages 1–11, 2024. 2

    2024

  48. [48]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. InICLR, 2023. 2

    2023

  49. [49]

    Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024. 2, 6, 7

    Pith/arXiv arXiv 2024

  50. [50]

    Kling image-to-video model, 2024

    KLING AI Team. Kling image-to-video model, 2024. 2

    2024

  51. [51]

    Mocogan: Decomposing motion and content for video generation

    Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. InCVPR, pages 1526–1535, 2018. 2

    2018

  52. [52]

    Neural discrete representation learning.NeurIPS, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.NeurIPS, 30, 2017. 2

    2017

  53. [53]

    Generating videos with scene dynamics

    Carl V ondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. 2016. 2

    2016

  54. [54]

    Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025. 2, 3, 5, 4

    Pith/arXiv arXiv 2025

  55. [55]

    Vggt: Vi- sual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Vi- sual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025. 4

    2025

  56. [56]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. InCVPR, pages 20310–20320, 2024. 2

    2024

  57. [57]

    Mo- tionbooth: Motion-aware customized text-to-video genera- tion.NeurIPS, 37:34322–34348, 2024

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Mo- tionbooth: Motion-aware customized text-to-video genera- tion.NeurIPS, 37:34322–34348, 2024. 2

    2024

  58. [58]

    Cat4d: Create anything in 4d with multi-view video diffusion mod- els

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Aleksander Holynski. Cat4d: Create anything in 4d with multi-view video diffusion mod- els. InCVPR, pages 26057–26068, 2025. 2

    2025

  59. [59]

    Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency.arXiv preprint arXiv:2407.17470, 2024

    Yiming Xie, Chun-Han Yao, Vikram V oleti, Huaizu Jiang, and Varun Jampani. Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency.arXiv preprint arXiv:2407.17470, 2024. 2

    Pith/arXiv arXiv 2024

  60. [60]

    Make-your-video: Customized video generation using textual and structural guidance.IEEE TVCG, 31(2):1526–1541, 2024

    Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, et al. Make-your-video: Customized video generation using textual and structural guidance.IEEE TVCG, 31(2):1526–1541, 2024. 2

    2024

  61. [61]

    4Real: Towards photorealis- tic 4d scene generation via video diffusion models.NeurIPS, 37:45256–45280, 2024

    Heng Yu, Chaoyang Wang, Peiye Zhuang, Willi Mena- pace, Aliaksandr Siarohin, Junli Cao, L ´aszl´o Jeni, Sergey Tulyakov, and Hsin-Ying Lee. 4Real: Towards photorealis- tic 4d scene generation via video diffusion models.NeurIPS, 37:45256–45280, 2024. 2, 6, 7

    2024

  62. [62]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024. 2

    Pith/arXiv arXiv 2024

  63. [63]

    Animate124: Animating one im- age to 4d dynamic scene.arXiv preprint arXiv:2311.14603,

    Yuyang Zhao, Zhiwen Yan, Enze Xie, Lanqing Hong, Zhen- guo Li, and Gim Hee Lee. Animate124: Animating one im- age to 4d dynamic scene.arXiv preprint arXiv:2311.14603,

    Pith/arXiv arXiv

  64. [64]

    GenXD: Generating Any 3D and 4D Scenes

    Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, and Lijuan Wang. GenXD: Generating Any 3D and 4D Scenes. InICLR, 2025. 6, 2, 3, 7

    2025

  65. [65]

    Nerflix: High-quality neu- ral view synthesis by learning a degradation-driven inter- viewpoint mixer

    Kun Zhou, Wenbo Li, Yi Wang, Tao Hu, Nianjuan Jiang, Xi- aoguang Han, and Jiangbo Lu. Nerflix: High-quality neu- ral view synthesis by learning a degradation-driven inter- viewpoint mixer. InCVPR, pages 12363–12374, 2023. 2

    2023

  66. [66]

    Fsgs: Real-time few-shot view synthesis using gaussian splatting

    Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. Fsgs: Real-time few-shot view synthesis using gaussian splatting. InECCV, pages 145–163. Springer, 2024. 2

    2024