Pith. sign in

REVIEW 3 major objections 4 minor 123 references

UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UniWorld-View claims that large-baseline novel view synthesis from monocular video or images can be made geometrically reliable by disambiguating point-cloud occlusion before diffusion-based generation.

desk verdict Genuinely new occlusion-aware rendering mechanism, but the zero-shot numbers are compromised by training/evaluation overlap and a training-stage contradiction; worth a careful revision, not a desk reject. read the letter →

arxiv 2608.04701 v1 pith:3X2WLAOL submitted 2026-08-05 cs.CV

Haiyang Zhou , Wangbo Yu , Chaoran Feng , Xunyu Zhou , Yonghong Tian , Li Yuan This is my paper
classification cs.CV
keywords large-baselinenovelviewsynthesisvideodiffusionmodelspointcloudrenderingocclusiondisambiguationtriplereprojectionnormal-basedvisibilitycorrectiondual-streamconditioningmonocular4Dreconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UniWorld-View sets out to make controllable novel view synthesis from a single monocular video or image work under large viewpoint changes. Its thesis is that the geometric guide fed to a generative model is the bottleneck: naive point-cloud rendering smears foreground texture over background and wrongly projects back-facing surfaces when the baseline is wide, so the diffusion model gets contradictory cues. The paper proposes an occlusion-aware rendering pipeline that first disambiguates which source pixels remain valid for each target frame, then filters by surface orientation, and couples the result with a dual-stream video diffusion model that ingests both the geometric render and the original appearance. If this works, point-based conditioning becomes dependable for large-baseline generation, and the same engine can synthesize multi-view videos that support dynamic 3D reconstruction from casual monocular input.

What carries the argument

The load-bearing mechanism is the occlusion-aware point cloud renderer. Triple reprojection (project source to target, back-project, re-project to source) yields per-frame masks that flag source pixels still valid after the round trip; the cumulative visibility mask enforces that a pixel must remain valid at every intermediate step, eliminating foreground-background tearing. Normal filtering then removes back-facing points by thresholding the dot product between the estimated surface normal and the target view direction. These masks and renders become the conditioning signal for a dual-stream video diffusion model, where the geometry stream feeds the aligned render into the pre-trained backbone's context blocks and a reference stream injects appearance from the source video through cross-attention.

What would settle it

Generate a target trajectory whose first frame is already far from the source pose, or one that moves away and then returns to the source; the paper's cumulative product of triple-reprojection masks will then mask out source pixels that are genuinely visible late in the trajectory. Comparing rendered conditions and final video quality for such trajectories against the monotonic case would show whether the assumption holds.

Watch

Extended reading notes

Core claim

The central claim is that the visibility ambiguity of naive point-cloud rendering is what breaks large-baseline point-conditioned view synthesis, and that it can be removed by a triple-reprojection procedure. Starting from the source video, the method renders the lifted point cloud into the target trajectory, back-projects that intermediate render, and re-renders it from the source camera; the resulting mask identifies source pixels that would survive the round trip and thereby marks valid versus torn regions. A cumulative product of these masks along the trajectory, together with a normal-based front-face filter, produces a clean geometric condition that is free of foreground-background tearing and spurious back-faces. Fed into a dual-stream conditional video diffusion model, one stream carrying the aligned geometric render and one cross-attention stream carrying the misaligned source appearance, this condition yields novel view videos that adhere to the requested camera path and stay geometrically consistent.

Load-bearing premise

The cumulative visibility mask in the paper's core equation assumes the target camera trajectory starts at the source pose and moves monotonically away, so a path that returns toward the source, or starts far away, will permanently erase valid pixels and undercut the geometric condition.

Editorial extensions

If this is right

  • Point-cloud rendering can serve as a reliable geometric condition for diffusion-based novel view synthesis even under extreme camera motion, not just small-baseline interpolation.
  • The two-stage multi-view generation strategy turns a monocular video into synchronized multi-view videos, giving downstream dynamic 3D Gaussian Splatting the supervision it needs without multi-view capture.
  • Single-image and monocular video inputs are handled by one unified framework, so large-baseline view synthesis and 4D reconstruction share a single engine.
  • Because view transformation is decoupled from content generation, the trained model can follow any user-specified camera trajectory rather than a fixed set of interpolation paths.
  • In evaluation against existing view-synthesis systems, the method reports the best camera-control and consistency scores, indicating that explicit geometric conditioning improves controllability as well as fidelity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cumulative visibility mask assumes the target trajectory departs monotonically from the source pose; for trajectories that swing away and then return to the start, the product of masks will permanently erase pixels that should reappear, so the method as stated would over-mask late frames.
  • A natural test extension is to replace the cumulative product with a per-frame visibility check that allows a pixel to be re-admitted when it becomes valid again, which would extend the method to arbitrary trajectories, including 360-degree sweeps.
  • The approach suggests that the remaining bottleneck for extreme baselines is the monocular depth estimator feeding the point cloud; improvements in depth consistency would likely transfer directly to view-synthesis quality.
  • The multi-view generation stage could be evaluated directly: if the synthesized multi-view videos are truly consistent, a 4D reconstruction from them should show sharp, coherent geometry rather than blur, which is a testable downstream check.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper introduces UniWorld-View, a framework for large-baseline novel view synthesis from monocular images or videos. The method first estimates depth and camera poses, lifts the source into a dynamic point cloud, and renders the target trajectory with an occlusion-aware strategy: triple-reprojection is used to build a cumulative visibility mask (Eq. 4) and normal-based filtering culls back-facing points (Eq. 5). The resulting rendered images, masks, and the original source video are fed into a dual-stream conditional video diffusion model built on VACE, with Ref-DiT cross-attention blocks for appearance reference (Sec. 3.3.1). Training combines self-supervised degraded re-projections from OpenVid-1M with static multi-view triplets from DL3DV and RealEstate10K (Sec. 3.3.2). The paper reports state-of-the-art or competitive results on the WorldScore benchmark (Table 1) and on zero-shot NVS benchmarks (Table 2), and describes a two-stage strategy for generating multi-view videos for 4D Gaussian Splatting reconstruction (Sec. 3.4). The central claim is that explicit occlusion disambiguation makes point-based geometric conditioning reliable for large-baseline diffusion-based view synthesis.

Significance. If the reported results hold, the paper makes a practical contribution to generative view synthesis: it shows that careful geometric preprocessing of point-cloud renderings, particularly the triple-reprojection visibility mask and normal-based culling, can prevent the foreground-background tearing that limits existing point-conditioned diffusion methods. The dual-stream design, separating geometry conditioning from appearance conditioning, is a sensible architectural choice. The WorldScore leaderboard submission is an independent data point, and the self-supervised dynamic training objective (degraded re-projection as input, original video as target) is an elegant way to create large-scale training signal without multi-view ground truth. However, the zero-shot evaluation in Table 2 is potentially contaminated because the same datasets (DL3DV, RealEstate10K) are used for training and evaluation, with no demonstrated disjoint split. That issue is load-bearing for the paper's central outperformance claim, so the quantitative significance of the results cannot be fully assessed until it is resolved.

major comments (3)
  1. [Section 3.3.2, Section 4.3, Table 2] The 'zero-shot' evaluation is not shown to be leak-free. The 100K static multi-view training samples are generated from DL3DV and RealEstate10K (Sec. 3.3.2), and the zero-shot evaluation in Sec. 4.3 uses those same two datasets with SEVA splits. The paper never states that the SEVA test sequences were excluded from the 100K training clips, nor does it report scene-level deduplication. Because Table 2 is the primary evidence for the claim that UniWorld-View outperforms GEN3C, Uni3C, and SEVA on zero-shot generalization, this overlap is a load-bearing correctness risk. Please provide overlap statistics or re-run the comparison with a held-out split; the CO3D results alone are insufficient to support the DL3DV and RealEstate10K claims.
  2. [Section 3.2.2, Eq. 4] The cumulative visibility mask in Eq. 4 relies on the assumption that the target trajectory begins near the source pose and deviates monotonically with the time index i. The paper explicitly states this assumption in Sec. 3.2.2, but the abstract and introduction advertise 'extreme camera motions and wide-baseline changes' without this qualification. Since the zero-shot benchmarks may contain trajectories that move away and then return, please state whether all evaluation trajectories satisfy the monotonic-deviation condition, and add an ablation on a non-monotonic trajectory to delineate the method's actual scope of applicability.
  3. [Section 3.3.3, Section 4.1] The training-stage order is described inconsistently between the method section and the implementation section. Section 3.3.3 says stage 1 uses the self-supervised dynamic data to train the Context Blocks and stage 2 uses static multi-view triplets to train the Ref-DiT layers, whereas Section 4.1 states that the first stage trains the VACE Context Blocks on 100K static multi-view triplets and the second stage trains the Ref-DiT blocks on 100K self-supervised dynamic pairs. This discrepancy affects the reproducibility of the two-stage protocol and should be reconciled.
minor comments (4)
  1. [Section 4.1] The values of the normal-view alignment threshold α (introduced in Eq. 5) and the momentum factor m (introduced in Eq. 8) are never reported; please provide them and a brief sensitivity analysis, since both are free parameters of the method.
  2. [Section 4.3] The text says the method obtains the second-best LPIPS on DL3DV, which is accurate, but on RealEstate10K the method's LPIPS (0.1678) is actually third-best behind Uni3C (0.1508) and GEN3C (0.1650); consider stating this directly to avoid any impression that the method is second-best on all datasets.
  3. [Figure 7] The qualitative comparison would benefit from zoomed-in crops around occluded boundaries, since the visual evidence for the occlusion-aware rendering is hard to inspect at full-scene scale.
  4. [Abstract] Please add the project page and GitHub repository to the references or a footnote; they are currently only mentioned in the abstract.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the conditioning chain is defined by explicit geometric equations, self-citations are not load-bearing, and the remaining benchmark concern is a data-splitting risk rather than a by-construction reduction.

full rationale

The claimed derivation chain is self-contained at the level of equations. The occlusion-aware rendering builds Mvis_i as a cumulative product of triple-reprojection masks M''_j (Eq. 4) and then renders P^s masked by Mvis and Mfront (Eq. 6). These masks are deterministic geometric quantities computed from source/target trajectories and depth, not parameters fitted to any evaluation metric, so no 'prediction' reduces to a fit by construction. The self-supervised dynamic training in Sec. 3.3.2 uses the degraded reprojection I'' and mask M'' as condition with the pristine source video Is as supervision; that is a reconstruction objective that does not directly optimize the reported PSNR/SSIM/LPIPS numbers. Static multi-view training uses genuine source-target video pairs, and the benchmark numbers are external measurements. Self-citations to TrajectoryCrafter [106] and ViewCrafter [109] describe a preliminary version and prior point-cloud-conditioned methods; they are not used to import an unverified uniqueness theorem or to forbid alternative designs, so they are not load-bearing circularity. The monotonic-trajectory assumption in Eq. 4 is explicitly stated as a restriction ('the deviation from the source pose is minimal at i = 1 and increases monotonically with i'), and an assumption, even if brittle, is not a circular definition. The reviewer's training/evaluation overlap concern (training on 100K samples from DL3DV and RealEstate10K, then reporting 'zero-shot' numbers on those datasets following SEVA splits) is a legitimate correctness and data-hygiene risk, but the paper's text does not exhibit the specific reduction required to call it circularity: there is no quoted equation or fitting step showing the Table 2 numbers are forced by the training objective. Under the stated hard rules, that concern belongs in correctness review, not in the circularity score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of upstream depth and pose estimators, on a monotone trajectory assumption baked into Eq. 4, and on the capacity of a large diffusion model to follow geometric conditions. No new physical entities are introduced.

free parameters (2)
  • Normal-view alignment threshold alpha = not reported (predefined in [0, pi/2])
    Eq. 5 culls points based on n dot v > cos(alpha). The value is chosen by hand and directly affects the front-facing validity mask and therefore all rendered geometric conditions.
  • Momentum factor m = not reported
    Eq. 8 uses a momentum-based scale and shift alignment for foreground depth in the 4D reconstruction pipeline. The value is chosen by hand and affects consistency of the reconstructed foreground depth.
assumptions (4)
  • domain assumption Feed-forward geometry estimators (VGGT, VideoDepthAnything, Stream3R) provide accurate, mutually consistent camera poses, intrinsics, and depth for arbitrary monocular video.
    Invoked in Sec. 3.2.1 and Sec. 3.4.1. If these estimators fail, the point cloud and all downstream renders are wrong, which invalidates the geometric conditioning.
  • ad hoc to paper The target trajectory T_r starts near the source pose and deviates monotonically with time index i, making Eq. 4's cumulative product a correct visibility signal.
    Eq. 4 assumes visibility from the source only decreases along the trajectory. This is a design assumption specific to the method and not validated for arbitrary user trajectories.
  • domain assumption A video diffusion model trained on degraded forward-and-back re-projections can learn to hallucinate disoccluded content while following the geometric conditions.
    The training scheme in Sec. 3.3.2 relies on this empirical capability of the pretrained VACE backbone and the new Ref-DiT blocks.
  • standard math Standard perspective projection, inverse projection, and the diffusion denoising objective in Eq. 1 are valid background tools.
    Eqs. 1-3 use standard diffusion training and pinhole camera projection. These are unremarkable background assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models." pith.science (2026). https://pith.science/paper/3X2WLAOL

@misc{pith2026260804701,
  author       = {Pith},
  title        = {Pith review of: UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3X2WLAOL}},
  note         = {Machine review of arXiv:2608.04701}
}
read the original abstract

The abundance of casually captured monocular videos and images on social media provides a valuable source for immersive content creation, where generating novel views from such sparse observations can greatly enhance user experiences. However, producing photorealistic and geometrically consistent views with precise camera control remains challenging when input coverage is extremely limited. Reconstruction-based approaches such as NeRF and 3D Gaussian Splatting (3DGS) deteriorate severely under sparse inputs and fail to explicitly handle occlusions. Generative methods ease data requirements but still struggle with large-baseline view synthesis due to inaccurate or implicit geometric guidance. To overcome these limitations, we introduce UniWorld-View, a unified framework for controllable large-baseline novel view synthesis from monocular inputs. UniWorld-View integrates explicit 3D guidance with generative diffusion modeling to enable precise camera control and geometrically consistent view generation. The geometric guidance is obtained through an occlusion-aware point cloud rendering strategy that resolves visibility ambiguities and provides accurate priors for diffusion-based synthesis. By coupling this rendering strategy with powerful video diffusion backbones, UniWorld-View achieves high-fidelity novel view generation even under extreme camera motions and wide-baseline changes, and can further provide multi-view videos for downstream dynamic 3DGS reconstruction. Experiments on the WorldScore benchmark and zero-shot NVS benchmarks demonstrate the effectiveness of UniWorld-View in controllability, geometric consistency, and visual fidelity.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

123 extracted references · 38 canonical work pages

  1. [1]

    HyperReel: High-fidelity 6-DoF video with ray-conditioned sampling

    Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O’Toole, and Changil Kim. HyperReel: High-fidelity 6-DoF video with ray-conditioned sampling. InCVPR, 2023

    2023

  2. [2]

    Vd3d: Taming large video diffusion transformers for 3d camera control

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffusion transformers for 3d camera control. InICLR, 2025

    2025

  3. [3]

    Recammaster: Camera-controlled generative rendering from a single video

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, and Pengfei Wan. Recammaster: Camera-controlled generative rendering from a single video. InICCV, 2025

    2025

  4. [4]

    Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints

    Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Xiao Fu, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di Zhang. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints. InICLR, 2025

    2025

  5. [5]

    Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. InICCV, 2021

    2021

  6. [6]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. InCVPR, 2022

    2022

  7. [7]

    Zip-nerf: Anti-aliased grid-based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. InICCV, 2023

    2023

  8. [8]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

Show all 123 references
  1. [9]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. InCVPR, 2023

    2023

  2. [10]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. OpenAI website, 2024. URL https://openai.com/research/video-ge...

    2024

  3. [11]

    Hexplane: A fast representation for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. InCVPR, 2023

    2023

  4. [12]

    Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation

    Chenjie Cao, Jingkai Zhou, shikai Li, Jingyun Liang, Chaohui Yu, Fan Wang, Xiangyang Xue, and Yanwei Fu. Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation. In SIGGRAPH Asia, 2025

    2025

  5. [13]

    Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo

    Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. InICCV, 2021

    2021

  6. [14]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023

    arXiv

  7. [15]

    Video depth anything: Consistent depth estimation for super-long videos.arXiv:2501.12375, 2025

    Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos.arXiv:2501.12375, 2025

    2025 arXiv

  8. [16]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. InECCV, 2024

    2024

  9. [17]

    Luciddreamer: Domain-free generation of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023

    Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free generation of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023

    2023 arXiv

  10. [18]

    Cogvideox-fun, 2024

    CogVideoX-Fun. Cogvideox-fun, 2024. URLhttps://github.com/aigc-apps/CogVideoX-Fun

    2024

  11. [19]

    Meva: A large-scale multiview, multimodal video dataset for activity detection

    Kellie Corona, Katie Osterdahl, Roderic Collins, and Anthony Hoogs. Meva: A large-scale multiview, multimodal video dataset for activity detection. InW ACV, 2021. 14

    2021

  12. [20]

    Instantsplat: Unbounded sparse-view pose-free gaussian splatting in 40 seconds.arXiv:2403.20309, 2024

    Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang, Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, et al. Instantsplat: Unbounded sparse-view pose-free gaussian splatting in 40 seconds.arXiv:2403.20309, 2024

    2024 arXiv

  13. [21]

    Ae-nerf: Augmenting event-based neural radiance fields for non-ideal conditions and larger scene

    Chaoran Feng, Wangbo Yu, Xinhua Cheng, Zhenyu Tang, Junwu Zhang, Li Yuan, and Yonghong Tian. Ae-nerf: Augmenting event-based neural radiance fields for non-ideal conditions and larger scene. InAAAI, 2025

    2025

  14. [22]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. InCVPR, 2023

    2023

  15. [23]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. InICCV, 2021

    2021

  16. [24]

    Gaussianflow: Splatting gaussian dynamics for 4d content creation.arXiv preprint arXiv:2403.12365, 2024

    Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Gaussianflow: Splatting gaussian dynamics for 4d content creation.arXiv preprint arXiv:2403.12365, 2024

    2024 arXiv

  17. [25]

    CAT3D: Create Anything in 3D with Multi-View Diffusion Models

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. CAT3D: Create Anything in 3D with Multi-View Diffusion Models. NeurIPS, 2024

    2024

  18. [26]

    Fastnerf: High-fidelity neural rendering at 200fps

    Stephan J Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin. Fastnerf: High-fidelity neural rendering at 200fps. InICCV, 2021

    2021

  19. [27]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InCVPR, 2022

    2022

  20. [28]

    Kubric: A scalable dataset generator

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. InCVPR, 2022

    2022

  21. [29]

    Diffusion as shader: 3d-aware video diffusion for versatile video generation control

    Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, et al. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. InSIGGRAPH, 2025

    2025

  22. [30]

    Sparsenerf: Distilling depth ranking for few-shot novel view synthesis

    Guangcong, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. InICCV, 2023

    2023

  23. [31]

    Cameractrl: Enabling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. InICLR, 2025

    2025

  24. [32]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InNeurIPS, 2020

    2020

  25. [33]

    Tri-miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields

    Wenbo Hu, Yuling Wang, Lin Ma, Bangbang Yang, Lin Gao, Xiao Liu, and Yuewen Ma. Tri-miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields. InICCV, 2023

    2023

  26. [34]

    Depthcrafter: Generating consistent long depth sequences for open-world videos.arXiv preprint arXiv:2409.02095, 2024

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos.arXiv preprint arXiv:2409.02095, 2024

    2024 arXiv

  27. [35]

    Vipe: Video pose engine for 3d geometric perception

    Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, et al. Vipe: Video pose engine for 3d geometric perception. arXiv preprint arXiv:2508.10934, 2025

    2025 arXiv

  28. [36]

    Roompainter: View-integrated diffusion for consistent indoor scene texturing

    Zhipeng Huang, Wangbo Yu, Xinhua Cheng, ChengShu Zhao, Yunyang Ge, Mingyi Guo, Li Yuan, and Yonghong Tian. Roompainter: View-integrated diffusion for consistent indoor scene texturing. InCVPR, 2025

    2025

  29. [37]

    Vace: All-in-one video creation and editing

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. InICCV, 2025

    2025

  30. [38]

    3d gaussian splatting for real-time radiance field rendering.ACM TOG, 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM TOG, 2023. 15

    2023

  31. [39]

    Stream3r: Scalable sequential 3d reconstruction with causal transformer.arXiv preprint arXiv:2508.10893, 2025

    Yushi Lan, Yihang Luo, Fangzhou Hong, Shangchen Zhou, Honghua Chen, Zhaoyang Lyu, Shuai Yang, Bo Dai, Chen Change Loy, and Xingang Pan. Stream3r: Scalable sequential 3d reconstruction with causal transformer.arXiv preprint arXiv:2508.10893, 2025

    2025 arXiv

  32. [40]

    Fast view synthesis of casual videos with soup-of-planes

    Yao-Chih Lee, Zhoutong Zhang, Kevin Blackburn-Matzen, Simon Niklaus, Jianming Zhang, Jia-Bin Huang, and Feng Liu. Fast view synthesis of casual videos with soup-of-planes. InECCV, 2025

    2025

  33. [41]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds.arXiv preprint arXiv:2405.17421, 2024

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds.arXiv preprint arXiv:2405.17421, 2024

    2024 arXiv

  34. [42]

    Nerfacc: A general nerf acceleration toolbox.arXiv preprint arXiv:2210.04847, 2022

    Ruilong Li, Matthew Tancik, and Angjoo Kanazawa. Nerfacc: A general nerf acceleration toolbox.arXiv preprint arXiv:2210.04847, 2022

    2022 arXiv

  35. [43]

    Spacetime gaussian feature splatting for real-time dynamic view synthesis

    Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. InCVPR, 2024

    2024

  36. [44]

    Neural scene flow fields for space-time view synthesis of dynamic scenes

    Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. InCVPR, 2021

    2021

  37. [45]

    Dynibar: Neural dynamic image-based rendering

    Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic image-based rendering. InCVPR, 2023

    2023

  38. [46]

    Wonderland: Navigating 3d scenes from a single image.arXiv preprint arXiv:2412.12091, 2024

    Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos Plataniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Navigating 3d scenes from a single image.arXiv preprint arXiv:2412.12091, 2024

    2024 arXiv

  39. [47]

    Analytic-splatting: Anti-aliased 3d gaussian splatting via analytic integration

    Zhihao Liang, Qi Zhang, Wenbo Hu, Lei Zhu, Ying Feng, and Kui Jia. Analytic-splatting: Anti-aliased 3d gaussian splatting via analytic integration. InECCV, 2024

    2024

  40. [48]

    Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    2024 arXiv

  41. [49]

    Barf: Bundle-adjusting neural radiance fields

    Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. Barf: Bundle-adjusting neural radiance fields. InICCV, 2021

    2021

  42. [50]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InCVPR, 2024

    2024

  43. [51]

    Rip-nerf: Anti-aliasing radiance fields with ripmap-encoded platonic solids

    Junchen Liu, Wenbo Hu, Zhuo Yang, Jianteng Chen, Guoliang Wang, Xiaoxue Chen, Yantong Cai, Huan-ang Gao, and Hao Zhao. Rip-nerf: Anti-aliasing radiance fields with ripmap-encoded platonic solids. InACM SIGGRAPH, 2024

    2024

  44. [52]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3d object. InICCV, 2023

    2023

  45. [53]

    Free4d: Tuning-free 4d scene generation with spatial-temporal consistency.arXiv preprint arXiv:2503.20785, 2025

    Tianqi Liu, Zihao Huang, Zhaoxi Chen, Guangcong Wang, Shoukang Hu, Liao Shen, Huiqiang Sun, Zhiguo Cao, Wei Li, and Ziwei Liu. Free4d: Tuning-free 4d scene generation with spatial-temporal consistency.arXiv preprint arXiv:2503.20785, 2025

    2025 arXiv

  46. [54]

    See4d: Pose-free 4d generation via auto-regressive video inpainting.arXiv preprint arXiv:2510.26796, 2025

    Dongyue Lu, Ao Liang, Tianxin Huang, Xiao Fu, Yuyang Zhao, Baorui Ma, Liang Pan, Wei Yin, Lingdong Kong, Wei Tsang Ooi, et al. See4d: Pose-free 4d generation via auto-regressive video inpainting.arXiv preprint arXiv:2510.26796, 2025

  47. [55]

    You see it, you got it: Learning 3d creation on pose-free videos at scale

    Baorui Ma, Huachen Gao, Haoge Deng, Zhengxiong Luo, Tiejun Huang, Lulu Tang, and Xinlong Wang. You see it, you got it: Learning 3d creation on pose-free videos at scale. InCVPR, 2025

    2025

  48. [56]

    Rose: Remove objects with side effects in videos.arXiv preprint arXiv:2508.18633, 2025

    Chenxuan Miao, Yutong Feng, Jianshu Zeng, Zixiang Gao, Hantang Liu, Yunfeng Yan, Donglian Qi, Xi Chen, Bin Wang, and Hengshuang Zhao. Rose: Remove objects with side effects in videos.arXiv preprint arXiv:2508.18633, 2025

    2025 arXiv

  49. [57]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InECCV, 2020

    2020

  50. [58]

    Multidiff: Consistent novel view synthesis from a single image

    Norman Müller, Katja Schwarz, Barbara Rössle, Lorenzo Porzi, Samuel Rota Bulò, Matthias Nießner, and Peter Kontschieder. Multidiff: Consistent novel view synthesis from a single image. InCVPR, 2024. 16

    2024

  51. [59]

    Instant neural graphics primitives with a multiresolution hash encoding.ACM Transactions on Graphics (ToG), 41(4):1–15, 2022

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding.ACM Transactions on Graphics (ToG), 41(4):1–15, 2022

    2022

  52. [60]

    Openvid-1m: A large-scale high-quality dataset for text-to-video generation.arXiv preprint arXiv:2407.02371, 2024

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation.arXiv preprint arXiv:2407.02371, 2024

    2024 arXiv

  53. [61]

    Carvekit: Image background remove tool

    OPHoperHPO. Carvekit: Image background remove tool. https://github.com/OPHoperHPO/image-background-remove-tool, 2024. Accessed: 2025-12-03

    2024

  54. [62]

    Bridging implicit and explicit geometric transformation for single-image view synthesis.IEEE TPAMI, 2024

    Byeongjun Park, Hyojun Go, and Changick Kim. Bridging implicit and explicit geometric transformation for single-image view synthesis.IEEE TPAMI, 2024

    2024

  55. [63]

    Barron, Sofien Bouaziz, Dan B Goldman, Steven M

    Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. InICCV, 2021

    2021

  56. [64]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InICCV, 2023

    2023

  57. [65]

    D-nerf: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. InCVPR, 2021

    2021

  58. [66]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2024 arXiv

  59. [67]

    Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. InICCV, 2021

    2021

  60. [68]

    Gen3c: 3d-informed world-consistent video generation with precise camera control

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world-consistent video generation with precise camera control. InCVPR, 2025

    2025

  61. [69]

    Pixelsynth: Generating a 3d-consistent experience from a single image

    Chris Rockwell, David F Fouhey, and Justin Johnson. Pixelsynth: Generating a 3d-consistent experience from a single image. InICCV, 2021

    2021

  62. [70]

    Geometry-free view synthesis: Transformers and no 3d priors

    Robin Rombach, Patrick Esser, and Björn Ommer. Geometry-free view synthesis: Transformers and no 3d priors. InICCV, 2021

    2021

  63. [71]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, 2022

    2022

  64. [72]

    ZeroNVS: Zero-shot 360-degree view synthesis from a single real image

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, and Jiajun Wu. ZeroNVS: Zero-shot 360-degree view synthesis from a single real image. InCVPR, 2024

    2024

  65. [73]

    Assembly101: A large-scale multi-view video dataset for understanding procedural activities

    Fadime Sener, Dibyadip Chatterjee, Daniel Shelepov, Kun He, Dipika Singhania, Robert Wang, and Angela Yao. Assembly101: A large-scale multi-view video dataset for understanding procedural activities. InCVPR, 2022

    2022

  66. [74]

    Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion.arXiv preprint arXiv:2404.07199, 2024

    Jaidev Shriram, Alex Trevithick, Lingjie Liu, and Ravi Ramamoorthi. Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion.arXiv preprint arXiv:2404.07199, 2024

    2024 arXiv

  67. [75]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. InICLR, 2021

    2021

  68. [76]

    Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields.IEEE TVCG, 2023

    Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields.IEEE TVCG, 2023

    2023

  69. [77]

    Dynamic gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In SIGGRAPH Asia 2024 Conference Papers, 2024

    2024

  70. [78]

    Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024. 17

    2024 arXiv

  71. [79]

    Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. InProceedings of the IEEE/CVF International Confer...

    2021

  72. [80]

    Megascenes: Scene-level view synthesis at scale

    Joseph Tung, Gene Chou, Ruojin Cai, Guandao Yang, Kai Zhang, Gordon Wetzstein, Bharath Hariharan, and Noah Snavely. Megascenes: Scene-level view synthesis at scale. InECCV, pages 197–214, 2024

    2024

  73. [81]

    Generative camera dolly: Extreme monocular dynamic novel view synthesis

    Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sargent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl Vondrick. Generative camera dolly: Extreme monocular dynamic novel view synthesis. InECCV, 2024

    2024

  74. [82]

    Ref-nerf: Structured view-dependent appearance for neural radiance fields

    Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. Ref-nerf: Structured view-dependent appearance for neural radiance fields. InCVPR, 2022

    2022

  75. [83]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pande...

    2025 arXiv

  76. [84]

    Vistadream: Sampling multiview consistent images for single-view scene reconstruction

    Haiping Wang, Yuan Liu, Ziwei Liu, Wenping Wang, Zhen Dong, and Bisheng Yang. Vistadream: Sampling multiview consistent images for single-view scene reconstruction. InCVPR, pages 26772–26782, 2025

    2025

  77. [85]

    Videoscene: Distilling video diffusion model to generate 3d scenes in one step

    Hanyang Wang, Fangfu Liu, Jiawei Chi, and Yueqi Duan. Videoscene: Distilling video diffusion model to generate 3d scenes in one step. InCVPR, pages 16475–16485, 2025

    2025

  78. [86]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  79. [87]

    Shape of motion: 4d reconstruction from a single video.arXiv preprint arXiv:2407.13764, 2024

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video.arXiv preprint arXiv:2407.13764, 2024

    2024

  80. [88]

    Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision

    Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. InCVPR, 2025

    2025

  81. [89]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InCVPR, 2024

    2024

  82. [90]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH, 2024

    2024

  83. [91]

    Synsin: End-to-end view synthesis from a single image

    Olivia Wiles, Georgia Gkioxari, Richard Szeliski, and Justin Johnson. Synsin: End-to-end view synthesis from a single image. InCVPR, 2020

    2020

  84. [92]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. InCVPR, 2024

    2024

  85. [93]

    Cat4d: Create anything in 4d with multi-view video diffusion models.arXiv preprint arXiv:2411.18613, 2024

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Aleksander Holynski. Cat4d: Create anything in 4d with multi-view video diffusion models.arXiv preprint arXiv:2411.18613, 2024

    2024 arXiv

  86. [94]

    Reconfusion: 3d reconstruction with diffusion priors

    Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P Srinivasan, Dor Verbin, Jonathan T Barron, Ben Poole, et al. Reconfusion: 3d reconstruction with diffusion priors. In CVPR, 2024

    2024

  87. [95]

    Trajectory attention for fine-grained video motion control

    Zeqi Xiao, Wenqi Ouyang, Yifan Zhou, Shuai Yang, Lei Yang, Jianlou Si, and Xingang Pan. Trajectory attention for fine-grained video motion control. InICLR, 2025. 18

    2025

  88. [96]

    Dynamicrafter: Animating open-domain images with video diffusion priors.arXiv preprint arXiv:2310.12190, 2023

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xintao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors.arXiv preprint arXiv:2310.12190, 2023

    2023 arXiv

  89. [97]

    Camco: Camera-controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. Camco: Camera-controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024

    2024 arXiv

  90. [98]

    4dgt: Learning a 4d gaussian transformer using real-world monocular videos.arXiv preprint arXiv:2506.08015, 2025

    Zhen Xu, Zhengqin Li, Zhao Dong, Xiaowei Zhou, Richard Newcombe, and Zhaoyang Lv. 4dgt: Learning a 4d gaussian transformer using real-world monocular videos.arXiv preprint arXiv:2506.08015, 2025

    2025

  91. [99]

    Depth anything v2.arXiv:2406.09414, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2.arXiv:2406.09414, 2024

    2024 arXiv

  92. [100]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. InICLR, 2024

    2024

  93. [101]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  94. [102]

    Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera

    Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera. InCVPR, 2020

    2020

  95. [103]

    Nvs-solver: Video diffusion model as zero-shot novel view synthesizer

    Meng You, Zhiyu Zhu, Hui Liu, and Junhui Hou. Nvs-solver: Video diffusion model as zero-shot novel view synthesizer. InICLR, 2025

    2025

  96. [104]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4578–4587, 2021

    2021

  97. [105]

    Wonderworld: Interactive 3d scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. InCVPR, pages 5916–5926, 2025

    2025

  98. [106]

    Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models

    Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models. InICCV, 2025

    2025

  99. [107]

    Nofa: Nerf-based one-shot facial avatar reconstruction

    Wangbo Yu, Yanbo Fan, Yong Zhang, Xuan Wang, Fei Yin, Yunpeng Bai, Yan-Pei Cao, Ying Shan, Yang Wu, Zhongqian Sun, et al. Nofa: Nerf-based one-shot facial avatar reconstruction. InACM SIGGRAPH, 2023

    2023

  100. [108]

    Evagaussians: Event stream assisted gaussian splatting from blurry images.arXiv preprint arXiv:2405.20224, 2024

    Wangbo Yu, Chaoran Feng, Jiye Tang, Jiashu Yang, Zhenyu Tang, Xu Jia, Yuchao Yang, Li Yuan, and Yonghong Tian. Evagaussians: Event stream assisted gaussian splatting from blurry images.arXiv preprint arXiv:2405.20224, 2024

    2024 arXiv

  101. [109]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    2024 arXiv

  102. [110]

    Hifi-123: Towards high-fidelity one image to 3d content generation

    Wangbo Yu, Li Yuan, Yan-Pei Cao, Xiangjun Gao, Xiaoyu Li, Wenbo Hu, Long Quan, Ying Shan, and Yonghong Tian. Hifi-123: Towards high-fidelity one image to 3d content generation. InECCV, 2024

    2024

  103. [111]

    Mip-splatting: Alias-free 3d gaussian splatting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. InCVPR, 2024

    2024

  104. [112]

    Recapture: Generative video camera controls for user-provided videos using masked video fine-tuning.arXiv preprint arXiv:2411.05003, 2024

    David Junhao Zhang, Roni Paiss, Shiran Zada, Nikhil Karnad, David E Jacobs, Yael Pritch, Inbar Mosseri, Mike Zheng Shou, Neal Wadhwa, and Nataniel Ruiz. Recapture: Generative video camera controls for user-provided videos using masked video fine-tuning.arXiv preprint arXiv:241...

    2024 arXiv

  105. [113]

    Monst3r: A simple approach for estimating geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024

    arXiv

  106. [114]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InICCV, 2023

    2023

  107. [115]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InCVPR, 2018. 19

    2018

  108. [116]

    Spatialcrafter: Unleashing the imagination of video diffusion models for scene reconstruction from limited observations

    Songchun Zhang, Huiyao Xu, Sitong Guo, Zhongwei Xie, Hujun Bao, Weiwei Xu, and Changqing Zou. Spatialcrafter: Unleashing the imagination of video diffusion models for scene reconstruction from limited observations. InICCV, pages 27794–27805, 2025

    2025

  109. [117]

    Pixel-gs: Density control with pixel-aware gradient for 3d gaussian splatting

    Zheng Zhang, Wenbo Hu, Yixing Lao, Tong He, and Hengshuang Zhao. Pixel-gs: Density control with pixel-aware gradient for 3d gaussian splatting. InECCV, 2024

    2024

  110. [118]

    Genxd: Generating any 3d and 4d scenes.arXiv preprint arXiv:2411.02319, 2024

    Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, and Lijuan Wang. Genxd: Generating any 3d and 4d scenes.arXiv preprint arXiv:2411.02319, 2024

    2024 arXiv

  111. [119]

    Pointodyssey: A large-scale synthetic dataset for long-term point tracking

    Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wetzstein, and Leonidas J Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. InICCV, 2023

    2023

  112. [120]

    Holodreamer: Holistic 3d panoramic world generation from text descriptions.arXiv preprint arXiv:2407.15187, 2024

    Haiyang Zhou, Xinhua Cheng, Wangbo Yu, Yonghong Tian, and Li Yuan. Holodreamer: Holistic 3d panoramic world generation from text descriptions.arXiv preprint arXiv:2407.15187, 2024

    2024 arXiv

  113. [121]

    Stable virtual camera: Generative view synthesis with diffusion models.arXiv preprint arXiv:2503.14489, 2025

    Jensen (Jinghao) Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models.arXiv preprint arXiv:2503.14489, 2025

    2025 arXiv

  114. [122]

    Stereo magnification: Learning view synthesis using multiplane images.ACM TOG, 2018

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images.ACM TOG, 2018

    2018

  115. [123]

    Fsgs: Real-time few-shot view synthesis using gaussian splatting

    Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. Fsgs: Real-time few-shot view synthesis using gaussian splatting. InECCV, 2024. 20

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.