Pith. sign in

REVIEW 4 major objections 5 minor 84 references

Complete Gaussian Splats from a Single Image with Denoising Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A latent diffusion model completes occluded 3D scenes from one photo

desk verdict A clean single-image 3D-from-2D pipeline with a genuinely neat training trick, but the diversity claim is not backed by the objective and needs much stronger evidence. read the letter →

arxiv 2508.21542 v1 pith:YCBRUKEH submitted 2025-08-29 cs.CV cs.AIcs.RO

Ziwei Liao , Mohamed Sayed , Steven L. Waslander , Sara Vicente , Daniyar Turmukhambetov , Michael Firman This is my paper · ORCID
classification cs.CVcs.AIcs.RO
keywords single-image3DreconstructionGaussiansplattinglatentdiffusionscenecompletiongenerativemodelVariationalAutoReconstructornovelviewsynthesisocclusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to prove that a single RGB photograph is enough to reconstruct a complete 3D scene as Gaussian splats, including surfaces that are occluded or outside the camera frustum. The route is generative rather than regression-based: instead of predicting a single averaged 3D output, the method learns a distribution of Gaussian-splat scenes and samples from it, producing diverse plausible completions of hidden regions. To avoid needing ground-truth 3D data, the paper introduces a Variational AutoReconstructor that learns a compact latent space of 3D splat scenes using only posed 2D images and differentiable rendering, then trains a denoising diffusion model on that latent space conditioned on the input image. If the claim holds, single-image 3D reconstruction becomes a real-time generative task with multi-view consistency.

What carries the argument

The load-bearing mechanism is the Variational AutoReconstructor (VAR), a self-supervised encoder-reconstructor that learns a latent space for Splatter Images from images alone. A Splatter Image is the paper's scene representation: for each input pixel ray, the network predicts several Gaussians, producing a total parameter matrix of shape $H \times W \times MN$, so that decoding a latent code yields a complete set of 3D Gaussians inside the input frustum. The VAR makes this trainable with only 2D posed images by rendering the Gaussians into target views and backpropagating image reconstruction losses, with skip connections from the encoder's first layer preserving high-frequency texture; a KL term regularizes the latent toward a Gaussian prior, over which the diffusion model is trained.

What would settle it

On a held-out category with known 360-degree geometry, generate 20 samples per input and measure the spread of predicted depth maps and renderings in the half of the scene not visible from the input; if the variance across samples is no larger than a regression baseline's, or if completed backs collapse to the same averaged shape, the claimed ability to sample diverse occluded completions is refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that the distribution of complete 3D Gaussian-splat scenes, conditioned on a single image, can be learned and sampled without ever seeing ground-truth 3D training data. The Variational AutoReconstructor encodes a reference image into a 4-channel latent distribution, decodes a sampled latent into a per-pixel Splatter Image with multiple Gaussians per ray, and is supervised only by rendering those Gaussians into the reference and nearby posed target views and comparing with real photographs using L2, SSIM, and LPIPS losses. A denoising diffusion model is then trained over these latents, conditioned on features of the input image, with classifier-free guidance and randomized skip connections to trade faithfulness against diversity. At inference, denoising random latents yields Splatter Images that render 360-degree views in real time, and the paper's experiments show these outputs are sharper and more complete than regression baselines in occluded regions while also supporting diverse samples.

Load-bearing premise

The compact latent space, learned only to make rendered views match photographs, must actually encode the unseen back and occluded structure of the scene rather than just the visible surfaces, so that diffusion sampling can generate diverse and plausible completions instead of blurry averages.

Editorial extensions

If this is right

  • With one input image, an operator can draw multiple complete scene hypotheses instead of one blurry average, each rendering consistently from novel viewpoints.
  • Training needs no 3D ground truth, only posed image collections, so the approach can scale to large monocular video datasets rather than expensive 3D capture.
  • Inference takes about 3 seconds for a 50-step diffusion sample plus 29 ms per rendered frame, versus minutes for diffusion-based NeRF baselines, making interactive single-view reconstruction plausible.
  • Guidance and skip-connection weights give explicit control: high fidelity to visible pixels for reconstruction, lower fidelity for diversity in occluded areas and data augmentation.
  • On evaluated categories (hydrants, teddy bears, room scenes) the method beats real-time feed-forward baselines on perceptual metrics and matches slower diffusion-NeRF quality on several settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the latent space really carries occluded structure, the same VAR could be reused for tasks beyond single-view sampling, such as 3D editing, inpainting a masked region while keeping the rest consistent, or interpolating between scene latents.
  • The dependency on a single frustum means large scenes are stitched from local predictions; extending the method to multiple latent frames or a global scene representation would be a natural next test.
  • A decisive experiment the paper does not run: fix an input image, sample many times, and measure per-pixel variance in the occluded region against ground-truth geometry; large variance with plausible structure would confirm the generative claim, while mode collapse would expose the weak assumption.
  • The method's conditioning features come from a pretrained image autoencoder; swapping those features for a semantic or text embedding could turn the same pipeline into a text-controllable 3D scene generator.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a latent diffusion model for reconstructing complete 3D Gaussian-splat scenes from a single RGB image. Since no large-scale dataset of ground-truth Splatter Images exists, the authors introduce a Variational AutoReconstructor (VAR) that encodes a reference image into a low-dimensional latent, decodes it to a Splatter Image, and supervises the result solely through differentiable rendering against posed multi-view images. A denoising diffusion model is then trained on latents sampled from the VAR encoder, conditioned on the input image. At inference, random latents are denoised and decoded to produce multiple plausible 3D reconstructions that can be rendered in real time. Experiments on CO3D Hydrants and TeddyBears and on RealEstate10K report improved sharpness and completeness over regression-based baselines, with qualitative demonstrations of diverse completions of occluded regions.

Significance. If the central claims are supported, the paper makes a useful contribution: it offers a scalable, image-only pipeline for learning a generative prior over 3D Gaussian-splat scenes, avoids expensive ground-truth 3D optimization, and retains real-time rendering. The proposed Variational AutoReconstructor and the use of skip connections with random zeroing are reasonable design choices, and the ablations in Table 4 support their importance. The qualitative results in Figures 5-8 are appealing, and the supplementary material is thorough about implementation details and limitations. However, the paper's most distinctive claim--that the diffusion model samples diverse, plausible completions of occluded geometry--rests on evidence that is either qualitative or oracle-based, and the training objective analyzed in Section 3.2 does not by itself guarantee a multi-modal latent space. This is the key load-bearing point that needs strengthening before the paper can be fully accepted.

major comments (4)
  1. [§3.2, Eq. (1)-(2), §B.3] The training procedure for the Variational AutoReconstructor does not explicitly encourage different latent samples to correspond to different plausible 3D completions. For a given reference image, every latent sample is decoded and supervised against the same target views through Eq. (1); the KL term in Eq. (2) only regularizes the latent toward a Gaussian prior. The random zeroing of skip features described in §B.3 prevents the reconstructor from ignoring the latent entirely, but it does not create multi-modality in the latent distribution. Since the diffusion model is trained with latent codes sampled from the encoder (last paragraph of §B.3), it can at best reproduce the distribution of the encoder latents; if that distribution is unimodal or dominated by encoder noise, the diffusion model cannot recover the claimed diversity of occluded-surface completions. The paper should provide direct evidence that the learned latent space is multi-modal and meaningful: for example, per-scene distributions of decoded samples in occluded regions, pairwise distances between decoded samples, or a metric measuring the coverage of plausible completions against held-out views.
  2. [§4.3, Table 2, Table D1] The only quantitative generative metric is the 20-best oracle evaluation. The reported improvements over single-sample inference are small (Hydrants PSNR 17.74 vs. 17.40; TeddyBears PSNR 17.08 vs. 16.77 on full images) and are compatible with samples being noisy perturbations of a mean prediction. The oracle metric also assumes a selection mechanism and does not measure whether the sample set actually spans distinct plausible 3D structures. The qualitative diversity in Figure D2 is produced by manually increasing the classifier-free guidance weight and decreasing the skip-connection weight, not by unguided sampling from a learned posterior. The paper should report direct diversity statistics--for instance, pairwise LPIPS between samples rendered from novel views, variance maps in occluded regions, or a conditional generative metric such as FID over sample distributions--and should quantify uncertainty via error bars or significance tests across the test set.
  3. [Abstract and §E] The abstract claims that the method can 'complete the occluded surfaces for high-quality 360-degree renderings,' but Section E explicitly concedes that the Splatter Image representation is 'largely constrained to the image frustum of a single view.' Encoding the scene only within the input frustum means that surfaces behind the camera, and large out-of-frustum regions, cannot be represented at all, so the 360-degree wording is misleading. The authors should either qualify the claim to refer to completion within the input frustum or demonstrate that the representation is extended to cover regions beyond the frustum.
  4. [Table 3 and §4.2] In the RealEstate10K comparison, the text states that the method 'achieve[s] performance comparable to DFM,' but in the same 100-scene, 128x128 setting, DFM reports FID 42.84 while the proposed method reports FID 49.83. This is a substantial gap in the generative metric. The comparison with ZeroNVS at 256x256 resolution and on a different number of test scenes is also not apples-to-apples. The claim of state-of-the-art or even comparable performance should be made with a balanced reading of these numbers, ideally supplemented by error bars or a statistical significance analysis over multiple runs.
minor comments (5)
  1. [§3.2, Eq. (1)] The loss weights lambda_1, lambda_2, lambda_3 are used in Eq. (1) but defined only in §B.3; their values should be stated in the main text for readability.
  2. [Fig. 6 caption] The caption mentions LGM and SplatterImage but not the settings used for each baseline; adding a sentence about how object-only baselines are adapted to full scenes would help.
  3. [§D.4] The phrase 'both on1 A100 GPU' appears to be a typo for 'both on one A100 GPU.'
  4. [§3.3, Eq. (4)] The notation 'epsilon_theta(h_t, phi_ref, t)' is slightly awkward; it should clearly separate the time-step embedding from the conditioning features, for example 'epsilon_theta(h_t, t, phi_ref)'.
  5. [Table C1] Some method names are written inconsistently across the text and table, e.g., 'Diffsplat' vs. 'DiffSplat' and 'pixelNeRF' vs. 'PixelNeRF'; please unify the naming.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reconstruction and diversity claims rest on held-out evaluation and do not reduce to their inputs.

full rationale

The derivation chain is self-contained. The Variational AutoReconstructor (Sec. 3.2) is trained with the differentiable-rendering loss of Eq. (1) and the KL term of Eq. (2); neither loss uses the diffusion outputs or the test predictions as an input. The diffusion model (Sec. 3.3, Eq. (4)) is trained on latents sampled from the trained encoder (Sup. B.3) and, at inference, denoises random noise conditioned on input-image features before decoding to a Splatter Image. Evaluation is on held-out CO3D and RealEstate10K sequences against ground-truth novel views (Sec. 4), so the reported PSNR/LPIPS/FID values are not forced by construction. The only self-referential aspect is that the diffusion training targets are the authors' own encoder latents rather than ground-truth 3D; this is a disclosed latent-diffusion design choice and does not make the output equal to the input. The 20-best oracle is explicitly labeled an upper bound, not a sample prediction. Section E's limitation that Splatter Images are constrained to the input frustum qualifies the '360-degree' wording but is a scope restriction, not a circular step. No load-bearing self-citation or ansatz-smuggling-via-citation occurs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

All components of the method rest on standard differentiable-rendering and diffusion assumptions, plus one hand-tuned architectural premise: the 4-channel latent. No new physical or ontological entities are introduced. The learned latent code and Variational AutoReconstructor are model components, not independent entities with falsifiable external handles.

free parameters (6)
  • loss weights lambda1, lambda2, lambda3 = 0.8, 0.2, 0.01
    Hand-set weights for L2, SSIM, and LPIPS in Eq. (1); they control the reconstruction objective for the AutoReconstructor.
  • Gaussians per pixel M = 2
    Chosen number of depth layers per pixel; the ablation tests only one layer versus two.
  • latent channels and resolution = 4 channels at H/8 x W/8
    Bottleneck size for the AutoReconstructor, taken from Stable Diffusion rather than derived from theory.
  • guidance dropout and skip zeroing probabilities = 20% and random zeroing during training
    Hyperparameters for classifier-free guidance and skip-connection regularization; hand-tuned.
  • inference guidance and skip weights = guidance (0.2, 0.5, 0.5) and skip (1.0, 0.5, 0.0) in Fig. D2
    Manual controls for diversity versus fidelity; not optimized.
  • number of diffusion inference steps = 50
    Number of denoising steps used at test time; no analysis of step count versus quality.
assumptions (4)
  • domain assumption Gaussian splatting is an adequate differentiable forward model: rendered images from splats are valid observations of the underlying scene.
    The entire VAR training uses f(X) as image projections; if splat rendering is not a faithful observation model, the latent will not represent true 3D scenes. Invoked in Section 3.2 and Figure 2.
  • domain assumption Camera poses and intrinsics from CO3D and RealEstate10K SfM are accurate and scenes are static during capture.
    Multi-view reprojection losses in Eq. (1) assume known poses and a single static scene; pose errors would corrupt the learned latent. Invoked in Section 4.1.
  • domain assumption A Splatter Image with M=2 Gaussians per pixel and DC-only spherical harmonics can represent the target scenes, including occluded layers.
    The output representation is H x W x MN; the paper relies on the Splatter Image formulation from [61] without proof of coverage for scene-level data. Invoked in Section A.1.
  • ad hoc to paper A 4-channel latent at H/8 resolution with skip connections has enough capacity and smoothness to support diffusion training over full scenes.
    This is an architectural premise validated only empirically by ablations; no theoretical bound or analysis is given. Invoked in Section 3.2 and B.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Complete Gaussian Splats from a Single Image with Denoising Diffusion Models." pith.science (2026). https://pith.science/paper/YCBRUKEH

@misc{pith2026250821542,
  author       = {Pith},
  title        = {Pith review of: Complete Gaussian Splats from a Single Image with Denoising Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YCBRUKEH}},
  note         = {Machine review of arXiv:2508.21542}
}
read the original abstract

Gaussian splatting typically requires dense observations of the scene and can fail to reconstruct occluded and unobserved areas. We propose a latent diffusion model to reconstruct a complete 3D scene with Gaussian splats, including the occluded parts, from only a single image during inference. Completing the unobserved surfaces of a scene is challenging due to the ambiguity of the plausible surfaces. Conventional methods use a regression-based formulation to predict a single "mode" for occluded and out-of-frustum surfaces, leading to blurriness, implausibility, and failure to capture multiple possible explanations. Thus, they often address this problem partially, focusing either on objects isolated from the background, reconstructing only visible surfaces, or failing to extrapolate far from the input views. In contrast, we propose a generative formulation to learn a distribution of 3D representations of Gaussian splats conditioned on a single input image. To address the lack of ground-truth training data, we propose a Variational AutoReconstructor to learn a latent space only from 2D images in a self-supervised manner, over which a diffusion model is trained. Our method generates faithful reconstructions and diverse samples with the ability to complete the occluded surfaces for high-quality 360-degree renderings.

Figures

Figures reproduced from arXiv: 2508.21542 by the authors.

Figure 1
Figure 1. We predict full Gaussian scenes from a single RGB input image. Our diffusion-based model outputs sharper results than existing methods, and is also able to sample diverse comple￾tion “modes” given a single image as input. regions outside the camera frustum. In cases where sparse or even a single view is available, prior knowledge beyond the observations becomes critical for constraining the re￾construction. We aim t… view at source ↗
Figure 2
Figure 2. Learning a latent space for 3D representations using only images, without ground-truth 3D data. (a) Variational Au￾toencoders require groundtruth samples of high-dimensional vari￾ables x to learn a latent space; (b) We propose the Variational AutoReconstructor, which learns a latent space for x using super￾vision from only their projections {m = f(x)}. els the distribution over the latent space. The latent space lea… view at source ↗
Figure 3
Figure 3. Learning a latent space for Splatter Images. Our encoder predicts the parameters of a normal distribution over latents. We reconstruct a sampled latent into H × W × MN Splatter Image representations. We render the Gaussian splats from the viewpoints of the target training images and optimize reprojection losses between the rendered and ground-truth RGB images. Skip connections are critical to preserving the high-fre… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Including skip connections helps preserve high￾frequency details from the input view in the AutoReconstructor, improving the faithfulness of appearance. (Fig. A2, in Sup. Mat). The input image is first encoded by a feature encoder to generate conditioning features. A r…
Figure 6
Figure 6. Figure 6: , our results appear considerably sharper than those of PixelNeRF and SplatterImage, and are comparable to DFM, while being much faster to compute (3 seconds ver￾sus 10 minutes). We output real-world scenes with both foreground and background, in comparison to object-o…
Figure 7
Figure 7. Figure 7: Qualitative Results on the RealEstate10K Dataset. Our method achieves comparable performance to DFM [65], a diffusion-based NeRF model, and performs better in some chal￾lenging regions (highlighted with dotted boxes), while being sig￾nificantly faster at inference time…
Figure 8
Figure 8. Figure 8: 3D Generative Performance. Our diffusion model demonstrates the ability to (1) sample diverse outputs in ambigu￾ous situations, and (2) fill in missing areas using 3D priors learned from large datasets with multi-view consistency. Note the model is trained purely from …

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

84 extracted references · 66 canonical work pages

  1. [1]

    Demystifying MMD GANs

    Mikołaj Bi ´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In ICLR, 2018. 6

    2018

  2. [2]

    SF3D: Stable fast 3D mesh reconstruction with uv-unwrapping and illumination disentanglement

    Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. SF3D: Stable fast 3D mesh reconstruction with uv-unwrapping and illumination disentanglement. In CVPR,

  3. [3]

    6D cam- era relocalization in ambiguous scenes via continuous multi- modal inference

    Mai Bui, Tolga Birdal, Haowen Deng, Shadi Albarqouni, Leonidas Guibas, Slobodan Ilic, and Nassir Navab. 6D cam- era relocalization in ambiguous scenes via continuous multi- modal inference. In ECCV, 2020. 8

    2020

  4. [4]

    pi-GAN: Periodic implicit generative ad- versarial networks for 3D-aware image synthesis

    Eric Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein. pi-GAN: Periodic implicit generative ad- versarial networks for 3D-aware image synthesis. In CVPR,

  5. [5]

    Chan, Connor Z

    Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, and Gordon Wetzstein. Efficient geometry-aware 3D generative adversarial networks. In CVPR, 2022. 16

    work page 2022

  6. [6]

    Chan, Koki Nagano, Matthew A

    Eric R. Chan, Koki Nagano, Matthew A. Chan, Alexan- der W. Bergman, Jeong Joon Park, Axel Levy, Miika Ait- tala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. GeNVS: Generative novel view synthesis with 3D-aware dif- fusion models. In arXiv, 2023. 2, 3, 12

    work page 2023

  7. [7]

    pixelSplat: 3D gaussian splats from im- age pairs for scalable generalizable 3D reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelSplat: 3D gaussian splats from im- age pairs for scalable generalizable 3D reconstruction. In CVPR, 2024. 1, 2, 3, 12

    work page 2024

  8. [8]

    MVSplat: Efficient 3D gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. MVSplat: Efficient 3D gaussian splatting from sparse multi-view images. In ECCV, 2024. 1, 2, 3

    work page 2024

Show all 84 references
  1. [9]

    Text-to-3D using gaussian splatting

    Zilong Chen, Feng Wang, Yikai Wang, and Huaping Liu. Text-to-3D using gaussian splatting. In CVPR, 2024. 2, 3

    2024

  2. [10]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 4

    2017

  3. [11]

    Shape completion using 3D-encoder-predictor cnns and shape synthesis

    Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner. Shape completion using 3D-encoder-predictor cnns and shape synthesis. In CVPR, 2017. 3

    2017

  4. [12]

    ScanComplete: Large- scale scene completion and semantic segmentation for 3D scans

    Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, J¨urgen Sturm, and Matthias Nießner. ScanComplete: Large- scale scene completion and semantic segmentation for 3D scans. In CVPR, 2018

    2018

  5. [13]

    SG-NN: Sparse generative neural networks for self-supervised scene completion of RGB-D scans

    Angela Dai, Christian Diller, and Matthias Nießner. SG-NN: Sparse generative neural networks for self-supervised scene completion of RGB-D scans. In CVPR, 2020. 3

    2020

  6. [14]

    Objaverse: A universe of annotated 3D objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3D objects. In CVPR, 2023. 2

    2023

  7. [15]

    PixelGaussian: gen- eralizable 3d gaussian reconstruction from arbitrary views

    Xin Fei, Wenzhao Zheng, Yueqi Duan, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Jiwen Lu. PixelGaussian: gen- eralizable 3d gaussian reconstruction from arbitrary views. arXiv preprint arXiv:2410.18979, 2024. 3

    2024 arXiv

  8. [16]

    FDGaussian: Fast gaussian splatting from single image via geometric-aware diffusion model

    Qijun Feng, Zhen Xing, Zuxuan Wu, and Yu-Gang Jiang. FDGaussian: Fast gaussian splatting from single image via geometric-aware diffusion model. arXiv, 2024. 3

    2024

  9. [17]

    SC-Diff: 3D shape completion with latent dif- fusion models

    Juan D Galvis, Xingxing Zuo, Simon Schaefer, and Stefan Leutengger. SC-Diff: 3D shape completion with latent dif- fusion models. arXiv, 2024. 3

    2024

  10. [18]

    GET3D: A generative model of high quality 3D tex- tured shapes learned from images

    Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. GET3D: A generative model of high quality 3D tex- tured shapes learned from images. In NeurIPS, 2022. 16

    2022

  11. [19]

    Cat3D: Create anything in 3D with multi-view diffusion models

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3D: Create anything in 3D with multi-view diffusion models. In NeurIPS, 2024. 3

    2024

  12. [20]

    NerfDiff: Single-image view synthesis with nerf-guided dis- tillation from 3D-aware diffusion

    Jiatao Gu, Alex Trevithick, Kai-En Lin, Joshua M Susskind, Christian Theobalt, Lingjie Liu, and Ravi Ramamoorthi. NerfDiff: Single-image view synthesis with nerf-guided dis- tillation from 3D-aware diffusion. In ICML, 2023. 2, 3

    2023

  13. [21]

    Multiple choice learning: Learning to produce multiple structured outputs

    Abner Guzman-Rivera, Dhruv Batra, and Pushmeet Kohli. Multiple choice learning: Learning to produce multiple structured outputs. NeurIPS, 25, 2012. 8

    2012

  14. [22]

    CameraCtrl: En- abling camera control for video diffusion models

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. CameraCtrl: En- abling camera control for video diffusion models. In ICLR,

  15. [23]

    Sampling 3D gaussian scenes in sec- onds with latent diffusion models

    Paul Henderson, Melonie de Almeida, Daniela Ivanova, and Titas Anciukeviˇcius. Sampling 3D gaussian scenes in sec- onds with latent diffusion models. arXiv, 2024. 2, 3, 8, 14

    2024

  16. [24]

    GANs trained by a two time-scale update rule converge to a local nash equi- librium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equi- librium. NeurIPS, 2017. 6

    2017

  17. [25]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 2020. 2, 3, 4, 5

    2020

  18. [26]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, and Tim Sali- mans. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022. 2, 4

    arXiv

  19. [27]

    ViewDiff: 3D-consistent image genera- tion with text-to-image models

    Lukas H ¨ollein, Aljaˇz Boˇziˇc, Norman M¨uller, David Novotny, Hung-Yu Tseng, Christian Richardt, Michael Zollh ¨ofer, and Matthias Nießner. ViewDiff: 3D-consistent image genera- tion with text-to-image models. In CVPR, 2024. 2, 3, 6

    2024

  20. [28]

    PF3plat: Pose-free feed-forward 3d gaussian splatting

    Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jisang Han, Jiaolong Yang, Chong Luo, and Seungryong Kim. PF3plat: Pose-free feed-forward 3d gaussian splatting. arXiv preprint arXiv:2410.22128, 2024. 3

    2024 arXiv

  21. [29]

    ZeroShape: Regression-based zero-shot shape reconstruction

    Zixuan Huang, Stefan Stojanov, Anh Thai, Varun Jampani, and James M Rehg. ZeroShape: Regression-based zero-shot shape reconstruction. In CVPR, 2024. 3

    2024

  22. [30]

    NViST: In the wild new view synthesis from a single image with transformers

    Wonbong Jang and Lourdes Agapito. NViST: In the wild new view synthesis from a single image with transformers. In CVPR, 2024. 3 9

    2024

  23. [31]

    Shap-E: Generating condi- tional 3D implicit functions

    Heewoo Jun and Alex Nichol. Shap-E: Generating condi- tional 3D implicit functions. arXiv, 2023. 2, 3, 4

    2023

  24. [32]

    HoloDiffusion: Training a 3D diffusion model using 2D images

    Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy Mitra. HoloDiffusion: Training a 3D diffusion model using 2D images. In CVPR, 2023. 16

    2023

  25. [33]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In CVPR, 2024. 5, 13

    2024

  26. [34]

    3D Gaussian Splatting for Real-Time Radiance Field Rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ToG, 2023. 1, 3, 12

    2023

  27. [35]

    Auto-encoding varia- tional bayes

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. In ICLR, 2014. 4, 5

    2014

  28. [36]

    Gaus- sianDiffusion: 3D gaussian splatting for denoising diffusion probabilistic models with structured noise

    Xinhai Li, Huaibin Wang, and Kuo-Kun Tseng. Gaus- sianDiffusion: 3D gaussian splatting for denoising diffusion probabilistic models with structured noise. arXiv, 2023. 2, 3

    2023

  29. [37]

    Wonderland: Navi- gating 3d scenes from a single image

    Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos N Pla- taniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Navi- gating 3d scenes from a single image. In CVPR, 2025. 3

    2025

  30. [38]

    Toward general object-level mapping from sparse views with 3D dif- fusion priors

    Ziwei Liao, Binbin Xu, and Steven L Waslander. Toward general object-level mapping from sparse views with 3D dif- fusion priors. In CoRL, 2024. 3

    2024

  31. [39]

    Diffsplat: Repurposing image diffusion models for scalable gaussian splat generation

    Chenguo Lin, Panwang Pan, Bangbang Yang, Zeming Li, and Yadong Mu. Diffsplat: Repurposing image diffusion models for scalable gaussian splat generation. In ICLR,

  32. [40]

    Vision transformer for nerf-based view synthesis from a single input image

    Kai-En Lin, Yen-Chen Lin, Wei-Sheng Lai, Tsung-Yi Lin, Yi-Chang Shih, and Ravi Ramamoorthi. Vision transformer for nerf-based view synthesis from a single input image. In WACV, 2023. 6

    2023

  33. [41]

    NovelGS: Consis- tent novel-view denoising via large gaussian reconstruction model

    Jinpeng Liu, Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Ying Shan, and Yansong Tang. NovelGS: Consis- tent novel-view denoising via large gaussian reconstruction model. arXiv preprint arXiv:2411.16779, 2024. 2

    2024 arXiv

  34. [42]

    Zero-1-to-3: Zero-shot one image to 3D object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3D object. In ICCV, 2023. 3

    2023

  35. [43]

    MonoSplat: Generalizable 3d gaussian splat- ting from monocular depth foundation models

    Yifan Liu, Keyu Fan, Weihao Yu, Chenxin Li, Hao Lu, and Yixuan Yuan. MonoSplat: Generalizable 3d gaussian splat- ting from monocular depth foundation models. In CVPR,

  36. [44]

    You see it, you got it: Learning 3d creation on pose-free videos at scale

    Baorui Ma, Huachen Gao, Haoge Deng, Zhengxiong Luo, Tiejun Huang, Lulu Tang, and Xinlong Wang. You see it, you got it: Learning 3d creation on pose-free videos at scale. In CVPR, pages 2016–2029, 2025. 3

    2016

  37. [45]

    Zero-1-to-G: Taming pretrained 2d diffusion model for direct 3d generation

    Xuyi Meng, Chen Wang, Jiahui Lei, Kostas Daniilidis, Ji- atao Gu, and Lingjie Liu. Zero-1-to-G: Taming pretrained 2d diffusion model for direct 3d generation. arXiv preprint arXiv:2501.05427, 2025. 2, 3, 14

    2025 arXiv

  38. [46]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 1, 2, 3

    2020

  39. [47]

    GSD: View-guided gaussian splatting diffusion for 3D reconstruction

    Yuxuan Mu, Xinxin Zuo, Chuan Guo, Yilin Wang, Juwei Lu, Xiaofeng Wu, Songcen Xu, Peng Dai, Youliang Yan, and Li Cheng. GSD: View-guided gaussian splatting diffusion for 3D reconstruction. In ECCV, 2024. 3, 6

    2024

  40. [48]

    Deepsdf: Learning con- tinuous signed distance functions for shape representation

    Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning con- tinuous signed distance functions for shape representation. In CVPR, 2019. 12

    2019

  41. [49]

    A lesson in splats: Teacher-guided diffusion for 3d gaussian splats generation with 2d supervision

    Chensheng Peng, Ido Sobol, Masayoshi Tomizuka, Kurt Keutzer, Chenfeng Xu, and Or Litany. A lesson in splats: Teacher-guided diffusion for 3d gaussian splats generation with 2d supervision. In ICCV, 2025. 3

    2025

  42. [50]

    DreamFusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. DreamFusion: Text-to-3d using 2d diffusion. In ICLR,

  43. [51]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021. 5

    2021

  44. [52]

    Com- mon objects in 3D: Large-scale learning and evaluation of real-life 3D category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3D: Large-scale learning and evaluation of real-life 3D category reconstruction. In ICCV, 2021. 2, 4, 5, 6

    2021

  45. [53]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2, 3, 4, 5, 12

    2022

  46. [54]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015. 5

    2015

  47. [55]

    ZeroNVS: Zero-shot 360-degree view synthesis from a single image

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry La- gun, Li Fei-Fei, Deqing Sun, and Jiajun Wu. ZeroNVS: Zero-shot 360-degree view synthesis from a single image. In CVPR, 2024. 3, 6, 7, 8, 14

    2024

  48. [56]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR, 2016. 6

    2016

  49. [57]

    Generative gaussian splatting: Generating 3d scenes with video diffusion priors

    Katja Schwarz, Norman Mueller, and Peter Kontschieder. Generative gaussian splatting: Generating 3d scenes with video diffusion priors. In ICCV, 2025. 3

    2025

  50. [58]

    Gamba: Marry gaussian splatting with mamba for single-view 3d recon- struction

    Qiuhong Shen, Zike Wu, Xuanyu Yi, Pan Zhou, Hanwang Zhang, Shuicheng Yan, and Xinchao Wang. Gamba: Marry gaussian splatting with mamba for single-view 3d recon- struction. TPAMI, 2025. 2

    2025

  51. [59]

    Semantic scene comple- tion from a single depth image

    Shuran Song, Fisher Yu, Andy Zeng, Angel X Chang, Mano- lis Savva, and Thomas Funkhouser. Semantic scene comple- tion from a single depth image. In CVPR, 2017. 3

    2017

  52. [60]

    Flash3D: Feed-forward gener- alisable 3D scene reconstruction from a single image

    Stanislaw Szymanowicz, Eldar Insafutdinov, Chuanxia Zheng, Dylan Campbell, Jo ˜ao F Henriques, Christian Rup- precht, and Andrea Vedaldi. Flash3D: Feed-forward gener- alisable 3D scene reconstruction from a single image. arXiv,

  53. [61]

    Splatter image: Ultra-fast single-view 3D recon- struction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3D recon- struction. In CVPR, 2024. 1, 2, 3, 4, 6, 7, 12, 13, 14, 15, 16 10

    2024

  54. [62]

    Bolt3d: Generating 3d scenes in seconds

    Stanislaw Szymanowicz, Jason Y Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Aleksander Holynski, Ricardo Martin-Brualla, Jonathan T Barron, and Philipp Henzler. Bolt3d: Generating 3d scenes in seconds. In ICCV, 2025. 2, 3, 14

    2025

  55. [63]

    DreamGaussian: Generative gaussian splatting for ef- ficient 3D content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. DreamGaussian: Generative gaussian splatting for ef- ficient 3D content creation. In ICLR, 2023. 2, 3

    2023

  56. [64]

    LGM: Large multi-view gaus- sian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. LGM: Large multi-view gaus- sian model for high-resolution 3d content creation. InECCV,

  57. [65]

    Diffusion with forward models: Solv- ing stochastic inverse problems without direct supervision

    Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Josh Tenenbaum, Fr ´edo Durand, Bill Freeman, and Vincent Sitzmann. Diffusion with forward models: Solv- ing stochastic inverse problems without direct supervision. NeurIPS, 2023. 2, 3, 6, 7, 8, 12, 14, 15, 16

    2023

  58. [66]

    VGGT: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. In CVPR, 2025. 3

    2025

  59. [67]

    DUSt3R: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3d vision made easy. In CVPR, 2024. 3

    2024

  60. [68]

    Shape inpainting using 3D generative ad- versarial network and recurrent convolutional networks

    Weiyue Wang, Qiangui Huang, Suya You, Chao Yang, and Ulrich Neumann. Shape inpainting using 3D generative ad- versarial network and recurrent convolutional networks. In ICCV, 2017. 3

    2017

  61. [69]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 2004. 5

    2004

  62. [70]

    latentSplat: Autoencoding variational gaussians for fast generalizable 3D reconstruction

    Christopher Wewer, Kevin Raj, Eddy Ilg, Bernt Schiele, and Jan Eric Lenssen. latentSplat: Autoencoding variational gaussians for fast generalizable 3D reconstruction. InECCV,

  63. [71]

    3D ShapeNets: A deep representation for volumetric shapes

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Lin- guang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3D ShapeNets: A deep representation for volumetric shapes. In CVPR, 2015. 3

    2015

  64. [72]

    Re- purposing 2d diffusion models with gaussian atlas for 3d generation

    Tiange Xiang, Kai Li, Chengjiang Long, Christian H ¨ane, Peihong Guo, Scott Delp, Ehsan Adeli, and Li Fei-Fei. Re- purposing 2d diffusion models with gaussian atlas for 3d generation. In ICCV, 2025. 2, 3, 4, 14

    2025

  65. [73]

    AGG: Amor- tized generative 3D gaussians for single image to 3D.TMLR,

    Dejia Xu, Ye Yuan, Morteza Mardani, Sifei Liu, Jiaming Song, Zhangyang Wang, and Arash Vahdat. AGG: Amor- tized generative 3D gaussians for single image to 3D.TMLR,

  66. [74]

    DepthSplat: Connecting gaussian splatting and depth

    Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Marc Pollefeys. DepthSplat: Connecting gaussian splatting and depth. In CVPR, 2025. 3

    2025

  67. [75]

    GRM: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. GRM: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. In ECCV, 2024. 2, 14, 15

    2024

  68. [76]

    DMV3D: Denoising multi- view diffusion using 3d large reconstruction model

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Ji- ahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, and Kai Zhang. DMV3D: Denoising multi- view diffusion using 3d large reconstruction model. InICLR,

  69. [77]

    Gaussiandreamer: Fast generation from text to 3D gaussians by bridging 2D and 3D diffusion models

    Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3D gaussians by bridging 2D and 3D diffusion models. In CVPR, 2024. 2, 3

    2024

  70. [78]

    pixelNeRF: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelNeRF: Neural radiance fields from one or few images. In CVPR, 2021. 3, 6, 7, 8, 14, 16

    2021

  71. [79]

    GeoLRM: Geometry-aware large recon- struction model for high-quality 3D gaussian generation

    Chubin Zhang, Hongliang Song, Yi Wei, Yu Chen, Jiwen Lu, and Yansong Tang. GeoLRM: Geometry-aware large recon- struction model for high-quality 3D gaussian generation. In NeurIPS, 2024. 2, 3

    2024

  72. [80]

    GS-LRM: Large recon- struction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. GS-LRM: Large recon- struction model for 3d gaussian splatting. In ECCV, 2024. 14, 15

    2024

  73. [81]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 5

    2018

  74. [82]

    Scene splatter: Momentum 3d scene generation from single image with video diffusion model

    Shengjun Zhang, Jinzhao Li, Xin Fei, Hao Liu, and Yueqi Duan. Scene splatter: Momentum 3d scene generation from single image with video diffusion model. In CVPR, 2025. 3

    2025

  75. [83]

    Stereo magnification: Learning view syn- thesis using multiplane images

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view syn- thesis using multiplane images. SIGGRAPH, 2018. 2, 4, 5, 6

    2018

  76. [84]

    Splatter Image

    Zhizhuo Zhou and Shubham Tulsiani. SparseFusion: Dis- tilling view-conditioned diffusion for 3D reconstruction. In CVPR, 2023. 2, 3, 6, 7, 8, 14 11 A. Methods Details A.1. Gaussian Splatting Representation We represent scenes as a set of Gaussian splats [34], and follow Splatt...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.