Pith. sign in

REVIEW 5 major objections 5 minor 69 references

DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DepR reconstructs a 3D scene from a single image by generating each object with depth-conditioned diffusion and assembling the layout, achieving state-of-the-art results on 3D-FRONT.

desk verdict A solid empirical paper that combines depth-conditioned instance diffusion with layout optimization; the object-level SOTA claim needs a careful check of the baseline evaluation protocol before you trust the big numbers. read the letter →

arxiv 2507.22825 v1 pith:GVHGX4SI submitted 2025-07-30 cs.CV

Qingcheng Zhao , Xiang Zhang , Haiyang Xu , Zeyuan Chen , Jianwen Xie , Yuan Gao , Zhuowen Tu This is my paper · ORCID
classification cs.CV
keywords single-viewscenereconstructiondepth-guideddiffusioninstance-levelgenerationcompositional3Dlayoutlatenttri-planemonoculardepthprior3D-FRONT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DepR is a pipeline that turns a single RGB image of an indoor scene into a full 3D reconstruction by generating each object separately with a diffusion model and then fitting the generated objects into a coherent layout. The paper's central claim is that monocular depth should be used as a shape prior throughout training and inference, not only for final pose alignment as in earlier compositional methods. DepR conditions a latent tri-plane diffusion model (a generative model that denoises a compressed three-plane 3D representation) on image features back-projected through the predicted depth, guides the sampling with a scale-invariant depth-rendering loss, and optimizes object poses against the depth-derived point cloud. On the 3D-FRONT benchmark the method reports object-level Chamfer distance $2.57\times10^{-3}$ and F-score 89.66, compared with 20.13 and 27.83 for DeepPriorAssembly, the strongest prior compositional baseline. If correct, this makes depth-conditioned instance diffusion plus depth-guided layout the best currently demonstrated recipe for single-view scene reconstruction.

What carries the argument

The load-bearing machinery is depth-guided conditioning of a latent tri-plane diffusion model. Raw tri-planes fitted to watertight signed distance fields (SDFs) are compressed by a variational autoencoder, and the denoiser receives, as conditioning, image features back-projected through predicted depth into a voxel volume and projected onto the three axis-aligned planes, together with a scene-global cross-attention feature that helps complete occluded regions. A second mechanism is depth-guided DDIM sampling—DDIM (Denoising Diffusion Implicit Model) being a fast deterministic sampler—where a differentiable SDF renderer produces a depth map from the intermediate generated shape and a scale-invariant log-depth loss yields a gradient that adjusts the predicted noise at each step. A third mechanism is layout optimization, which matches the depth-derived partial point cloud to the generated surface by Chamfer distance in 3D and in 2D projection. Together these make the predicted depth a source of geometric supervision in shape generation, not only a final alignment cue.

What would settle it

A direct check is to rerun DepR on a held-out split of 3D-FRONT with the monocular depth estimator swapped for a different one that has different scale characteristics; if object-level F-score stays near 89.66, the method generalizes across depth sources, whereas a large drop would show the reported gains hinge on the specific depth model. The paper's own Table 4 gives a reference point: replacing estimated depth with ground-truth depth raises scene F-score from 24.63 to 26.09, so depth-error sensitivity is measurable in the current setting.

Watch

Extended reading notes

Core claim

DepR's claim is that monocular depth carries strong shape priors that should condition the object reconstruction itself, rather than being reserved for pose estimation at inference time. The paper builds an object-level generative model in which instance masks and image features are back-projected through the estimated depth into a voxel volume, projected onto three orthogonal planes, and concatenated with the noised latent tri-plane of a diffusion U-Net, so the denoiser sees a depth-aligned partial observation enriched with global scene context. At inference, a differentiable signed-distance-field (SDF) renderer converts the intermediate shape into a depth map, and a scale-invariant log-depth loss supplies a gradient that steers the DDIM sampling trajectory; after an initial pose estimate, a second guided sampling pass refines the shape in camera space. The generated objects are then composed by optimizing per-object translation, rotation about the vertical axis, and scale with a Chamfer distance loss in 3D and in 2D projection against the depth-derived partial point cloud. The authors report state-of-the-art results on 3D-FRONT, including object-level CD $2.57\times10^{-3}$ and F-score 89.66 versus 20.13 and 27.83 for DeepPriorAssembly, and demonstrate qualitative generalization to real images.

Load-bearing premise

The method treats the monocular depth map as trusted geometry: any systematic error or scale ambiguity in the depth estimator flows directly into the conditioning volume, the sampling gradient, and the layout point cloud, so the whole reconstruction inherits the depth model's mistakes.

Editorial extensions

If this is right

  • Because depth conditions shape generation at training time, the model can be trained on limited synthetic data yet transfer to real images; the paper shows this qualitatively on Pix3D and self-collected photos.
  • Depth-guided sampling refines generated shapes to match observed depth, and the ablations show it improves alignment of fine details with the input image.
  • The large object-level margin over the prior compositional baseline (F-score 89.66 vs 27.83) indicates that depth-conditioned generation, rather than layout optimization alone, is the main driver of per-object fidelity.
  • With ground-truth layout the scene-level F-score exceeds 67, showing that the optimization-based layout stage—not object generation—is the current bottleneck for full-scene accuracy.
  • Inference takes about 1.2 minutes per scene, with guided sampling and layout optimization contributing most of the runtime, so the gains are achieved at a moderate compute cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the conditioning volume is built only from visible, back-projected surfaces, the model has no explicit mechanism to reason about the unseen backside of an object; a natural extension would be to add category-level symmetry or prior-shape regularization, which the current design leaves implicit.
  • Applying a consistent global scale shift to all estimated depths during inference should shift reconstructed object sizes and poses in a predictable way; measuring that sensitivity would reveal how much of the quality is carried by the depth prior versus the image prior.
  • The two-stage guided sampling effectively couples shape generation with pose estimation, so iterating the guided sampling and layout optimization more than once, or replacing the optimizer with a learned pose regressor, could reduce the 16.1 s layout cost and the local-minima failures shown in the appendix.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes DepR, a compositional single-view scene reconstruction framework that combines an instance-level latent tri-plane diffusion model with depth guidance. Depth from a pretrained estimator is used in three places: to back-project image features into a conditioning volume for object generation (Eq. 10, Sec. 3.2.2), to guide DDIM sampling via a rendered-depth scale-invariant loss (Eqs. 13-15, Sec. 3.2.3), and to provide partial point clouds for layout optimization (Eq. 16, Sec. 3.3). On the 3D-FRONT test set, the paper reports large improvements over existing methods, especially at the object level (CD 2.57e-3, F-score 89.66 vs. DeepPriorAssembly's 20.13 and 27.83), and shows qualitative generalization to real images from Pix3D and collected photos. Ablations decompose the contributions of depth-conditioned denoising, local-global attention, and depth-guided sampling.

Significance. If the empirical claims are confirmed, DepR would be a useful step for single-view scene reconstruction: it demonstrates that injecting depth into the generative object model and using the same depth during layout optimization can improve shape and layout accuracy, and the runtime analysis is a practical asset. The paper also provides an ablation decomposition and clearly identifies layout optimization as the main remaining bottleneck (Table 4). However, the headline 'state-of-the-art' claim is presently supported by a comparison that mixes in-domain training with zero-shot baselines, an underspecified object-level normalization protocol, and single-run point estimates without error bars. The depth-guided sampling contribution, which is one of the three main technical components, shows an effect of only 0.02 F-score in the ablation, so the current evidence does not yet support the claimed benefit. These issues are fixable but require additional experiments and re-framing.

major comments (5)
  1. [Sec. 4.3, Sec. 4.4, Table 1] The object-level evaluation protocol is underspecified. The text says 'both predicted and ground-truth meshes are normalized to a unit sphere,' but it does not state whether each mesh is normalized independently, whether the same fitted scale and center are applied to both, or whether the same normalization was used for all baselines in Table 1. If each mesh is independently normalized, pose and scale information is removed before scoring, so the object-level metrics do not measure scene reconstruction accuracy. Because the object-level gap is the largest claimed gain (CD 2.57 vs. 20.13, F-score 89.66 vs. 27.83), the authors should describe the exact normalization and re-run or re-derive the baseline object-level numbers under an agreed protocol.
  2. [Sec. 4.1, Sec. 4.4] The train/test split appears to be at the scene-image level rather than the object-identity level. DepR trains on 3D-FUTURE tri-planes and is evaluated on 3D-FRONT objects, but the paper does not state that the same 3D-FUTURE object instances are excluded from training when they appear in test scenes. Since 3D-FRONT is built from 3D-FUTURE furniture models, an object may appear in both a training scene and a test scene. This would inflate object-level CD/F-score relative to zero-shot baselines. Please specify whether the object-level evaluation is disjoint at the object-identity level, or provide an evaluation on held-out object identities.
  3. [Sec. 4.4, Table 1] The comparison in Table 1 mixes training regimes: DepR is trained on the 3D-FRONT training split, while DeepPriorAssembly and Gen3DSR are zero-shot methods that use pretrained object reconstruction models without training on 3D-FRONT. Reporting all methods in one 'state-of-the-art' table without labeling this distinction is misleading. The authors should either retrain or fine-tune the baselines on the same target split, or explicitly frame the comparison as in-domain versus zero-shot and temper the SOTA claim accordingly.
  4. [Sec. 4.5, Table 2] The claimed benefit of depth-guided sampling is not supported by the reported numbers. Removing depth-guided sampling changes object-level F-score from 89.66 to 89.64 and CD from 2.57e-3 to 2.61e-3, a difference far smaller than the effect of the other components and likely within stochastic variation. The paper nevertheless states in Sec. 4.5 that guided sampling provides a 'noticeable enhancement.' The authors should report means and standard deviations over multiple seeds for all ablations, and either demonstrate a statistically significant effect or revise the claim to reflect the negligible measured impact.
  5. [Sec. 4.4, Fig. 5] The real-world generalization claim is supported only by qualitative results on Pix3D and collected images. The abstract and conclusion state that DepR 'demonstrates strong generalization' on real-world datasets, but no quantitative real-world metric is reported. Please add a quantitative evaluation on Pix3D or another real-world benchmark, or explicitly scope the contribution to synthetic data with qualitative real-world transfer.
minor comments (5)
  1. [Sec. 4.3] The F-score threshold is given as 0.002, but after unit-sphere normalization the units are ambiguous; please state whether the threshold is applied in normalized units and whether the same threshold was used for all baselines.
  2. [Sec. 3.2.3, Eqs. (13)-(14)] Equation (14) writes the scale-invariant offset as α(D_i, D), but the second argument should be D_i to be consistent with Eq. (13); also clarify that the offset is computed from the same valid-pixel mask m.
  3. [Sec. 3.2.3, Eq. (15)] The paper says the network predicts the clean latent z0, but the guidance update modifies the noise ε_t. Please clarify how ε_t is obtained from the predicted z0 during DDIM sampling, since the parameterization is not specified.
  4. [Sec. 4.5, Tables 2 and 3] The first row of Table 3 appears identical to the 'w/o Depth-guided Sampling' row of Table 2; please confirm these are the same configuration and state this explicitly to avoid confusion.
  5. [Sec. 4.2] The guidance strength λ_guide is set to 100, but no sensitivity analysis is provided; a short study of this hyperparameter would help readers understand its effect on the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular dependency found: DepR's reported results are empirical comparisons against external baselines, with no fitted quantity re-presented as a prediction.

full rationale

DepR is an empirical systems paper: the object and scene reconstruction metrics in Tables 1-4 are computed from generated meshes against ground-truth 3D-FRONT meshes using Chamfer distance and F-score, with the evaluation protocol specified in Sec. 4.3. No derived quantity is defined as the fitted value it later reports; the only hand-set scalar, lambda_guide (Eq. 15), controls the strength of depth guidance and is not a fitted parameter that determines the headline metric by construction. Self-citations (Uni-3D [65], Bayesian diffusion [57]) appear only in related-work context and are not used to certify DepR's own benchmark numbers. The unit-sphere normalization described in Sec. 4.3 could raise a question about whether object-level numbers are comparable across methods, but that is a metric-protocol correctness concern rather than a circularity: the paper does not define the target result in terms of its inputs. The depth-conditioned diffusion objective (Eqs. 10-12), the depth-guided sampling loss (Eq. 13), and the layout Chamfer loss (Eq. 16) are all supervised signals from the observed image/depth and ground-truth geometry, not quantities that reduce to themselves. Accordingly, no step in the claimed derivation chain is circular, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central pipeline leans on two pretrained models (Depth Pro, Grounded-SAM), a pretrained image encoder (DINOv2), and a synthetic training distribution. The hand-tuned guidance strength lambda_guide and SDF rendering beta are the only explicit free scalars. No new physical entities are postulated.

free parameters (2)
  • lambda_guide = 100
    Guidance strength for depth-guided DDIM sampling (Sec. 3.2.3, Sec. 4.2); selected by hand without a reported validation protocol or ablation.
  • SDF-to-density beta = 0.001
    Hyperparameter in the volumetric depth renderer used during guided sampling (Appendix A.2.1); chosen without ablation.
assumptions (4)
  • domain assumption Depth Pro monocular depth predictions are sufficiently accurate metric depth for conditioning, guidance, and layout point clouds.
    Invoked in Secs. 3.2.2, 3.2.3, and 3.3; if depth is systematically biased, all downstream losses are biased. Table 4 shows GT depth improves scene F-score from 24.63 to 26.09, quantifying sensitivity.
  • domain assumption Grounded-SAM instance masks are accurate; missed or incorrect masks directly lose objects.
    Sec. 4.2 uses Grounded-SAM; Table 4 shows replacing learned segmentation with GT segmentation boosts scene F-score from 24.63 to 30.47, and the paper admits segmentation errors degrade composition.
  • domain assumption 3D-FUTURE furniture shapes trained on are representative of test scenes and real-world Pix3D objects.
    Sec. 4.1 trains on 3D-FRONT and 3D-FUTURE; the generalization claim in Sec. 4.4 assumes the synthetic shape prior transfers to real photos.
  • domain assumption Latent tri-plane VAE and diffusion objective faithfully represent 3D geometry as SDFs.
    Adopted from BlockFusion [56], Sec. 3.2.1; if tri-plane capacity is insufficient, reconstructed shapes are limited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion." pith.science (2026). https://pith.science/paper/GVHGX4SI

@misc{pith2026250722825,
  author       = {Pith},
  title        = {Pith review of: DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GVHGX4SI}},
  note         = {Machine review of arXiv:2507.22825}
}
read the original abstract

We propose DepR, a depth-guided single-view scene reconstruction framework that integrates instance-level diffusion within a compositional paradigm. Instead of reconstructing the entire scene holistically, DepR generates individual objects and subsequently composes them into a coherent 3D layout. Unlike previous methods that use depth solely for object layout estimation during inference and therefore fail to fully exploit its rich geometric information, DepR leverages depth throughout both training and inference. Specifically, we introduce depth-guided conditioning to effectively encode shape priors into diffusion models. During inference, depth further guides DDIM sampling and layout optimization, enhancing alignment between the reconstruction and the input image. Despite being trained on limited synthetic data, DepR achieves state-of-the-art performance and demonstrates strong generalization in single-view scene reconstruction, as shown through evaluations on both synthetic and real-world datasets.

Figures

Figures reproduced from arXiv: 2507.22825 by the authors.

Figure 1
Figure 1. Illustration of DepR. Given a single 2D input image, we [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our method. Depth is utilized in three key stages: 1) to back-project features to condition the latent tri-plane [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of depth-guided conditioning. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons the on the 3D-FRONT [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons on real images, including Pix3D [ [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparisons on proposed components. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 42 canonical work pages

  1. [1]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2

    arXiv 2023

  2. [2]

    Depth pro: Sharp monocular metric depth in less than a second

    Alexey Bochkovskiy, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. InThe Thirteenth International Conference on Learning Representations, 2025. 6

    work page 2025

  3. [3]

    Single-view 3d scene reconstruc- tion with high-fidelity shape and texture

    Yixin Chen, Junfeng Ni, Nan Jiang, Yaowei Zhang, Yixin Zhu, and Siyuan Huang. Single-view 3d scene reconstruc- tion with high-fidelity shape and texture. In2024 Interna- tional Conference on 3D Vision (3DV), pages 1456–1467. IEEE, 2024. 2

    work page 2024

  4. [4]

    Comboverse: Compositional 3d as- sets creation using spatially-aware diffusion guidance

    Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, and Ziwei Liu. Comboverse: Compositional 3d as- sets creation using spatially-aware diffusion guidance. In European Conference on Computer Vision, pages 128–146. Springer, 2024. 2

    work page 2024

  5. [5]

    Buol: A bottom-up framework with occupancy-aware lifting for panoptic 3d scene reconstruction from a single image

    Tao Chu, Pan Zhang, Qiong Liu, and Jiaqi Wang. Buol: A bottom-up framework with occupancy-aware lifting for panoptic 3d scene reconstruction from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4937–4946, 2023. 1, 2, 6, 13

    work page 2023

  6. [6]

    Panoptic 3d scene reconstruction from a single rgb image

    Manuel Dahnert, Ji Hou, Matthias Nießner, and Angela Dai. Panoptic 3d scene reconstruction from a single rgb image. Advances in Neural Information Processing Systems, 34: 8282–8293, 2021. 1, 2, 3, 4

    work page 2021

  7. [7]

    Objaverse-xl: A universe of 10m+ 3d objects.Advances in Neural Informa- tion Processing Systems, 36:35799–35813, 2023

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects.Advances in Neural Informa- tion Processing Systems, 36:35799–35813, 2023. 1, 2

    work page 2023

  8. [8]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13142–13153, 2023. 1, 2

    work page 2023

Show all 69 references
  1. [9]

    Gen3DSR: Generalizable 3d scene reconstruction via divide and conquer from a single view

    Andreea Dogaru, Mert ¨Ozer, and Bernhard Egger. Gen3DSR: Generalizable 3d scene reconstruction via divide and conquer from a single view. InInternational Conference on 3D Vision 2025, 2025. 1, 2, 6

    2025

  2. [10]

    Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans

    Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10786–10796, 2021. 2

    2021

  3. [11]

    Depth map prediction from a single image using a multi-scale deep net- work.Advances in neural information processing systems, 27, 2014

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work.Advances in neural information processing systems, 27, 2014. 5

    2014

  4. [12]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 10933–10942,

  5. [13]

    3d-future: 3d fur- niture shape with texture.International Journal of Computer Vision, 129:3313–3337, 2021

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture.International Journal of Computer Vision, 129:3313–3337, 2021. 6

    2021

  6. [14]

    Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024

    Daoyi Gao, D ´avid Rozenberszki, Stefan Leutenegger, and Angela Dai. Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024. 2

    2024

  7. [15]

    Panopticdepth: A unified framework for depth-aware panoptic segmentation

    Naiyu Gao, Fei He, Jian Jia, Yanhu Shan, Haoyang Zhang, Xin Zhao, and Kaiqi Huang. Panopticdepth: A unified framework for depth-aware panoptic segmentation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1632–1642, 2022. 2

    2022

  8. [16]

    Learn- ing 3d object shape and layout without 3d supervision

    Georgia Gkioxari, Nikhila Ravi, and Justin Johnson. Learn- ing 3d object shape and layout without 3d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 1695–1704, 2022. 2

    2022

  9. [17]

    Roca: Ro- bust cad model retrieval and alignment from a single image

    Can G ¨umeli, Angela Dai, and Matthias Nießner. Roca: Ro- bust cad model retrieval and alignment from a single image. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4022–4031, 2022. 2

    2022

  10. [18]

    Reparo: Compositional 3d assets generation with differentiable 3d layout alignment.arXiv preprint arXiv:2405.18525, 2024

    Haonan Han, Rui Yang, Huan Liao, Jiankai Xing, Zun- nan Xu, Xiaoming Yu, Junwei Zha, Xiu Li, and Wan- hua Li. Reparo: Compositional 3d assets generation with differentiable 3d layout alignment.arXiv preprint arXiv:2405.18525, 2024. 2

    2024 arXiv

  11. [19]

    LRM: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3d. InThe Twelfth International Conference on Learning Representations, 2024. 1, 2

    2024

  12. [20]

    Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries

    Junjie Hu, Mete Ozay, Yan Zhang, and Takayuki Okatani. Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries. In2019 IEEE winter conference on applications of computer vision (WACV), pages 1043–1051. IEEE, 2019. 2

    2019

  13. [21]

    Holistic 3d scene parsing and re- construction from a single rgb image

    Siyuan Huang, Siyuan Qi, Yixin Zhu, Yinxue Xiao, Yuanlu Xu, and Song-Chun Zhu. Holistic 3d scene parsing and re- construction from a single rgb image. InProceedings of the European conference on computer vision (ECCV), pages 187–203, 2018. 2

    2018

  14. [22]

    Epidiff: Enhancing multi-view synthesis via localized epipolar-constrained diffusion

    Zehuan Huang, Hao Wen, Junting Dong, Yaohui Wang, Yangguang Li, Xinyuan Chen, Yan-Pei Cao, Ding Liang, Yu Qiao, Bo Dai, et al. Epidiff: Enhancing multi-view synthesis via localized epipolar-constrained diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and ...

    2024

  15. [23]

    Hamid Izadinia, Qi Shan, and Steven M Seitz. Im2cad. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5134–5143, 2017. 2

    2017

  16. [24]

    Shap-e: Generat- ing conditional 3d implicit functions.arXiv preprint arXiv:2305.02463, 2023

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions.arXiv preprint arXiv:2305.02463, 2023. 1, 2

    2023 arXiv

  17. [25]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1,

  18. [26]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 2

    2023

  19. [27]

    Mask2cad: 3d shape prediction by learning to segment and retrieve

    Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, and Angela Dai. Mask2cad: 3d shape prediction by learning to segment and retrieve. InComputer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part III 16, pages 260–277. Springer, 2020. 2

    2020

  20. [28]

    Patch2cad: Patchwise embedding learning for in-the- wild shape retrieval from a single image

    Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, and Angela Dai. Patch2cad: Patchwise embedding learning for in-the- wild shape retrieval from a single image. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 12589–12599, 2021

    2021

  21. [29]

    Sparc: Sparse render-and-compare for cad model alignment in a single rgb image.arXiv preprint arXiv:2210.01044, 2022

    Florian Langer, Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Sparc: Sparse render-and-compare for cad model alignment in a single rgb image.arXiv preprint arXiv:2210.01044, 2022. 2

    2022 arXiv

  22. [30]

    Towards high-fidelity single-view holistic reconstruction of indoor scenes

    Haolin Liu, Yujian Zheng, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Towards high-fidelity single-view holistic reconstruction of indoor scenes. InEuropean Con- ference on Computer Vision, pages 429–446. Springer, 2022. 2, 3, 6, 13

    2022

  23. [31]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion.Advances in Neural Information Processing Systems, 36:22226–22246, 2023

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion.Advances in Neural Information Processing Systems, 36:22226–22246, 2023. 1, 2

    2023

  24. [32]

    One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d dif- fusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d dif- fusion. InProceedings of the IEEE/CVF conference on c...

    2024

  25. [33]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. InProceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 1, 2

    2023

  26. [34]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pages 38–55. Springer...

    2024

  27. [35]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. InThe Twelfth International Conference on Learning Representations, 2024. 2

    2024

  28. [36]

    Wonder3d: Sin- gle image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. InProceed- ings of the IEEE/CVF conference on computer vision and pat...

    2024

  29. [37]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 2

    2021

  30. [38]

    Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models. InInternational Conference on Machine Learning, pa...

    2022

  31. [39]

    Total3dunderstanding: Joint lay- out, object pose and mesh reconstruction for indoor scenes from a single image

    Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. Total3dunderstanding: Joint lay- out, object pose and mesh reconstruction for indoor scenes from a single image. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogn...

    2020

  32. [40]

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael ...

    2024

  33. [41]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. InThe Twelfth Interna- tional Conference on Learning Representations, 2024. 2

    2024

  34. [42]

    Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    2022 arXiv

  35. [43]

    Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159,

    arXiv

  36. [44]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2 10

    2022

  37. [45]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, par...

    2015

  38. [46]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information ...

    2022

  39. [47]

    3d neural field generation using triplane diffusion

    J Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Wetzstein. 3d neural field generation using triplane diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20875–20886, 2023. 2

    2023

  40. [48]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. InInternational Conference on Learning Representations, 2021. 5

    2021

  41. [49]

    Pix3d: Dataset and methods for single-image 3d shape modeling

    Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman. Pix3d: Dataset and methods for single-image 3d shape modeling. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pag...

    2018

  42. [50]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. InEuropean Conference on Computer Vision, pages 1–18. Springer, 2024. 2

    2024

  43. [51]

    Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. InEuropean Conference on Computer ...

    2024

  44. [52]

    Crm: Single image to 3d textured mesh with convo- lutional reconstruction model

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convo- lutional reconstruction model. InEuropean Conference on Computer Vision, pages 57–74. Springer, 2024. 2

    2024

  45. [53]

    Unique3d: High-quality and efficient 3d mesh generation from a single image

    Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, Yueqi Duan, and Kaisheng Ma. Unique3d: High-quality and efficient 3d mesh generation from a single image. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 2

    2024

  46. [54]

    Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer

    Shuang Wu, Youtian Lin, Yifei Zeng, Feihu Zhang, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. InThe Thirty-eighth Annual Conference on Neural Informa- tion Processing Systems, 2024. 2

    2024

  47. [55]

    Casa: Category-agnostic skeletal an- imal reconstruction.Advances in Neural Information Pro- cessing Systems, 35:28559–28574, 2022

    Yuefan Wu, Zeyuan Chen, Shaowei Liu, Zhongzheng Ren, and Shenlong Wang. Casa: Category-agnostic skeletal an- imal reconstruction.Advances in Neural Information Pro- cessing Systems, 35:28559–28574, 2022. 2

    2022

  48. [56]

    Blockfusion: Expandable 3d scene gen- eration using latent tri-plane extrapolation.ACM Transac- tions on Graphics (TOG), 43(4):1–17, 2024

    Zhennan Wu, Yang Li, Han Yan, Taizhang Shang, Weixuan Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato, Hongdong Li, et al. Blockfusion: Expandable 3d scene gen- eration using latent tri-plane extrapolation.ACM Transac- tions on Graphics (TOG), 43(4):1–17, 2024. 2, 3, 4, 5, 13

    2024

  49. [57]

    Bayesian diffusion models for 3d shape reconstruction

    Haiyang Xu, Yu Lei, Zeyuan Chen, Xiang Zhang, Yue Zhao, Yilin Wang, and Zhuowen Tu. Bayesian diffusion models for 3d shape reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10628–10638, 2024. 2

    2024

  50. [58]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models.arXiv preprint arXiv:2404.07191,

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models.arXiv preprint arXiv:2404.07191,

    arXiv

  51. [59]

    Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. InEuropean Con- ference on Computer Vision, pages 1–20. Springer, 2024. 2

    2024

  52. [60]

    Frankenstein: Generating semantic- compositional 3d scenes in one tri-plane

    Han Yan, Yang Li, Zhennan Wu, Shenzhou Chen, Weix- uan Sun, Taizhang Shang, Weizhe Liu, Tian Chen, Xiaqiang Dai, Chao Ma, et al. Frankenstein: Generating semantic- compositional 3d scenes in one tri-plane. InSIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 2, 3

    2024

  53. [61]

    Psdr-room: Single photo to scene using differentiable rendering

    Kai Yan, Fujun Luan, Milo ˇs Ha ˇsan, Thibault Groueix, Valentin Deschaintre, and Shuang Zhao. Psdr-room: Single photo to scene using differentiable rendering. InSIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023. 2

    2023

  54. [62]

    Monosdf: Exploring monocu- lar geometric cues for neural implicit surface reconstruc- tion.Advances in neural information processing systems, 35:25018–25032, 2022

    Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sat- tler, and Andreas Geiger. Monosdf: Exploring monocu- lar geometric cues for neural implicit surface reconstruc- tion.Advances in neural information processing systems, 35:25018–25032, 2022. 5, 13

    2022

  55. [63]

    Holistic 3d scene un- derstanding from a single image with implicit representation

    Cheng Zhang, Zhaopeng Cui, Yinda Zhang, Bing Zeng, Marc Pollefeys, and Shuaicheng Liu. Holistic 3d scene un- derstanding from a single image with implicit representation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8833–8842, 2021. 2

    2021

  56. [64]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets.ACM Transactions on Graphics (TOG), 43(4):1–20, 2024

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets.ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 2

    2024

  57. [65]

    Uni-3d: A universal model for panoptic 3d scene reconstruc- tion

    Xiang Zhang, Zeyuan Chen, Fangyin Wei, and Zhuowen Tu. Uni-3d: A universal model for panoptic 3d scene reconstruc- tion. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 9256–9266, 2023. 1, 2, 3, 4, 6

    2023

  58. [66]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation.Advances in neural information processing systems, 36:73969–73982,

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation.Advances in neural information processing systems, 36:73969–73982,

  59. [67]

    Zero-shot scene reconstruction from single images with deep prior as- sembly.Advances in Neural Information Processing Sys- tems, 37:39104–39127, 2024

    Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Zero-shot scene reconstruction from single images with deep prior as- sembly.Advances in Neural Information Processing Sys- tems, 37:39104–39127, 2024. 1, 2, 3, 5, 6 11

    2024

  60. [68]

    Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers

    Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern ...

    2024

  61. [69]

    hallucinate

    Among compositional methods, DepR consistently pro- duces more visually coherent surfaces and superior overall geometry. A.4. Limitations Fig. A.2 illustrates several failure cases of DepR. Due to its generative nature, DepR may incorrectly reconstruct or “hallucinate” objects...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.