Pith. sign in

REVIEW 3 major objections 6 minor 80 references

Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a two-stage depth diffusion model, which adds simulated scanning noise to clean CAD depth, produces 3D data realistic enough to improve real-world 3D reconstruction and understanding when used for pretraining.

desk verdict A genuinely useful two-stage depth diffusion for sim2real with consistent downstream gains; the core method is solid, but the evaluation needs a missing ablation and a sign fix before publication. read the letter →

arxiv 2507.23483 v1 pith:VGEDWUW6 submitted 2025-07-31 cs.CV

Mutian Xu , Chongjie Ye , Haolin Liu , Yushuang Wu , Jiahao Chang , Xiaoguang Han This is my paper
classification cs.CV
keywords 3Ddatasimulationsim-to-realdepthdiffusionStablefinetuningpointcloudpretrainingresidualgenerationinstancereconstructionnoise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Stable-Sim2Real sets out to solve a practical bottleneck: real 3D training data is expensive and hard to collect, while synthetic CAD data lacks the noise of real scans. The paper argues that instead of hand-coding physical priors for a depth sensor, one should learn the mapping from clean synthetic depth to noisy real depth from a large paired dataset. Its method finetunes a 2D image diffusion model on depth images to generate the residual between CAD and real-captured depth, then runs a second diffusion stage that concentrates on the local regions a 3D classifier marks as unrealistic. The payoff, if the claim holds, is that pretraining 3D networks on such simulated data improves their real-world performance across reconstruction, classification, and segmentation, judged by few-shot and zero-shot evaluations.

What carries the argument

The load-bearing machinery is a two-stage latent diffusion pipeline on depth images. Stage-I finetunes Stable Diffusion with ControlNet-style conditioning to output the residual $D_{\text{res}} = S - R$, added to the CAD depth $S$; the paper argues this is Gaussian-equivalent to modeling real depth directly. A PointNet binary classifier, trained on 128x128 patches of the Stage-I output projected to 3D, labels patches as satisfactory or unsatisfactory. Stage-II diffusion is conditioned on $(D_I, S)$ and trained with the reweighted loss $L = \omega L_{\text{sim}} + \lambda L_{\text{dis}}$ ($\omega = 0.5$, $\lambda = 1.5$), which shifts learning capacity onto the flagged patches. The reweighted loss is what carries the claimed enhancement: it makes the second stage specialize on the distribution of the unsatisfactory regions while preserving already-good areas.

What would settle it

A direct test: on a held-out set of LASA CAD inputs, compute a per-patch geometric error between Stage-I output and the paired real depth (e.g., point-to-surface Chamfer distance). Then check whether the patches flagged as 'unsatisfactory' by the PointNet classifier have significantly higher error than unflagged patches; if the agreement is no better than random, the classifier is not finding the semantically wrong regions. A second test: replace the classifier-chosen patch weighting with weights on randomly chosen patches of the same size; if the Stage-II improvement over Stage-I does not degrade, the specific localization mechanism is not what drives the gain.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that an image-level diffusion foundation model can be repurposed to synthesize the noise of real depth capture, and that the synthesis is improved by explicitly localizing where the first pass fails. Stage-I learns the residual between paired real and CAD depth maps, which the paper shows is theoretically equivalent to learning the real depth distribution when real noise is Gaussian. The resulting depth is stable but has locally unrealistic patches. Stage-II then conditions on both the CAD depth and the Stage-I output and reweights its denoising loss so that patches flagged by a PointNet-based 3D binary classifier receive higher weight. The paper presents this two-stage pipeline as a new direction for data-driven 3D Sim2Real, and supports it with consistent gains over the compared baselines.

Load-bearing premise

The whole Stage-II gain depends on the PointNet classifier correctly identifying which local patches of the Stage-I output would look unrealistic to a real depth sensor; if the classifier is just separating its training distributions by some proxy that does not match true geometric error, the reweighted loss may be working for the wrong reason and the reported gains could be specific to this benchmark setup.

Editorial extensions

If this is right

  • Replacing half of the synthetic pretraining data with Stable-Sim2Real output improves few-shot 3D reconstruction metrics (mIoU, Chamfer L2, F-score) over no-simulated-data and over GAN and single-stage diffusion baselines on LASA categories.
  • With zero-shot evaluation, models pretrained only on simulated data still outperform the same pretraining on any compared baseline, suggesting the simulated data itself carries real-world signal rather than relying on finetuning.
  • The simulated depth transfers beyond the paired dataset: applying the pipeline to ShapeNet CAD models produces realistic-looking depth and point clouds without real ground truth.
  • Direct-similarity measurements and the user study indicate the simulated data approximates real captures (62.9 mIoU vs 59.2 for no-sim on the similarity table; picked-as-real rate 88.6% vs 94.8% for real GT).
  • Adding a third diffusion stage does not improve, and a fourth slightly hurts, the reconstruction result, indicating the two-stage design already captures the identifiable unsatisfactory areas.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Gaussian-equivalence argument for residual generation is the paper's cleanest theoretical claim, but it assumes real capture noise is Gaussian with a single variance; if real noise is sensor-specific or spatially structured, the same residual trick would likely need retraining, which the paper itself flags as a limitation.
  • The benchmark logic—pretrain then few-shot fine-tune—measures simulation quality through downstream task gains; a natural extension the authors do not run is testing whether the advantage persists when 100% of the pretraining data is simulated rather than half, and when the downstream categories are outside LASA's 17 classes.
  • The two-stage recipe is effectively a general clean-to-noisy translation scheme: a coarse generative pass plus a local discriminator that focuses a refinement pass. The paper hints at this for image noise simulation in its Potentials section but leaves open whether the same pattern works for RGB noise, video, or other sensor modalities.
  • Because the discriminator is used only during training, inference cost is just two diffusion passes; a testable simplification is to distill the two stages into a single model with learned per-pixel noise weighting, which would make the method cheaper to deploy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Stable-Sim2Real, a two-stage depth-diffusion pipeline for converting CAD/synthetic depth maps into simulated versions of real-captured noisy depth maps, using the synthetic-real paired LASA dataset. Stage I fine-tunes Stable Diffusion in a ControlNet-style architecture to generate the residual between real and synthetic depth, producing a stable but coarse result. Stage II conditions on both the synthetic depth and the Stage-I output, and re-weights the denoising loss toward local patches judged unsatisfactory by a 3D PointNet-based binary classifier. The simulated depth maps are fused into point clouds and used to pretrain DisCo and OcCo, which are then evaluated on real-world 3D instance reconstruction, classification, and scene segmentation, under few-shot and zero-shot protocols. The paper reports consistent gains over GAN and Stable-Diffusion baselines, direct similarity metrics to real captures, ablations on the table category, and a supplementary user study.

Significance. If the reported gains are reproducible, this is a useful step for the relatively stagnant area of data-driven 3D Sim2Real. The framing of the task as learning to generate real-captured noise from clean synthetic depth, and the use of paired LASA data together with a 2D diffusion foundation model, are sensible and timely. The paper also provides a clear benchmark protocol and compares against several baseline generative methods using their official code, which is a practical strength. However, the empirical support has important gaps: downstream results are single runs without error bars or significance tests, key hyperparameters are tuned on one category before reporting the full tables, and no ablation isolates the contribution of the second diffusion stage from the effect of simply training an additional diffusion pass.

major comments (3)
  1. [Sec. 4.5 and Suppl. C, Tabs. II-IV] All downstream results in Tabs. 1-3 are reported as single numbers with no repeated runs, error bars, or significance tests, while the free hyperparameters of the method (patch size, PointNet width, and loss weights omega/lambda) are selected by tuning on the table-category zero-shot metric before the full tables are computed. This makes the central claim of a "significant" enhancement difficult to evaluate and carries a risk of selection bias. Please provide results over at least three seeds for the main tables, and either use a separate validation category for hyperparameter selection or show that the conclusions are stable across the swept values.
  2. [Sec. 3.4, Eq. (6), and Tab. 5] The ablations do not isolate the proposed Stage-II mechanism from the effect of running a second diffusion pass. In Tab. 5, both "w/o re-w." and "w/o bin." still train a full second-stage diffusion model conditioned on S and Stage-I output; the only difference is the loss weighting or the way patches are selected. Without a control that trains Stage-II with uniform loss on the same conditioning, or a control that runs Stage-I twice or trains Stage-I for twice as long, the observed gains in Tab. 5 could plausibly come from extra model capacity and additional fine-tuning rather than from the 3D discriminator and re-weighted loss. Please add these controls.
  3. [Sec. 3.3 and Sec. 4.5] The evidence that the PointNet discriminator identifies semantically wrong regions is limited to the qualitative illustration in Fig. 6 and the classifier accuracy drop from 67.8% to 32.5%. A drop below the 50% chance level is not by itself evidence that the generated patches match real-captured patterns; a classifier can also be confused by high-frequency or noisier content. Please report precision/recall of the flagged patches against local differences to real ground-truth, or show that re-weighting toward the flagged patches outperforms uniform weighting on the same patches under a metric that does not depend on the discriminator itself.
minor comments (6)
  1. [Sec. 3.2] The sign in Dgt_res = S - R is inconsistent with the surrounding text and with the rest of the paper, which adds the generated residual to the CAD depth to recover the real depth; the definition should be R - S (or the addition should be a subtraction). Please correct this.
  2. [Sec. 3.2] The "theoretically identical" Gaussian argument reduces to the tautology R = S + (R - S) and relies on the idealized assumption that the diffusion model exactly maps N(0,I) to the target residual distribution. It is fine as motivating intuition, but it should be stated as a heuristic rather than a theoretical equivalence.
  3. [Eq. (2)] The displayed backward-diffusion update is malformed: the denominator involving the cumulative product of alpha_t is not written correctly, and the usual notation with \bar{alpha}_t would be much clearer. Please revise Eq. (2) to standard DDPM form.
  4. [Sec. 3.4 and Sec. 3.5] Eq. (5) conditions on (DI_res, S), while Sec. 3.5 and Fig. 2 say the Stage-II model is conditioned on the Stage-I output DI and synthetic depth S. Since DI = DI_res + S, the two are equivalent only when S is known, but the paper should state this explicitly to avoid confusion.
  5. [Sec. 4.4 and Suppl. B] The direct similarity metrics in Tab. 4 are computed on the LASA validation set, which is from the same distribution as the training set on which the simulation models were trained; this should be stated as a limitation of that table. The supplementary user study also has a small sample (17 subjects, 10 samples per method) and lacks confidence intervals or a statistical test.
  6. [Sec. 4.3] The heading contains a typo, "Quatitative results"; it should be "Quantitative results."

Circularity Check

1 steps flagged · score 2.0 of 10

Central claim is validated on held-out real benchmarks; only a minor self-referential discriminator-accuracy check is partly forced by the Stage-II training objective.

  1. fitted input called prediction [Sec. 4.5 ('Analyzing Stage-II model at 3D patch-level'), with Eq. (6) and Fig. 6 caption]
    "When sending the newly generated output patches from Stage-II to our 3D binary discriminator, the classification accuracy notably drops from 67.8% to 32.5%, suggesting that the generated patches successfully approximate the real-captured distributions, making it challenging for the classifier to discriminate. This again verifies the effectiveness of our Stage-II model."

    The same PointNet discriminator produces the per-patch 'unsatisfactory' labels used to re-weight the Stage-II loss in Eq. (6), where L_dis is the loss on 'previously identified distinct (i.e., unsatisfactory) areas.' Training Stage-II to minimize that reweighted loss directly optimizes the generated patches to be less separable by this very discriminator. The subsequent drop in discriminator accuracy from 67.8% to 32.5% is therefore a by-construction consequence of the training objective, not an independent measure of realism. This evidence is secondary; the main claims rest on held-out LASA validation, ScanObjectNN, and S3DIS evaluations that are not produced by this fitted discriminator, so the central derivation is not forced by this step.

full rationale

The paper's primary derivation chain is not circular. Stage-I is trained to generate the depth residual D_gt_res = S - R on LASA training pairs, and the Sec. 3.2 Gaussian argument that S + (R - S) = R is an algebraic identity presented only as motivation for residual learning, not as a source of the downstream gains. The method is evaluated by generating simulated data on the external synthetic datasets used to pretrain DisCo and OcCo, then measuring few-shot and zero-shot performance on held-out real LASA validation data, ScanObjectNN, and S3DIS. These evaluations are independent of the fitted simulation parameters. The use of LASA and DisCo from overlapping authors is ordinary reuse of a published dataset and backbone, and it is not load-bearing in a circular sense: the comparison against Stage-Diffusion, GAN baselines, and 'w/o sim.' settings on external real benchmarks gives the central claim independent content. The one mildly circular element is the reported drop in the 3D binary discriminator's accuracy after Stage-II training, since that discriminator's own labels defined the reweighting in Eq. (6); this is a by-construction consequence and is used only as supporting evidence. Because the central downstream improvements are held-out and externally benchmarked, the overall circularity is minor and non-load-bearing.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the Gaussian/additive noise assumption, the transferability of a LASA-trained model to other object and scene domains, and the reliability of the PointNet patch classifier as a guide for Stage-II reweighting. The loss weights, patch size, classifier width, stage count, and pretraining mixture ratio are all selected by the authors, mostly via table-category zero-shot ablations. No new physical entities are introduced.

free parameters (5)
  • Stage-II loss weights omega and lambda = omega=0.5, lambda=1.5
    Chosen as best on the table-category zero-shot reconstruction metric (Supplement Tab. IV); directly sets the Stage-II loss balance.
  • 3D discriminator patch size = 128x128
    Ablated in Supplement Tab. II; 128x128 gives the best downstream result on the table category.
  • PointNet classifier width = 64
    Ablated in Supplement Tab. III; width 64 gives the best downstream result on the table category.
  • Number of refinement stages = 2
    Chosen over Stage-III and Stage-IV based on the zero-shot table reconstruction ablation (Tab. 5), where extra stages do not improve or slightly hurt.
  • Pretraining mixture ratio = 0.5 simulated plus 0.5 original synthetic
    The paper replaces half of the pretraining point clouds with simulated data without ablating this ratio; the reported downstream gains depend on this choice.
assumptions (4)
  • domain assumption Real-captured depth noise is Gaussian and additive: R ~ N(alpha_R, theta), S ~ N(alpha_S, 0)
    Invoked in Sec. 3.2 to argue residual learning is identical to direct real-depth learning. Real sensor noise is not strictly Gaussian and often contains structured artifacts.
  • domain assumption A PointNet binary classifier trained on local 3D patches can identify distribution boundaries corresponding to visually unsatisfactory regions
    Core to Stage-II; introduced in Sec. 3.3 and used in Sec. 3.4. Only qualitatively validated in Fig. 6 and through the downstream ablation on the table category.
  • domain assumption Stable Diffusion finetuned on LASA depth pairs transfers to other CAD datasets (ShapeNet, ABO, 3D-Future) and to real scans (ScanObjectNN, S3DIS)
    Used to generate pretraining data on non-LASA CAD models in Sec. 4.2 and 4.3. The paper's Limitations section notes retraining may be needed for new domains, which weakens this assumption.
  • domain assumption View-consistency of CAD depth is preserved after adding generated residuals, so depth fusion yields valid 3D data
    Stated in Supplement C: 'the view-variation of the resulting simulated depth maps remains small, making the simulated depth can always be effectively fused into 3D data.' This is asserted rather than demonstrated with fusion error metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion." pith.science (2026). https://pith.science/paper/VGEDWUW6

@misc{pith2026250723483,
  author       = {Pith},
  title        = {Pith review of: Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VGEDWUW6}},
  note         = {Machine review of arXiv:2507.23483}
}
read the original abstract

3D data simulation aims to bridge the gap between simulated and real-captured 3D data, which is a fundamental problem for real-world 3D visual tasks. Most 3D data simulation methods inject predefined physical priors but struggle to capture the full complexity of real data. An optimal approach involves learning an implicit mapping from synthetic to realistic data in a data-driven manner, but progress in this solution has met stagnation in recent studies. This work explores a new solution path of data-driven 3D simulation, called Stable-Sim2Real, based on a novel two-stage depth diffusion model. The initial stage finetunes Stable-Diffusion to generate the residual between the real and synthetic paired depth, producing a stable but coarse depth, where some local regions may deviate from realistic patterns. To enhance this, both the synthetic and initial output depth are fed into a second-stage diffusion, where diffusion loss is adjusted to prioritize these distinct areas identified by a 3D discriminator. We provide a new benchmark scheme to evaluate 3D data simulation methods. Extensive experiments show that training the network with the 3D simulated data derived from our method significantly enhances performance in real-world 3D visual tasks. Moreover, the evaluation demonstrates the high similarity between our 3D simulated data and real-captured patterns. Project page: https://mutianxu.github.io/stable-sim2real/.

Figures

Figures reproduced from arXiv: 2507.23483 by the authors.

Figure 1
Figure 1. We delve back into a substantial yet stagnant problem [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our Stable-Sim2Real. Conditioned on CAD (i.e., synthetic) depth maps, our model first generates the residual (i.e., difference) between real-captured and CAD depth maps. By adding the generated residual to the CAD depth, Stage-I diffusion produces stable but coarse depth maps (Sec. 3.2). Next, the Stage-I output is projected into 3D and sent to a 3D discriminator, to identify the unsatisfactory local … view at source ↗
Figure 3
Figure 3. Qualitative comparison of few-shot evaluation on 3D [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative simulation results on LASA [ [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Qualitative simulation results on ShapeNet [ [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

80 extracted references · 67 canonical work pages

  1. [1]

    Armeni, O

    I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. 3d semantic parsing of large-scale indoor spaces. In CVPR, 2016. 1, 7

    work page 2016

  2. [2]

    Close the sim2real gap via physically-based structured light synthetic data simulation

    Kaixin Bai, Lei Zhang, Zhaopeng Chen, Fang Wan, and Jian- wei Zhang. Close the sim2real gap via physically-based structured light synthetic data simulation. In ICRA, 2024. 1, 3

    work page 2024

  3. [3]

    ARK- itscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Yuri Fei- gin, Peter Fu, Thomas Gebauer, Daniel Kurz, Tal Dimry, Brandon Joffe, Arik Schwartz, and Elad Shulman. ARK- itscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. In NeurIPS, 2021. 5

    work page 2021

  4. [4]

    Unsupervised pixel- level domain adaptation with generative adversarial net- works

    Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. Unsupervised pixel- level domain adaptation with generative adversarial net- works. In CVPR, 2017. 1, 3

    work page 2017

  5. [5]

    Using sim- ulation and domain adaptation to improve efficiency of deep robotic grasping

    Konstantinos Bousmalis, Alex Irpan, Paul Wohlhart, Yunfei Bai, Matthew Kelcey, Mrinal Kalakrishnan, Laura Downs, Julian Ibarz, Peter Pastor, Kurt Konolige, et al. Using sim- ulation and domain adaptation to improve efficiency of deep robotic grasping. In ICRA, 2018. 3

    work page 2018

  6. [6]

    Sim2real: Gen- erative ai to enhance photorealism through domain transfer with gan and seven-chanel-360°-paired-images dataset

    Marc Bresson, Yang Xing, and Weisi Guo. Sim2real: Gen- erative ai to enhance photorealism through domain transfer with gan and seven-chanel-360°-paired-images dataset. Sen- sors, 24(1):94, 2023. 3

    work page 2023

  7. [7]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Hal- ber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. In 3DV, 2017. 1

    work page 2017

  8. [8]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 1, 6, 7, 8, 12, 14

    arXiv 2015

Show all 80 references
  1. [9]

    Generic 3d diffusion adapter using controlled multi-view editing

    Hansheng Chen, Ruoxi Shi, Yulin Liu, Bokui Shen, Ji- ayuan Gu, Gordon Wetzstein, Hao Su, and Leonidas Guibas. Generic 3d diffusion adapter using controlled multi-view editing. arXiv preprint arXiv:2403.12032, 2024. 5

    2024 arXiv

  2. [10]

    Abo: Dataset and benchmarks for real-world 3d object un- derstanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achlesh- war Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object un- derstanding. In CVPR, 2022. 1, 6

    2022

  3. [11]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 1

    2017

  4. [12]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In CVPR, 2023. 1

    2023

  5. [13]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, 2021. 3

    2021

  6. [14]

    Diffusiondepth: Diffusion denoising approach for monocular depth estima- tion

    Yiquan Duan, Xianda Guo, and Zheng Zhu. Diffusiondepth: Diffusion denoising approach for monocular depth estima- tion. In ECCV, 2025. 3

    2025

  7. [15]

    Anygrasp: Robust and efficient grasp perception in spa- tial and temporal domains

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spa- tial and temporal domains. T-RO, 2023. 1

    2023

  8. [16]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In ICCV, 2021. 1

    2021

  9. [17]

    3d-future: 3d fur- niture shape with texture

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture. IJCT, 2021. 6

    2021

  10. [18]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 2020. 1, 3

    2020

  11. [19]

    Coupled real-synthetic domain adaptation for real- world deep depth enhancement

    Xiao Gu, Yao Guo, Fani Deligianni, and Guang-Zhong Yang. Coupled real-synthetic domain adaptation for real- world deep depth enhancement. TIP, 2020. 1, 3

    2020

  12. [20]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 2, 3, 4

    2020

  13. [21]

    Cascaded diffu- sion models for high fidelity image generation

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffu- sion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1–33, 2022. 3

    2022

  14. [22]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In CVPR, 2017. 3, 6, 7, 12, 14

    2017

  15. [23]

    Large scale multi-view stereopsis evalu- ation

    Rasmus Jensen, Anders Dahl, George V ogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evalu- ation. In CVPR, 2014. 1

    2014

  16. [24]

    Bal- anced domain randomization for safe reinforcement learn- ing

    Cheongwoong Kang, Wonjoon Chang, and Jaesik Choi. Bal- anced domain randomization for safe reinforcement learn- ing. Applied Sciences, 14(21):9710, 2024. 3

    2024

  17. [25]

    Scaling up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In CVPR, 2023. 6, 7, 12

    2023

  18. [26]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In CVPR, 2024. 3

    2024

  19. [27]

    Differen- tiable diffusion for dense depth estimation from multi-view images

    Numair Khan, Min H Kim, and James Tompkin. Differen- tiable diffusion for dense depth estimation from multi-view images. In CVPR, 2021. 3

    2021

  20. [28]

    Auto-encoding varia- tional bayes

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. STAT, 1050:1, 2014. 3

    2014

  21. [29]

    Abernethy, James Hays, and Zsolt Kira

    Naveen Kodali, Jacob D. Abernethy, James Hays, and Zsolt Kira. On convergence and stability of gans. arXiv preprint arXiv:1705.07215, 2017. 1, 3

    2017 arXiv

  22. [30]

    Simulating kinect infrared and depth images

    Michael J Landau, Benjamin Y Choo, and Peter A Beling. Simulating kinect infrared and depth images. IEEE transac- tions on cybernetics, 2015. 1, 3

    2015

  23. [31]

    Lasa: Instance reconstruction from real scans using a large-scale aligned shape annotation dataset

    Haolin Liu, Chongjie Ye, Yinyu Nie, Yingfan He, and Xi- aoguang Han. Lasa: Instance reconstruction from real scans using a large-scale aligned shape annotation dataset. In CVPR, 2024. 2, 4, 5, 6, 7, 8, 12, 13, 14, 15

    2024

  24. [32]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2019. 14

    2019 arXiv

  25. [33]

    Fine-grained image editing by pixel-wise guidance using diffusion models

    Naoki Matsunaga, Masato Ishii, Akio Hayakawa, Kenji Suzuki, and Takuya Narihira. Fine-grained image editing by pixel-wise guidance using diffusion models. arXiv preprint arXiv:2212.02024, 2022. 3

    2022 arXiv

  26. [34]

    Sim- ulation of time-of-flight sensors using global illumination

    Stephan Meister, Rahul Nair, and Daniel Kondermann. Sim- ulation of time-of-flight sensors using global illumination. In VMV, 2013. 1, 3

    2013

  27. [35]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. In ICLR, 2021. 3

    2021

  28. [36]

    Dit-3d: Exploring plain diffusion transformers for 3d shape generation

    Shentong Mo, Enze Xie, Ruihang Chu, Lewei Yao, Lanqing Hong, Matthias Nießner, and Zhenguo Li. Dit-3d: Exploring plain diffusion transformers for 3d shape generation. arXiv preprint arXiv: 2307.01831, 2023. 14

    2023 arXiv

  29. [37]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 3

    2021 arXiv

  30. [38]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021. 3

    2021

  31. [39]

    Learning to augment synthetic images for sim2real policy transfer

    Alexander Pashevich, Robin Strudel, Igor Kalevatykh, Ivan Laptev, and Cordelia Schmid. Learning to augment synthetic images for sim2real policy transfer. In IROS, 2019. 3

    2019

  32. [40]

    Pytorch: an im- perative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K ¨opf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu F...

    2019

  33. [41]

    Diffusion- based image translation with label guidance for domain adaptive semantic segmentation

    Duo Peng, Ping Hu, Qiuhong Ke, and Jun Liu. Diffusion- based image translation with label guidance for domain adaptive semantic segmentation. In ICCV, 2023. 3

    2023

  34. [42]

    Depthsynth: Real- time realistic synthetic data generation from cad models for 2.5 d recognition

    Benjamin Planche, Ziyan Wu, Kai Ma, Shanhui Sun, Stefan Kluckner, Oliver Lehmann, Terrence Chen, Andreas Hutter, Sergey Zakharov, Harald Kosch, et al. Depthsynth: Real- time realistic synthetic data generation from cad models for 2.5 d recognition. In 3DV, 2017. 1, 3

    2017

  35. [43]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017. 5, 12, 14

    2017

  36. [44]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021. 4

    2021

  37. [45]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 1, 2, 3, 4, 5, 6, 7, 12, 14

    2022

  38. [46]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2023. 3

    2023

  39. [47]

    Image super- resolution via iterative refinement

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Sali- mans, David J Fleet, and Mohammad Norouzi. Image super- resolution via iterative refinement. TPAMI, 2022. 3

    2022

  40. [48]

    Monocular depth estimation using diffusion models

    Saurabh Saxena, Abhishek Kar, Mohammad Norouzi, and David J Fleet. Monocular depth estimation using diffusion models. arXiv preprint arXiv:2302.14816, 2023. 3

    2023 arXiv

  41. [49]

    The surprising effectiveness of diffusion models for optical flow and monocular depth estimation

    Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, and David J Fleet. The surprising effectiveness of diffusion models for optical flow and monocular depth estimation. NeurIPS, 2024. 3

    2024

  42. [50]

    Dcl: Differential contrastive learning for geometry-aware depth synthesis

    Yuefan Shen, Yanchao Yang, Youyi Zheng, C Karen Liu, and Leonidas J Guibas. Dcl: Differential contrastive learning for geometry-aware depth synthesis. IEEE Robotics and Au- tomation Letters, 2022. 1, 3

    2022

  43. [51]

    Dsr-diff: Depth map super-resolution with diffusion model

    Yuan Shi, Huiyun Cao, Bin Xia, Rui Zhu, Qingmin Liao, and Wenming Yang. Dsr-diff: Depth map super-resolution with diffusion model. Pattern Recognition Letters, 184:225–231,

  44. [52]

    Learning from simulated and unsupervised images through adversarial training

    Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, and Russell Webb. Learning from simulated and unsupervised images through adversarial training. In CVPR, 2017. 3

    2017

  45. [53]

    Weiss, Niru Mah- eswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Mah- eswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML,

  46. [54]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3, 5

    2010 arXiv

  47. [55]

    Lichtenberg, and Jianxiong Xiao

    Shuran Song, Samuel P. Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, 2015. 1

    2015

  48. [56]

    Generative modeling by es- timating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by es- timating gradients of the data distribution. InNeurIPS, 2019. 2, 4

    2019

  49. [57]

    Improved techniques for training score-based generative models

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In NeurIPS, 2020

    2020

  50. [58]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  51. [59]

    A supervised approach to predicting noise in depth images

    Chris Sweeney, Greg Izatt, and Russ Tedrake. A supervised approach to predicting noise in depth images. InICRA, 2019. 1, 3

    2019

  52. [60]

    Domain randomization for transferring deep neural networks from simulation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Woj- ciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In IROS, 2017. 2

    2017

  53. [61]

    Domain randomization and generative models for robotic grasping

    Josh Tobin, Lukas Biewald, Rocky Duan, Marcin Andrychowicz, Ankur Handa, Vikash Kumar, Bob McGrew, Alex Ray, Jonas Schneider, Peter Welinder, et al. Domain randomization and generative models for robotic grasping. In IROS, 2018. 2

    2018

  54. [62]

    Dif- fusion models for monocular depth estimation: Overcoming challenging conditions

    Fabio Tosi, Pierluigi Zama Ramirez, and Matteo Poggi. Dif- fusion models for monocular depth estimation: Overcoming challenging conditions. arXiv preprint arXiv:2407.16698 ,

    arXiv

  55. [63]

    Training deep networks with synthetic data: Bridging the reality gap by do- main randomization

    Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cam- eracci, Shaad Boochoon, and Stan Birchfield. Training deep networks with synthetic data: Bridging the reality gap by do- main randomization. In CVPRW, 2018. 2

    2018

  56. [64]

    Revisiting point cloud classification: A new benchmark dataset and classifi- cation model on real-world data

    Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Duc Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classifi- cation model on real-world data. In ICCV, 2019. 7

    2019

  57. [65]

    Score-based generative modeling in latent space

    Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. In NeurIPS, 2021. 2

    2021

  58. [66]

    Hanchen Wang, Qi Liu, Xiangyu Yue, Joan Lasenby, and Matthew J. Kusner. Unsupervised point cloud pre-training via occlusion completion. In ICCV, 2021. 7

    2021

  59. [67]

    Digging into contrastive learning for robust depth estimation with diffusion models

    Jiyuan Wang, Chunyu Lin, Lang Nie, Kang Liao, Shuwei Shao, and Yao Zhao. Digging into contrastive learning for robust depth estimation with diffusion models. In ACM MM,

  60. [68]

    L2r gan: Lidar-to-radar translation

    Leichen Wang, Bastian Goldluecke, and Carsten Anklam. L2r gan: Lidar-to-radar translation. In ACCV, 2020. 3

    2020

  61. [69]

    Sarma, Michael M

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph cnn for learning on point clouds. TOG, 2019. 7

    2019

  62. [70]

    Mvdd: Multi-view depth diffusion mod- els

    Zhen Wang, Qiangeng Xu, Feitong Tan, Menglei Chai, Shichen Liu, Rohit Pandey, Sean Fanello, Achuta Kadambi, and Yinda Zhang. Mvdd: Multi-view depth diffusion mod- els. In ECCV, 2025. 3

    2025

  63. [71]

    3d shapenets: A deep representation for volumetric shapes

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Lin- guang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, 2015. 1, 7

    2015

  64. [72]

    To- scene: A large-scale dataset for understanding 3d tabletop scenes

    Mutian Xu, Pei Chen, Haolin Liu, and Xiaoguang Han. To- scene: A large-scale dataset for understanding 3d tabletop scenes. In ECCV, 2022. 1

    2022

  65. [73]

    Mvimgnet: A large-scale dataset of multi-view images

    Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Mvimgnet: A large-scale dataset of multi-view images. In CVPR, 2023. 1

    2023

  66. [74]

    Sim-to-real transfer of robotic assembly with visual inputs using cyclegan and force control

    Chengjie Yuan, Yunlei Shi, Qian Feng, Chunyang Chang, Michael Liu, Zhaopeng Chen, Alois Christian Knoll, and Jianwei Zhang. Sim-to-real transfer of robotic assembly with visual inputs using cyclegan and force control. In2022 IEEE International Conference on Robotics and Biomim...

    2022

  67. [75]

    Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data

    Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni-Vincentelli, Kurt Keutzer, and Boqing Gong. Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data. In Proceedings of the IEEE/CVF international conference ...

    2019

  68. [76]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 4, 14

    2023

  69. [77]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 3

    2023

  70. [78]

    Close the optical sensing domain gap by physics- grounded active stereo sensor simulation.IEEE Transactions on Robotics, 2023

    Xiaoshuai Zhang, Rui Chen, Ang Li, Fanbo Xiang, Yuzhe Qin, Jiayuan Gu, Zhan Ling, Minghua Liu, Peiyu Zeng, Songfang Han, Zhiao Huang, Tongzhou Mu, Jing Xu, and Hao Su. Close the optical sensing domain gap by physics- grounded active stereo sensor simulation.IEEE Transactions o...

    2023

  71. [79]

    Betterdepth: Plug-and-play diffu- sion refiner for zero-shot monocular depth estimation

    Xiang Zhang, Bingxin Ke, Hayko Riemenschneider, Nando Metzger, Anton Obukhov, Markus Gross, Konrad Schindler, and Christopher Schroers. Betterdepth: Plug-and-play diffu- sion refiner for zero-shot monocular depth estimation. arXiv preprint arXiv:2407.17952, 2024. 3

    2024 arXiv

  72. [80]

    Rand.” means random perturbation. “w/o sim

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In ICCV, 2017. 3, 6, 7, 12, 14 Supplement for Stable-Sim2Real A . More Qualitative Results 12 A.1 . 3D Instance Reconstruction . . . ...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.