pith. sign in

arxiv: 2401.16764 · v5 · submitted 2024-01-30 · 💻 cs.CV

BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion

Yonghao Yu , Shunan Zhu , Huai Qin , Haorui Li This is my paper

Pith reviewed 2026-05-24 04:34 UTC · model grok-4.3

classification 💻 cs.CV
keywords text-to-3D generationscore distillation samplingmulti-view diffusion3D refinementJanus problemfeed-forward 3D generation
0
0 comments X

The pith

BoostDream refines coarse feed-forward 3D assets into high-quality models by distilling them and applying a multi-view SDS loss guided by consistent normal maps.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents BoostDream as a plug-and-play refining method that takes coarse 3D assets produced quickly by feed-forward generators and upgrades them to high-fidelity results. It works through three steps: fitting a differentiable 3D representation to the initial asset, applying a new multi-view SDS loss drawn from a multi-view aware 2D diffusion model, and using both text prompts and multi-view consistent normal maps for guidance. This combination is shown to run faster than standard SDS methods while avoiding the Janus problem of inconsistent multi-view geometry. A reader would care because current text-to-3D pipelines face a speed-quality tradeoff that this approach aims to resolve by bridging the two dominant paradigms.

Core claim

BoostDream is a highly efficient plug-and-play 3D refining method that transforms coarse 3D assets from feed-forward generation into high-quality ones through 3D model distillation, a novel multi-view SDS loss that draws on a multi-view aware 2D diffusion model, and guidance from prompts together with multi-view consistent normal maps; experiments across differentiable representations demonstrate rapid generation of high-quality assets that overcome the Janus problem compared with conventional SDS-based methods.

What carries the argument

The multi-view SDS loss, which uses signals from a multi-view aware 2D diffusion model to enforce cross-view consistency during refinement of the 3D asset.

If this is right

  • BoostDream can be inserted after any feed-forward text-to-3D generator to upgrade its output without retraining the generator.
  • The same refining pipeline applies across multiple differentiable 3D representations such as NeRF or mesh-based forms.
  • The method produces usable 3D assets in less time than pure SDS optimization while maintaining geometric consistency.
  • Normal-map guidance together with the multi-view loss directly mitigates view-inconsistent artifacts that plague single-view SDS.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Hybrid generation pipelines that always follow a fast feed-forward step with this style of multi-view refinement could become the default workflow for practical 3D asset creation.
  • The reliance on consistent normal maps suggests that supplying additional geometric priors at refinement time may be a general lever for improving distillation-based 3D methods.
  • If the multi-view diffusion model itself improves, the same BoostDream structure could be reused without changing the distillation or guidance components.

Load-bearing premise

The multi-view aware 2D diffusion model supplies consistent refinement signals across different views without introducing new inconsistencies.

What would settle it

Running the full BoostDream pipeline on an already high-quality 3D asset and observing either slower convergence or the emergence of Janus artifacts such as duplicated faces would falsify the claim that the method reliably improves quality and consistency.

Figures

Figures reproduced from arXiv: 2401.16764 by Haorui Li, Huai Qin, Shunan Zhu, Yonghao Yu.

Figure 1
Figure 1. Figure 1: Comparison of 3D Generation Results of baseline and BoostDream. Provided with a coarse 3D asset and text prompt pair, BoostDream can refine it into a high-quality 3D asset efficiently. In each set of images, the image on the left is the coarse 3D asset generated by Shap-E [Jun and Nichol, 2023a], and the three images on the right are our refined 3D asset. refinement and comparison experiments. As shown in … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed BoostDream. BoostDream is a three-stage framework for refining a coarse 3D asset into a high-quality 3D asset. In the initialization stage, we use the feed-forward generation method to get a coarse 3D asset and fit it into differentiable 3D representations to make it trainable. The boost stage is guided by the multi-view normal maps of the coarse 3D asset to ensure stability from t… view at source ↗
Figure 3
Figure 3. Figure 3: The first column is the Shap-E [Jun and Nichol, 2023a] results and the remaining column is the refined results of our method. The results show that BoostDream can refine and edit 3D assets according to different prompts based on input 3D assets. where α and β are the weights for the orientation and opacity losses, respectively. This comprehensive loss function fosters the generation of 3D content that is b… view at source ↗
Figure 4
Figure 4. Figure 4: Comparision with Shap-E [Jun and Nichol, 2023a], DreamFusion [Poole et al., 2022] and Magic3D [Lin et al., 2023] for the same text-to-3D generation task. Our model has significantly stronger prompt relevancy and much better quality (best viewed by zooming in). See the results of our method on DMTet [Shen et al., 2021] and 3D Gaussian Splatting [Kerbl et al., 2023] in the Appendix [Yonghao et al., 2024] [… view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study. Fig(a) is without the initialization stage. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: User study. Our model demonstrates a significant superi [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Simply Combination Ablation Study. The first column is the input coarse model generated by Shap-E [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Control Condition Ablation Study. The first column is the input coarse model generated by Shap-E [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Result on Different 3D Representations.The first column is the input coarse model generated by Shap-E [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
read the original abstract

Witnessing the evolution of text-to-image diffusion models, significant strides have been made in text-to-3D generation. Currently, two primary paradigms dominate the field of text-to-3D: the feed-forward generation solutions, capable of swiftly producing 3D assets but often yielding coarse results, and the Score Distillation Sampling (SDS) based solutions, known for generating high-fidelity 3D assets albeit at a slower pace. The synergistic integration of these methods holds substantial promise for advancing 3D generation techniques. In this paper, we present BoostDream, a highly efficient plug-and-play 3D refining method designed to transform coarse 3D assets into high-quality. The BoostDream framework comprises three distinct processes: (1) We introduce 3D model distillation that fits differentiable representations from the 3D assets obtained through feed-forward generation. (2) A novel multi-view SDS loss is designed, which utilizes a multi-view aware 2D diffusion model to refine the 3D assets. (3) We propose to use prompt and multi-view consistent normal maps as guidance in refinement. Our extensive experiment is conducted on different differentiable 3D representations, revealing that BoostDream excels in generating high-quality 3D assets rapidly, overcoming the Janus problem compared to conventional SDS-based methods. This breakthrough signifies a substantial advancement in both the efficiency and quality of 3D generation processes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript presents BoostDream, a plug-and-play 3D refining framework that converts coarse assets from feed-forward text-to-3D methods into higher-quality outputs. It comprises three processes: (1) distilling differentiable 3D representations from the initial assets, (2) applying a novel multi-view SDS loss that leverages a multi-view aware 2D diffusion model for refinement, and (3) incorporating prompt and multi-view consistent normal maps as additional guidance. The authors claim that experiments across multiple differentiable 3D representations demonstrate rapid generation of high-quality assets that overcome the Janus problem relative to standard SDS baselines.

Significance. If the experimental claims are substantiated with quantitative evidence, the approach could meaningfully advance text-to-3D generation by efficiently combining the speed of feed-forward methods with the fidelity of SDS, while addressing view inconsistency issues through multi-view refinement.

major comments (2)
  1. [Abstract] Abstract, process (2): the headline claim that the multi-view SDS loss overcomes the Janus problem rests on the premise that the multi-view aware 2D diffusion model supplies view-consistent refinement gradients. The abstract provides no description of the model's training procedure, any consistency regularization, or ablation isolating the multi-view SDS term, so it is impossible to verify whether residual view-dependent biases are suppressed or amplified.
  2. [Abstract] Abstract: the assertion of 'extensive experiment' and superiority is unsupported by any reported metrics, ablation tables, baseline comparisons, or error analysis in the provided text, leaving the central empirical claims unverified and the soundness assessment at the level of an untested proposal.
minor comments (1)
  1. [Abstract] Abstract: the sentence 'transform coarse 3D assets into high-quality' is grammatically incomplete and should be revised for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback on our manuscript. We address each major comment below and propose targeted revisions to the abstract.

read point-by-point responses

  1. Referee: [Abstract] Abstract, process (2): the headline claim that the multi-view SDS loss overcomes the Janus problem rests on the premise that the multi-view aware 2D diffusion model supplies view-consistent refinement gradients. The abstract provides no description of the model's training procedure, any consistency regularization, or ablation isolating the multi-view SDS term, so it is impossible to verify whether residual view-dependent biases are suppressed or amplified.

    Authors: We agree that the abstract's brevity omits key details on the multi-view aware 2D diffusion model's training and consistency mechanisms. The full manuscript describes these elements, including training on multi-view data and regularization for view consistency, in the methods section, with isolating ablations in the experiments. We will revise the abstract to briefly note the multi-view consistency aspect of the diffusion model to better support the claim. revision: yes

  2. Referee: [Abstract] Abstract: the assertion of 'extensive experiment' and superiority is unsupported by any reported metrics, ablation tables, baseline comparisons, or error analysis in the provided text, leaving the central empirical claims unverified and the soundness assessment at the level of an untested proposal.

    Authors: The provided text is the abstract, which by convention summarizes outcomes without full quantitative details. The complete manuscript reports metrics, ablation tables, baseline comparisons, and error analysis across multiple 3D representations. We will revise the abstract to more precisely indicate the empirical validation performed, while maintaining standard abstract length. revision: yes

Circularity Check

0 steps flagged

No circularity: method is a plug-and-play composition of existing components with no self-referential derivations.

full rationale

The paper presents BoostDream as three sequential processes—distillation of a differentiable 3D representation, application of a multi-view SDS loss, and guidance via prompt plus normal maps—without any equations, fitted parameters, or uniqueness theorems shown in the provided text. No step reduces a claimed output to an input by construction, no self-citation is invoked as load-bearing justification, and the multi-view SDS term is described as novel rather than derived from the paper's own results. The framework therefore remains self-contained against external benchmarks and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Abstract-only; the method presupposes the existence of a multi-view aware 2D diffusion model and differentiable 3D representations that can be fitted to coarse assets. No free parameters, axioms, or invented entities are explicitly introduced in the provided text.

axioms (2)
  • domain assumption A multi-view aware 2D diffusion model exists and can supply consistent refinement signals across camera views.
    Invoked in the description of the novel multi-view SDS loss (abstract, process 2).
  • domain assumption Coarse feed-forward 3D assets can be represented in a differentiable form suitable for further optimization.
    Stated as the first process of the framework.

pith-pipeline@v0.9.0 · 5796 in / 1518 out tokens · 18784 ms · 2026-05-24T04:34:26.323601+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 43 canonical work pages · 7 internal anchors

  1. [1]

    eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers

    [Balajiet al., 2022 ] Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, et al. ediffi: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324,

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  2. [2]

    A computational approach to edge detection.IEEE Transactions on pattern analysis and machine intelligence, (6):679–698,

    [Canny, 1986] John Canny. A computational approach to edge detection.IEEE Transactions on pattern analysis and machine intelligence, (6):679–698,

    work page 1986

  3. [3]

    Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction.arXiv preprint arXiv:2304.06714,

    [Chenet al., 2023a ] Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, and Hao Su. Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction.arXiv preprint arXiv:2304.06714,

    work page arXiv

  4. [4]

    Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation

    [Chenet al., 2023b ] Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. arXiv preprint arXiv:2303.13873,

    work page arXiv

  5. [5]

    Objaverse-XL: A Universe of 10M+ 3D Objects

    [Deitkeet al., 2023 ] Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. arXiv preprint arXiv:2307.05663,

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    threestudio: A unified frame- work for 3d content generation

    [Guoet al., 2023 ] Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram V oleti, Guan Luo, Chia- Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Song-Hai Zhang. threestudio: A unified frame- work for 3d content generation. https://github.com/ threestudio-project/threestudio,

    work page 2023

  7. [7]

    3dgen: Triplane la- tent diffusion for textured mesh generation.arXiv preprint arXiv:2303.05371,

    [Guptaet al., 2023 ] Anchit Gupta, Wenhan Xiong, Yixin Nie, Ian Jones, and Barlas O ˘guz. 3dgen: Triplane la- tent diffusion for textured mesh generation.arXiv preprint arXiv:2303.05371,

    work page arXiv 2023

  8. [8]

    Classifier-free diffusion guidance,

    [Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance,

    work page 2022

  9. [9]

    Denoising diffusion probabilistic models

    [Hoet al., 2020 ] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors,Advances in Neural Information Process- ing Systems, volume 33, pages 6840–6851. Curran Asso- ciates, Inc.,

    work page 2020

  10. [10]

    Zero-shot text- guided object generation with dream fields

    [Jainet al., 2022 ] Ajay Jain, Ben Mildenhall, Jonathan T Barron, Pieter Abbeel, and Ben Poole. Zero-shot text- guided object generation with dream fields. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 867–876,

    work page 2022

  11. [11]

    Shap-E: Generating Conditional 3D Implicit Functions

    [Jun and Nichol, 2023a] Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463,

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    3d gaus- sian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July

    [Kerblet al., 2023 ] Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaus- sian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July

    work page 2023

  13. [13]

    Adam: A Method for Stochastic Optimization

    [Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  14. [14]

    Magic3d: High-resolution text-to-3d content creation

    [Linet al., 2023 ] Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 300–309,

    work page 2023

  15. [15]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization, 2023

    [Liuet al., 2023a ] Minghua Liu, Chao Xu, Haian Jin, Ling- hao Chen, Zexiang Xu, Hao Su, et al. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization.arXiv preprint arXiv:2306.16928,

    work page arXiv

  16. [16]

    Meshdiffusion: Score-based generative 3d mesh model- ing.arXiv preprint arXiv:2303.08133,

    [Liuet al., 2023c ] Zhen Liu, Yao Feng, Michael J Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. Meshdiffusion: Score-based generative 3d mesh model- ing.arXiv preprint arXiv:2303.08133,

    work page arXiv

  17. [17]

    Att3d: Amortized text-to-3d object synthe- sis.arXiv preprint arXiv:2306.07349,

    [Lorraineet al., 2023 ] Jonathan Lorraine, Kevin Xie, Xiao- hui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming-Yu Liu, Sanja Fidler, and James Lucas. Att3d: Amortized text-to-3d object synthe- sis.arXiv preprint arXiv:2306.07349,

    work page arXiv 2023

  18. [18]

    Realfusion: 360deg reconstruction of any object from a single image

    [Melas-Kyriaziet al., 2023 ] Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, and Andrea Vedaldi. Realfusion: 360deg reconstruction of any object from a single image. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8446–8455,

    work page 2023

  19. [19]

    Latent-nerf for shape-guided generation of 3d shapes and textures

    [Metzeret al., 2023 ] Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12663–12673,

    work page 2023

  20. [20]

    Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106,

    [Mildenhallet al., 2021 ] Ben Mildenhall, Pratul P Srini- vasan, Matthew Tancik, Jonathan T Barron, Ravi Ra- mamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106,

    work page 2021

  21. [21]

    Point-E: A System for Generating 3D Point Clouds from Complex Prompts

    [Nicholet al., 2022 ] Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751,

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  22. [22]

    Autodecoding latent 3d diffusion mod- els.arXiv preprint arXiv:2307.05445,

    [Ntaveliset al., 2023 ] Evangelos Ntavelis, Aliaksandr Siaro- hin, Kyle Olszewski, Chaoyang Wang, Luc Van Gool, and Sergey Tulyakov. Autodecoding latent 3d diffusion mod- els.arXiv preprint arXiv:2307.05445,

    work page arXiv 2023

  23. [23]

    DreamFusion: Text-to-3D using 2D Diffusion

    [Pooleet al., 2022 ] Ben Poole, Ajay Jain, Jonathan T Bar- ron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988,

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  24. [24]

    Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors.arXiv preprint arXiv:2306.17843,

    [Qianet al., 2023 ] Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, et al. Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors.arXiv preprint arXiv:2306.17843,

    work page arXiv 2023

  25. [25]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE transactions on pat- tern analysis and machine intelligence, 44(3):1623–1637,

    [Ranftlet al., 2020 ] Ren´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE transactions on pat- tern analysis and machine intelligence, 44(3):1623–1637,

    work page 2020

  26. [26]

    High-resolution image synthesis with latent diffusion models

    [Rombachet al., 2022 ] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695,

    work page 2022

  27. [27]

    Photorealistic text-to-image diffusion models with deep language un- derstanding.Advances in Neural Information Processing Systems, 35:36479–36494,

    [Sahariaet al., 2022 ] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language un- derstanding.Advances in Neural Information Processing Systems, 35:36479–36494,

    work page 2022

  28. [28]

    Clip-sculptor: Zero- shot generation of high-fidelity and diverse shapes from natural language

    [Sanghiet al., 2023 ] Aditya Sanghi, Rao Fu, Vivian Liu, Karl DD Willis, Hooman Shayani, Amir H Khasahmadi, Srinath Sridhar, and Daniel Ritchie. Clip-sculptor: Zero- shot generation of high-fidelity and diverse shapes from natural language. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 18339–18348,

    work page 2023

  29. [29]

    Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in Neural Information Processing Sys- tems, 35:25278–25294,

    [Schuhmannet al., 2022 ] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wight- man, Mehdi Cherti, Theo Coombes, Aarush Katta, Clay- ton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in Neural Information Processing Sys- tems, 35:25278–25294,

    work page 2022

  30. [30]

    Deep marching tetrahe- dra: a hybrid representation for high-resolution 3d shape synthesis

    [Shenet al., 2021 ] Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, and Sanja Fidler. Deep marching tetrahe- dra: a hybrid representation for high-resolution 3d shape synthesis. InAdvances in Neural Information Processing Systems (NeurIPS),

    work page 2021

  31. [31]

    MVDream: Multi-view Diffusion for 3D Generation

    [Shiet al., 2023 ] Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation.arXiv preprint arXiv:2308.16512,

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  32. [32]

    Deepfloyd

    [StabilityAI, 2023] StabilityAI. Deepfloyd. https:// huggingface.co/DeepFloyd,

    work page 2023

  33. [33]

    Mvdif- fusion: Enabling holistic multi-view image generation with correspondence-aware diffusion.arXiv preprint arXiv:2307.01097,

    [Tanget al., 2023 ] Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. Mvdif- fusion: Enabling holistic multi-view image generation with correspondence-aware diffusion.arXiv preprint arXiv:2307.01097,

    work page arXiv 2023

  34. [34]

    Neus: Learning neural implicit surfaces by volume ren- dering for multi-view reconstruction.NeurIPS,

    [Wanget al., 2021 ] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume ren- dering for multi-view reconstruction.NeurIPS,

    work page 2021

  35. [35]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    [Wanget al., 2023b ] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lificdreamer: High-fidelity and diverse text-to-3d gener- ation with variational score distillation.arXiv preprint arXiv:2305.16213,

    work page arXiv

  36. [36]

    Boostdream: Efficient refining for high- quality text-to-3d generation from multi-view diffusion

    [Yonghaoet al., 2024 ] Yu Yonghao, Zhu Shunan, Qin Huai, and Li Haorui. Boostdream: Efficient refining for high- quality text-to-3d generation from multi-view diffusion. https://boostdream.github.io/,

    work page 2024

  37. [37]

    Lion: Latent point diffusion models for 3d shape generation.arXiv preprint arXiv:2210.06978,

    [Zenget al., 2022 ] Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation.arXiv preprint arXiv:2210.06978,

    work page arXiv 2022

  38. [38]

    Efficientdreamer: High-fidelity and ro- bust 3d creation via orthogonal-view diffusion prior.arXiv preprint arXiv:2308.13223,

    [Zhaoet al., 2023 ] Minda Zhao, Chaoyi Zhao, Xinyue Liang, Lincheng Li, Zeng Zhao, Zhipeng Hu, Changjie Fan, and Xin Yu. Efficientdreamer: High-fidelity and ro- bust 3d creation via orthogonal-view diffusion prior.arXiv preprint arXiv:2308.13223,

    work page arXiv 2023

  39. [39]

    3d shape generation and completion through point-voxel diffusion

    [Zhouet al., 2021 ] Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through point-voxel diffusion. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5826–5835,

    work page 2021

  40. [40]

    Appendix A Implementation Details In the training process of the NeRF [Mildenhallet al., 2021 ] setting, we set rendering resolution to128×128, and batch size to

    work page 2021

  41. [41]

    , 3d asset

    We apply our random multi-view render system to capture a combined image with four sub-images with rotation angleαset to90 ◦. We use AdamW optimizer [Kingma and Ba, 2014] with learning rate1×10 −2 and1×10 −3 for geom- etry and background modeling. The background is replaced with random colors with80%of chance. In the DMTet [Shen et al., 2021] setting, mos...

    work page 2014

  42. [42]

    C Control Condition Ablation Study We also test our method with different multi-view control conditions replacing the normal map

    We can see in the first row even with the proper initialization, DreamFusion still suffers from the Janus problem and has coarse results compared to our BoostDream results. C Control Condition Ablation Study We also test our method with different multi-view control conditions replacing the normal map. We choose canny edge [Canny, 1986] and depth map [Ranf...

    work page 1986

  43. [43]

    Intu- itively, it is unsuitable as a control condition when generating high-quality 3D assets

    Canny edge just contains the edge information of the 3D asset. Intu- itively, it is unsuitable as a control condition when generating high-quality 3D assets. The results also illustrate this point: when using canny edge as the control condition, the 3D asset suffers from incomplete generation. Especially in the second row, the bear turns out to be unnatur...

    work page 2021