Pith. sign in

REVIEW 3 major objections 4 minor 67 references

ModeDreamer: Mode Guiding Score Distillation for Text-to-3D Generation using Reference Image Prompts

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that text-to-3D generation can be steered to a chosen visual mode by a reference image, using a new score distillation loss (ISD) that cuts gradient variance and optimization time to about 30–40 minutes.

desk verdict Clever mode-guiding score distillation with a real speedup, but the near-SOTA claim is weakened by tuning on the evaluation benchmark and the control-variate unbiasedness is not established. read the letter →

arxiv 2411.18135 v2 pith:MJMXNNT7 submitted 2024-11-27 cs.CV

Uy Dieu Tran , Minh Luu , Phong Ha Nguyen , Khoi Nguyen , Binh-Son Hua This is my paper · ORCID
classification cs.CV
keywords text-to-3DgenerationscoredistillationsamplingmodeseekingcontrolvariateIP-Adapterreferenceimagepromptmulti-viewregularizationvariancereduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles the mode-seeking instability of score distillation sampling (SDS) in text-to-3D generation. Its central claim is that the diffusion prior can be decomposed as an integral over reference-image-conditioned modes, so conditioning the distillation on a reference image via IP-Adapter guides the optimization to a specific, high-quality mode rather than oscillating between many. To make this practical, the paper introduces the ISD loss, which subtracts the text-only noise prediction from the image-conditioned prediction; because the two share the same text prompt, the text-only score acts as a control variate that lowers gradient variance without training any extra LoRA modules. Combined with MVDream for multi-view consistency, the method reports near state-of-the-art T3Bench quality with the best text-alignment scores, and reduces optimization time to around 30–40 minutes compared to the several hours needed by VSD.

What carries the argument

The central object is the Image prompt Score Distillation (ISD) loss built on IP-Adapter, a decoupled cross-attention adapter that injects a reference-image embedding into a pretrained text-to-image UNet. The load-bearing identity is Eq. (7), where the image-conditioned noise prediction $\epsilon_{IP}$ acts as the guided score and the text-only prediction $\epsilon_{SD}$ acts as a control variate; the claimed correlation between the two (shared text, shared backbone) is what turns naive image-prompt SDS into a lower-variance estimator. A separate multi-view term from MVDream, with its weight decaying over training, stabilizes geometry and counteracts the view bias of the reference image.

What would settle it

For a fixed prompt, render, and timestep, sample many noise seeds and compute the gradient estimators: if the variance of $(\epsilon_{IP} - \epsilon_{SD})$ is not lower than that of $(\epsilon_{IP} - \epsilon)$, or if the empirical mean of $(\epsilon_{IP} - \epsilon_{SD})$ differs substantially from $(\epsilon_{IP} - \epsilon)$, the control variate claim fails. A simpler ablation: run ISD and IP-SDS with identical settings and check whether T3Bench quality and alignment actually improve.

Watch

Extended reading notes

Core claim

The central claim is that the ISD gradient $\nabla_\theta \mathcal{L}_{ISD} = \mathbb{E}[\omega(t)(\epsilon_{IP}(x_t, t, y, x_{ref}) - \epsilon_{SD}(x_t, t, y)) \partial g/\partial \theta]$ both selects the mode suggested by the reference image and reduces variance in the score estimate. The paper argues that $\epsilon_{IP}$ and $\epsilon_{SD}$ are positively correlated because they share the same UNet backbone and text prompt, so the subtraction cancels common noise without biasing the update direction. Empirically, this makes ISD behave like IP-VSD (which trains a LoRA) while being much cheaper, and the multi-view term from MVDream counteracts the Janus artifacts that strong image prompting tends to introduce.

Load-bearing premise

The load-bearing premise is that the text-only UNet noise prediction is a valid control variate for the image-conditioned prediction, meaning subtracting it reduces gradient variance while leaving the optimized objective unchanged; the paper justifies this by positive correlation, not by proving the difference has zero mean.

Editorial extensions

If this is right

  • Supplying a reference image turns text-to-3D optimization into mode-guided generation: one prompt produces many distinct, high-quality 3D objects simply by changing the reference.
  • The control variate removes the need for per-prompt LoRA fine-tuning, cutting optimization time from hours (VSD) to about 30–40 minutes at comparable quality.
  • ISD reports the highest text-alignment scores across all three T3Bench complexity categories, and second-best or best average quality depending on category.
  • The scheduled combination of ISD and MVDream regularization reduces the Janus problem relative to using either component alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The control variate argument is plausible but not proven: the paper shows correlation between $\epsilon_{IP}$ and $\epsilon_{SD}$, not that the difference has zero mean; a formal bias-variance decomposition or a Rao-Blackwell-style averaging argument would settle whether ISD optimizes exactly the intended objective.
  • The method's ceiling is tied to the two fixed priors: if a multi-view model with stronger geometry replaced MVDream, composite-object generation (the paper's acknowledged weak category) should improve.
  • The same subtract-the-text-only-score trick could transfer to other distillation settings, such as single-image-to-3D or video-guided generation, wherever a strong text-conditional counterpart of the image-conditional score is available.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ModeDreamer, a text-to-3D generation method built on a new image-prompt score distillation (ISD) loss. The ISD loss uses an IP-Adapter to condition score distillation on a reference image, thereby selecting a specific mode of the diffusion prior, and subtracts the text-only UNet noise prediction as a control variate to reduce gradient variance. The method combines ISD with MVDream-based multi-view regularization and is evaluated on T3Bench and GPTEval3D, reporting near-state-of-the-art quality with a 40-minute optimization time and improved prompt-alignment scores.

Significance. If the central claims hold, the paper offers a practical and conceptually clean improvement over SDS-based text-to-3D: it shows how to steer generation toward a desired mode with a reference image, and it avoids the expensive LoRA fine-tuning used by VSD while achieving comparable quality. The idea of using the IP-Adapter's image prompt as a mode selector is simple and potentially impactful for downstream applications such as controlled 3D asset generation. The paper also includes qualitative results across diverse prompts and a supplementary GPTEval3D evaluation, which strengthens the empirical case. However, the two load-bearing pillars of the paper—the unbiasedness of the control variate in Eq. (7) and the fairness of the T3Bench comparison—require additional support before the claims can be fully trusted.

major comments (3)
  1. [Section 4.2, Eq. (7)] The control-variate argument is not established. For a control variate to reduce variance without changing the optimized expectation, the subtracted term must have zero (or known) expectation, or the coefficient must be chosen accordingly. The paper only argues that ϵSD is positively correlated with ϵIP, which is a variance-reduction condition, not an unbiasedness condition. Since ϵSD is a learned noise prediction at a finite timestep, E[ϵSD] is not obviously equal to E[ϵ] = 0. The 2D toy experiment in Fig. 3 reports gradient-norm traces but never verifies that the mean gradient of ISD equals that of IP-SDS. Please provide a proof of unbiasedness or revise the claim to describe ISD as a heuristic variance-reduction term whose expected gradient may differ from IP-SDS.
  2. [Section 5.2 and Table 1] The hyperparameters (IP scale 0.5, UNet-at-same-t control variate, CFG scales, and the α/β schedule) are selected using 20 prompts drawn from T3Bench, but the paper does not state that these prompts are excluded from the 300-prompt evaluation reported in Table 1. If the tuning prompts are included in the aggregate, the reported scores are partially in-sample for the method's hyperparameters, whereas the baseline methods' hyperparameters were not tuned on this benchmark. Please either exclude the 20 tuning prompts from Table 1 (and Table 4) or report the results separately for in-sample and out-of-sample prompts.
  3. [Section 4.2, Eqs. (5)-(6)] The derivation of Eq. (6) is internally inconsistent. Eq. (2) defines the SDS gradient as E[ω(t)(ϵϕ(xt,t,y) − ϵ) ∂g/∂θ], but Eq. (6) is written as E[ω(t)ϵθ(xt,t,y) ∂g/∂θ], with the noise term dropped. The surrounding text says the path derivative and parameter score 'sum to zero,' yet also says SDS 'keeps the path derivative (ie. ϵ).' Please correct the derivation so that the relationship among Eqs. (2), (6), and (7) is mathematically coherent; as written, Eq. (6) is not the SDS gradient and the control-variate rationale is unclear.
minor comments (4)
  1. [Tables 2 and 3] All quantitative results are point estimates without error bars or significance tests; given the small differences in some rows (e.g., IP scale 0.2 vs. 0.5 in Table 2), please report standard deviations across prompt subsets or random seeds.
  2. [Section 4.3, Eq. (9)] The α and β schedules are described only qualitatively ('gradually decreases/increases'); please provide the exact schedule or a plot to make the method reproducible.
  3. [Supplementary, Table 4] The GPTEval3D comparison would benefit from a statement about whether the 20 tuning prompts overlap with the 110 GPTEval3D prompts, since hyperparameter tuning on any benchmark subset can affect the reported Elo scores.
  4. [References] There are duplicate references for T3Bench ([10] and [11] are the same paper) and for CSD ([64] and [65] appear to be the same paper); please consolidate.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial evaluation circularity: T3Bench hyperparameters are tuned on a 20-prompt subset of the same benchmark used for the main quality claim; the ISD loss itself is not derived from its own outputs.

  1. fitted input called prediction [Section 5 'Benchmark and metrics'; Section 5.2 'Ablation study'; Table 1]
    "T3bench consists of 300 prompts, categorized into three groups: single object, single object with surroundings, and multiple objects. ... We perform an ablation study using 20 prompts selected from three categories of the T3bench benchmark. ... We select an IP scale of 0.5 to balance reference image alignment with a reduced likelihood of the Janus problem. ... Thus, we select Unet at t as our default control variate, achieving an effective balance of generation speed and 3D quality. ... Quantitative results are presented in Tab. 1."

    The main quantitative claim—near-SOTA T3Bench scores in Table 1—is computed on the full 300-prompt T3Bench set after the method's free parameters (IP scale, control variate, α/β schedule) were selected to maximize quality/alignment on a 20-prompt subset of the same benchmark. The paper never states that the 20 tuning prompts are excluded from the 300-prompt evaluation, so the reported aggregate is partially determined by the same data used for fitting. This is a fitted-input/called-prediction cycle in the evaluation: the T3Bench numbers are not an independent test of the method against baselines whose hyperparameters were not tuned on this benchmark. The core ISD derivation (Eq. 7) is not itself circular, and the supplementary GPTEval3D evaluation provides partly independent support.

full rationale

The derivation chain of the ISD loss is not circular: Eq. (7) is a new combination of an IP-Adapter noise prediction and a text-only UNet prediction, motivated by a marginalization identity (Eq. 3) and an empirical variance-reduction claim, not by fitting a parameter to the target result. The control-variate unbiasedness is asserted rather than proven, but that is a soundness gap, not circularity. The main circular element is in the evaluation: hyperparameters are tuned on a 20-prompt subset of T3Bench and the headline Table 1 scores are reported on the full T3Bench set that includes that subset, so the benchmark comparison is partially self-referential. The paper's self-citation to the authors' DiverseDream is background/comparison only and does not carry the argument. Supplementary GPTEval3D results, an external benchmark, provide independent evidence for the quality claim, which keeps the overall circularity partial rather than total.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a statistical assumption about control variates that is stated but not proven, plus several hand-chosen hyperparameters. No new physical entities are introduced.

free parameters (3)
  • IP scale = 0.5
    Chosen by ablation (Table 2) to balance reference alignment and Janus artifact; central to mode guidance strength.
  • alpha/beta schedule = alpha 0.4 to 0.8, beta 0.6 to 0.02
    Manual schedule in Eq. (9) for balancing ISD and MVDream losses; no principled selection.
  • CFG scales = 7.5 for ISD, 50 for SDS-MVD
    Manual; high CFG for MVDream is reported to mitigate Janus.
assumptions (4)
  • domain assumption The text-to-image prior can be decomposed as pϕ(zt|y) = ∫ pϕ(zt|y,xref) p(xref|y) dxref.
    Eq. (3) in Section 3.2; assumes a well-defined conditional distribution over reference images. Standard probability but used to justify mode selection.
  • domain assumption IP-Adapter's image-prompted noise prediction ϵIP approximates pϕ(zt|y,xref) for SD 1.5.
    Section 4.1; the method relies on IP-Adapter being a faithful conditional prior.
  • ad hoc to paper The text-only UNet prediction ϵSD is an unbiased control variate for ϵIP.
    Section 4.2, Eq. (7); no proof that E[ϵSD] preserves the SDS objective; positive correlation is claimed but not sufficient.
  • domain assumption MVDream provides a multi-view prior that mitigates the Janus problem when combined with ISD.
    Section 4.3; relies on MVDream's cross-view consistency, and that the weighted combination does not introduce view bias.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ModeDreamer: Mode Guiding Score Distillation for Text-to-3D Generation using Reference Image Prompts." pith.science (2026). https://pith.science/paper/MJMXNNT7

@misc{pith2026241118135,
  author       = {Pith},
  title        = {Pith review of: ModeDreamer: Mode Guiding Score Distillation for Text-to-3D Generation using Reference Image Prompts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MJMXNNT7}},
  note         = {Machine review of arXiv:2411.18135}
}
read the original abstract

Existing Score Distillation Sampling (SDS)-based methods have driven significant progress in text-to-3D generation. However, 3D models produced by SDS-based methods tend to exhibit over-smoothing and low-quality outputs. These issues arise from the mode-seeking behavior of current methods, where the scores used to update the model oscillate between multiple modes, resulting in unstable optimization and diminished output quality. To address this problem, we introduce a novel image prompt score distillation loss named ISD, which employs a reference image to direct text-to-3D optimization toward a specific mode. Our ISD loss can be implemented by using IP-Adapter, a lightweight adapter for integrating image prompt capability to a text-to-image diffusion model, as a mode-selection module. A variant of this adapter, when not being prompted by a reference image, can serve as an efficient control variate to reduce variance in score estimates, thereby enhancing both output quality and optimization stability. Our experiments demonstrate that the ISD loss consistently achieves visually coherent, high-quality outputs and improves optimization speed compared to prior text-to-3D methods, as demonstrated through both qualitative and quantitative evaluations on the T3Bench benchmark suite.

Figures

Figures reproduced from arXiv: 2411.18135 by the authors.

Figure 1
Figure 1. Our mode-guiding score distillation using the ISD loss explicitly selects a desired mode captured in a diffusion prior using a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison between vanilla SDS and IP-SDS, IP-SDS [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A 2D toy experiment. Our ISD can generate results [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Different control variate settings including Gaussian noise [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: An overview of our method. Starting with input prompt [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison between our approach and prior methods including MVDream, SDS, and VSD. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: The effect of the IP scale indicates that as the IP scale increases, the generated 3D results align more closely with the reference [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results with 3D Gaussian splatting. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Diverse results of our method when varying the input reference images for each text prompt. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Diverse results of our method when comparing with DiverseDream [ [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Additional qualitative results (1/3). 5 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Additional qualitative results (2/3). 6 [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Additional qualitative results (3/3). 7 [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 46 canonical work pages

  1. [1]

    Eliminating oversaturation and artifacts of high guidance scales in diffusion models

    Anonymous. Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In Submitted to The Thirteenth International Conference on Learning Represen- tations, 2024. under review. 3

    work page 2024

  2. [2]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 4, 7, 8, 2

    work page 2023

  3. [3]

    Vp3d: Unleashing 2d visual prompt for text-to-3d gen- eration

    Yang Chen, Yingwei Pan, Haibo Yang, Ting Yao, and Tao Mei. Vp3d: Unleashing 2d visual prompt for text-to-3d gen- eration. In CVPR, 2024. 1, 4

    work page 2024

  4. [4]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Chris- tian Laforte, Vikram V oleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl V ondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-xl: A universe of 10m+ 3d objects. arXiv preprint arXiv:2307.05663, 2023. 3

    arXiv 2023

  5. [5]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13142–13153, 2023. 3, 8

    work page 2023

  6. [6]

    Text-to-3d generation with bidirectional diffusion using both 2d and 3d priors

    Lihe Ding, Shaocong Dong, Zhanpeng Huang, Zibin Wang, Yiyuan Zhang, Kaixiong Gong, Dan Xu, and Tianfan Xue. Text-to-3d generation with bidirectional diffusion using both 2d and 3d priors. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5115–5124, 2024. 4

    work page 2024

  7. [7]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learn- ing, 2024. 1

    2024

  8. [8]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 3

    2014

Show all 67 references
  1. [9]

    three- studio: A unified framework for 3d content generation, 2023

    Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram V oleti, Guan Luo, Chia-Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Song-Hai Zhang. three- studio: A unified framework for 3d content generation, 2023. 8

    2023

  2. [10]

    T3bench: Benchmarking current progress in text-to-3d gen- eration, 2023

    Yuze He, Yushi Bai, Matthieu Lin, Wang Zhao, Yubin Hu, Jenny Sheng, Ran Yi, Juanzi Li, and Yong-Jin Liu. T3bench: Benchmarking current progress in text-to-3d gen- eration, 2023. 2

    2023

  3. [11]

    T3bench: Benchmarking current progress in text-to-3d gen- eration, 2023

    Yuze He, Yushi Bai, Matthieu Lin, Wang Zhao, Yubin Hu, Jenny Sheng, Ran Yi, Juanzi Li, and Yong-Jin Liu. T3bench: Benchmarking current progress in text-to-3d gen- eration, 2023. 1, 6, 8

    2023

  4. [12]

    LRM: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3d. In The Twelfth International Conference on Learning Representations, 2024. 3

    2024

  5. [13]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InIn- ternational Conference on Learning Representations , 2022. 4

    2022

  6. [14]

    Dreamtime: An improved opti- mization strategy for diffusion-guided 3d generation

    Yukun Huang, Jianan Wang, Yukai Shi, Boshi Tang, Xian- biao Qi, and Lei Zhang. Dreamtime: An improved opti- mization strategy for diffusion-guided 3d generation. In The Twelfth International Conference on Learning Representa- tions, 2024. 4

    2024

  7. [15]

    Zero-shot text-guided object genera- tion with dream fields

    Ajay Jain, Ben Mildenhall, Jonathan T Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object genera- tion with dream fields. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 867–876, 2022. 3

    2022

  8. [16]

    Jointdreamer: Ensur- ing geometry consistency and text congruence in text-to-3d generation via joint score distillation

    Chenhan Jiang, Yihan Zeng, Tianyang Hu, Songcun Xu, Wei Zhang, Hang Xu, and Dit-Yan Yeung. Jointdreamer: Ensur- ing geometry consistency and text congruence in text-to-3d generation via joint score distillation. In European Confer- ence on Computer Vision , pages 439–456. Spri...

    2025

  9. [17]

    Shap-e: Generating condi- tional 3d implicit functions, 2023

    Heewoo Jun and Alex Nichol. Shap-e: Generating condi- tional 3d implicit functions, 2023. 2

    2023

  10. [18]

    Noise-free score distillation

    Oren Katzir, Or Patashnik, Daniel Cohen-Or, and Dani Lischinski. Noise-free score distillation. In The Twelfth In- ternational Conference on Learning Representations , 2024. 4

    2024

  11. [19]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 1, 4

    2023

  12. [20]

    Dream- flow: High-quality text-to-3d generation by approximating probability flow

    Kyungmin Lee, Kihyuk Sohn, and Jinwoo Shin. Dream- flow: High-quality text-to-3d generation by approximating probability flow. In The Twelfth International Conference on Learning Representations, 2024. 4

    2024

  13. [21]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML,

  14. [22]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. In The Twelfth International Conference on Learning Represe...

    2024

  15. [23]

    Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner

    Weiyu Li, Jiarui Liu, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner. arXiv preprint arXiv:2405.14979, 2024. 3 9

    2024 arXiv

  16. [24]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2023. 1, 4, 7, 8, 2

    2023

  17. [25]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per- shape optimization

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per- shape optimization. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 2

    2023

  18. [26]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems , 36, 2024. 3

    2024

  19. [27]

    Stein variational gradient de- scent: A general purpose bayesian inference algorithm

    Qiang Liu and Dilin Wang. Stein variational gradient de- scent: A general purpose bayesian inference algorithm. Ad- vances in neural information processing systems , 29, 2016. 1

    2016

  20. [28]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9298–9309, 2023. 2, 3, 4

    2023

  21. [29]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. In The Twelfth International Conference on Learning Representations, 2024. 3

    2024

  22. [30]

    Att3d: Amortized text-to-3d object synthesis

    Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming- Yu Liu, Sanja Fidler, and James Lucas. Att3d: Amortized text-to-3d object synthesis. The International Conference on Computer Vision (ICCV), 2023. 4

    2023

  23. [31]

    Scalable 3d captioning with pretrained models

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin John- son. Scalable 3d captioning with pretrained models. In Advances in Neural Information Processing Systems , pages 75307–75337. Curran Associates, Inc., 2023. 8

    2023

  24. [32]

    Geodream: Disentan- gling 2d and geometric priors for high-fidelity and consistent 3d generation

    Baorui Ma, Haoge Deng, Junsheng Zhou, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Geodream: Disentan- gling 2d and geometric priors for high-fidelity and consistent 3d generation. arXiv preprint arXiv:2311.17971, 2023. 1, 3, 4, 7, 8

    2023 arXiv

  25. [33]

    Scaledreamer: Scalable text-to- 3d synthesis with asynchronous score distillation

    Zhiyuan Ma, Yuxiang Wei, Yabin Zhang, Xiangyu Zhu, Zhen Lei, and Lei Zhang. Scaledreamer: Scalable text-to- 3d synthesis with asynchronous score distillation. The 18th European Conference on Computer Vision (ECCV), 2024. 3, 4, 5

    2024

  26. [34]

    Realfusion: 360deg reconstruction of any object from a single image

    Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, and Andrea Vedaldi. Realfusion: 360deg reconstruction of any object from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8446–8455, 2023. 3

    2023

  27. [35]

    Latent-nerf for shape-guided generation of 3d shapes and textures

    Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12663–12673, 2023. 1, 4, 7, 8, 2

    2023

  28. [36]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 1, 4

    2020

  29. [37]

    Point-e: A system for generating 3d point clouds from complex prompts, 2022

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts, 2022. 2

    2022

  30. [38]

    Barron, and Ben Milden- hall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference on Learning Representa- tions, 2023. 1, 3, 4, 5, 7, 8, 2

    2023

  31. [39]

    Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to- 3d

    Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mu- tian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong, Liefeng Bo, and Xiaoguang Han. Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to- 3d. In Proceedings of the IEEE/CVF Conference on Com- ...

  32. [40]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In Proceedings of the 38th International Conference on Machine Learning , pages 8821–8831. PMLR, 2021. 3

    2021

  33. [41]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 1

    2022

  34. [42]

    Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo-Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep ...

    2022

  35. [43]

    Laion-5b: An open large-scale dataset for training next generation image-text models, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Lai...

    2022

  36. [44]

    Gamba: Marry gaussian splatting with mamba for single view 3d reconstruc- tion

    Qiuhong Shen, Zike Wu, Xuanyu Yi, Pan Zhou, Hanwang Zhang, Shuicheng Yan, and Xinchao Wang. Gamba: Marry gaussian splatting with mamba for single view 3d reconstruc- tion. arXiv preprint arXiv:2403.18795, 2024. 3

    2024 arXiv

  37. [45]

    Zero123++: a single image to consistent multi-view dif- fusion base model, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model, 2023. 2, 3

    2023

  38. [46]

    MVDream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3d gen- eration. In The Twelfth International Conference on Learn- ing Representations, 2024. 2, 3, 6, 7, 8

    2024

  39. [47]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation, 2024

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation, 2024. 3 10

    2024

  40. [48]

    Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation. InThe Twelfth International Con- ference on Learning Representations, 2024. 4, 7, 8, 2

    2024

  41. [49]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, , Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 3

    2024 arXiv

  42. [50]

    Diverse text-to-3d synthesis with aug- mented text embedding

    Uy Dieu Tran, Minh Luu, Phong Ha Nguyen, Khoi Nguyen, and Binh-Son Hua. Diverse text-to-3d synthesis with aug- mented text embedding. In European Conference on Com- puter Vision, pages 217–235. Springer, 2025. 4, 1

    2025

  43. [51]

    Yeh, and Greg Shakhnarovich

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12619–12629, 202...

    2023

  44. [52]

    Imagedream: Image-prompt multi-view diffusion for 3d generation

    Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation. arXiv preprint arXiv:2312.02201, 2023. 3

    2023 arXiv

  45. [53]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. NeurIPS, 2023. 1, 3, 4, 5, 6, 7, 8, 2

    2023

  46. [54]

    Crm: Single image to 3d textured mesh with convo- lutional reconstruction model

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convo- lutional reconstruction model. In European Conference on Computer Vision, pages 57–74. Springer, 2025. 3

    2025

  47. [55]

    Consistent123: Improve consistency for one image to 3d object synthesis

    Haohan Weng, Tianyu Yang, Jianan Wang, Yu Li, Tong Zhang, CL Chen, and Lei Zhang. Consistent123: Improve consistency for one image to 3d object synthesis. arXiv preprint arXiv:2310.08092, 2023. 3

    2023 arXiv

  48. [56]

    Gpt- 4v(ision) is a human-aligned evaluator for text-to-3d genera- tion

    Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gordon Wetzstein. Gpt- 4v(ision) is a human-aligned evaluator for text-to-3d genera- tion. In CVPR, 2024. 1, 2

    2024

  49. [57]

    Latte3d: Large-scale amortized text-to-enhanced3d synthe- sis

    Kevin Xie, Jonathan Lorraine, Tianshi Cao, Jun Gao, James Lucas, Antonio Torralba, Sanja Fidler, and Xiaohui Zeng. Latte3d: Large-scale amortized text-to-enhanced3d synthe- sis. The 18th European Conference on Computer Vision (ECCV), 2024. 3

    2024

  50. [58]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. In Advances in Neural Information Processing Systems, pages 15903–15935. Curran Associates, I...

    2023

  51. [59]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

    arXiv

  52. [60]

    Flow score dis- tillation for diverse text-to-3d generation

    Runjie Yan, Kailu Wu, and Kaisheng Ma. Flow score dis- tillation for diverse text-to-3d generation. arXiv preprint arXiv:2405.10988, 2024. 4

    2024 arXiv

  53. [61]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  54. [62]

    Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models

    Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models. In CVPR,

  55. [63]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content...

    2022

  56. [65]

    Text-to-3d with classifier score distillation

    Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song- Hai Zhang, and Xiaojuan Qi. Text-to-3d with classifier score distillation. In The Twelfth International Conference on Learning Representations, 2024. 4

    2024

  57. [66]

    Gs-lrm: Large recon- struction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large recon- struction model for 3d gaussian splatting. European Confer- ence on Computer Vision, 2024. 3 11 ModeDreamer: Mode Guiding Score Distillation for Text-to-3D Generat...

    2024

  58. [67]

    Evaluation with GPTEval3D We provide an additional assessment of our methods using GPTEval3D [56], a human-aligned evaluator for text-to-3D generation. Specifically, GPTEval3D evaluates the quality of 3D objects over 110 prompts across five criteria: Text- Asset Alignment, 3D ...

  59. [68]

    We further demonstrate the effectiveness of our method by optimizing a 3D Gaus- sian splatting (3DGS) representation [19]

    More Qualitative Results Qualitative results with 3DGS. We further demonstrate the effectiveness of our method by optimizing a 3D Gaus- sian splatting (3DGS) representation [19]. Our results are shown in Fig. 8, demonstrating that our method generalizes effectively to differen...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.