Pith. sign in

REVIEW 4 major objections 5 minor 41 references

LTM3D: Bridging Token Spaces for Conditional 3D Generation with Auto-Regressive Diffusion Framework

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LTM3D combines auto-regressive and diffusion modeling to generate 3D shapes from images and text, outperforming prior state-of-the-art methods on ShapeNet and Objaverse.

desk verdict A sensible MAR-style autoregressive diffusion framework for 3D generation, with solid experiments, but the state-of-the-art claim leans on an unverifiable self-reimplementation of CLAY. read the letter →

arxiv 2505.24245 v1 pith:6XTUCQNZ submitted 2025-05-30 cs.CV cs.AI

Xin Kang , Zihan Zheng , Lei Chu , Yue Gao , Jiahao Li , Hao Pan , Xuejin Chen , Yan Lu This is my paper
classification cs.CVcs.AI
keywords conditional3Dgenerationlatenttokenmodelingauto-regressivediffusionmaskedautoencoderprefixlearningreconstruction-guidedsamplingmulti-representationShapeNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces LTM3D, a framework for conditional 3D shape generation that combines auto-regressive token dependency modeling with diffusion-based distribution modeling. The central claim is that factorizing the joint conditional distribution over shape latent tokens into per-token conditional probabilities, learned by a masked auto-encoder and an MLP-based denoiser, captures both global structure and fine detail better than pure diffusion or pure auto-regressive models. Two additional components—Prefix Learning, which aligns condition tokens with shape tokens during generation, and a Latent Token Reconstruction module with Reconstruction-Guided Sampling, which reduces token uncertainty—improve prompt fidelity and structural accuracy. The paper reports state-of-the-art results on image- and text-conditioned generation on ShapeNet and Objaverse across SDF, point cloud, mesh, and 3D Gaussian Splatting representations. If correct, LTM3D offers a general token-space recipe for conditional 3D generation that is decoupled from the shape encoder and adaptable to new condition modalities.

What carries the argument

The central machinery is the conditional distribution modeling backbone, which combines a Masked Auto-Encoder and an MLP-based DenoiseNet in the style of masked autoregressive diffusion. The MAE computes dependency vectors $z_i = \text{MAE}(x_{<i}, T)$ for each token given the preceding tokens and condition, and the DenoiseNet predicts the noise $\epsilon$ for token $x_i$ at diffusion time step $t$, optimized by $\mathbb{E}_{\epsilon,t}[\|\epsilon - \epsilon_\theta(x_i|t,z_i)\|^2]$. Prefix Learning generates prefix tokens $T = \text{FF}(\text{CrossAttn}(Q,C)+Q)$ from learnable queries $Q$ and condition tokens $C$, aligning the condition to the shape token space during generation. Latent Token Reconstruction produces reconstructed tokens $\hat{X} = \text{SelfAttn}(\text{CrossAttn}(Q_S,C)+Q_S)$, which are linearly blended with sampled tokens during the first 30 sampling steps to guide the MAE conditioning.

What would settle it

Run the same evaluation protocol with the official CLAY model (released weights or a faithful reproduction) and compare CD, EMD, F-Score, and P-FID on the same ShapeNet and Objaverse splits; if official CLAY matches or exceeds LTM3D, the paper's state-of-the-art claim fails. A second test: disable Reconstruction-Guided Sampling entirely (set $\alpha_i=1$ for all steps) and measure whether the metric gap persists, isolating the module's contribution.

Watch

Extended reading notes

Core claim

LTM3D claims that the joint conditional distribution $p(X|T)$ over shape latent tokens can be approximated by the product $\prod_i p(x_i | x_{<i}, T)$, where each per-token conditional distribution is computed by a masked auto-encoder that encodes the preceding tokens and the condition into $z_i$, and a diffusion denoiser that models $p(x_i | z_i)$ through a noise-prediction objective. This factorization avoids the exponential cost of modeling the full joint distribution while retaining inter-token dependencies. The paper further proposes Prefix Learning to project image or text condition tokens into prefix tokens during generation, and a Latent Token Reconstruction module that reconstructs shape tokens from conditions and blends them with sampled tokens early in autoregressive sampling. On image- and text-conditioned generation tasks, the framework outperforms Michelangelo and the authors' reproduced CLAY on Chamfer Distance, EMD, F-Score, and P-FID, and also improves cross-view consistency.

Load-bearing premise

The reported state-of-the-art comparison with CLAY rests on the authors' own reimplementation of CLAY (540M parameters, same VAE and training hours), since CLAY's code and data are unavailable; if that reimplementation is weaker than the official model, the central 'outperforms existing methods' claim would be overstated.

Editorial extensions

If this is right

  • The framework can be applied to any 3D representation with a suitable token encoder, removing the need for separate generative backbones per representation.
  • Decoupling condition alignment from shape encoding means new condition modalities, such as audio, can be added without retraining the shape encoder.
  • Reconstruction-Guided Sampling offers a general way to improve prompt fidelity in autoregressive-diffusion hybrids, potentially transferable to other token-space generative models.
  • The factorization approach addresses the exponential complexity of joint distribution modeling over long token sequences, making it feasible to scale to larger 3D datasets and longer token sequences.
  • State-of-the-art results on both image- and text-conditioned generation suggest that the autoregressive-diffusion hybrid is a competitive paradigm for 3D generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the state-of-the-art claim largely hinges on a self-implemented 540M-parameter CLAY, a direct comparison against the official CLAY release (if it becomes available) would be a stronger test of that claim; the actual gap could be smaller or larger than reported.
  • The paper's finding that joint training across representations produces low-quality shapes suggests that a unified latent space across different 3D encoders is a key open bottleneck; future work could pursue shared codebooks or contrastive alignment of latent spaces.
  • The MAE-based factorization with random shuffling is a 3D analogue of masked autoregressive diffusion for images; the same design might benefit other 3D-conditioned tasks such as reconstruction, completion, or part-based generation.
  • The reported ULIP gains on image-conditioned ShapeNet are modest while geometric metrics (CD, EMD, F-Score) improve substantially, which may indicate that faithfulness to geometry is easier to achieve than fine-grained semantic alignment with conditions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. LTM3D proposes a token-space conditional 3D shape generation framework that combines an auto-regressive masked autoencoder with a per-token diffusion denoiser, inspired by MAR. It introduces Prefix Learning to align image/text condition tokens with shape latent tokens during generation, and a Latent Token Reconstruction module with Reconstruction-Guided Sampling to stabilize autoregressive token sampling. The framework is evaluated on image- and text-conditioned SDF generation on ShapeNet and Objaverse, with additional experiments on 3D Gaussian Splatting, point cloud, and mesh representations. The paper claims state-of-the-art results against Michelangelo and a CLAY baseline, and provides ablations of the two proposed modules.

Significance. If confirmed, LTM3D is a well-designed contribution to conditional 3D generation: it provides a unified token-space formulation that supports multiple output representations, includes a sensible decoupling of conditioning from shape encoding, and reports extensive comparisons on standard benchmarks with both geometric and semantic metrics. The proposed Prefix Learning and Reconstruction-Guided Sampling are clearly motivated and ablated. The manuscript is honest about limitations, including the need for per-representation models and the suboptimal results of joint multi-representation training. However, the central state-of-the-art claim depends on an unvalidated reimplementation of CLAY and on single-run metric comparisons with small margins; these issues need to be addressed before the claim is externally secured.

major comments (4)
  1. [Section 4.2, Tables 1, 2, 4] The central claim of outperforming CLAY rests on a 540M-parameter reimplementation of CLAY because the official code and data are unavailable. The manuscript gives no evidence that this reimplementation matches the official CLAY's behavior, and the reported margins against it are small and sometimes direction-dependent (e.g., in Table 4, LTM3D's EMD is worse than the reproduced CLAY's, and in Tables 1 and 2 the P-IS is lower). As written, the SOTA claim is not externally validated; please either compare against the official CLAY model or its published metrics, provide a detailed parity check, and report what design choices (conditioning injection, noise schedule, training data mixture, checkpoint selection) were made in the reimplementation.
  2. [Section 4.2, Tables 1-7] All main results and ablations are reported as single runs without error bars or repeated-seed statistics, yet several decisive comparisons are close: Table 1 shows CD of 0.0058 vs. 0.0063 and Table 2 shows CD of 0.0280 vs. 0.0365. With single runs it is impossible to assess whether these differences are statistically significant. Please provide means and standard deviations over at least three runs (or equivalent confidence measures) for the main tables, and ideally for the ablations in Tables 5-7.
  3. [Eqs. (4)-(5), Supplementary B.5 and D] The Reconstruction-Guided Sampling schedule is described inconsistently. Eq. (5) defines the blend as xF = (1 - alpha_i)*xhat + alpha_i*xS; Supplementary B.5 states alpha_i = 0.1 for i <= 30, while Supplementary D states that sampled tokens are blended with reconstructed tokens using a linear weight alpha = 0.9. The roles of alpha in the two statements appear contradictory, and the relation between the fusion 'step' (Table 6) and 'ratio' (Table 7) is not defined. Please clarify the exact schedule and explicitly state which setting was used to produce the final numbers.
  4. [Section 4.3, Tables 5-7] The ablation tables show only point estimates and the differences are often within what one would expect from sampling noise (e.g., Table 7 shows CD varying only from 0.0054 to 0.0053, and F-Score from 0.2600 to 0.2612 across fusion ratios). The claim that Reconstruction-Guided Sampling reduces uncertainty would be more convincing with a direct measurement of output variance across repeated samples (e.g., per-prompt variance of CD or ULIP), rather than point estimates of average metrics.
minor comments (5)
  1. [Table 2] The F-Score entries for LTM3D are written as '0.1108(8)' and '0.1109(4)'; the parenthetical digits are unexplained and appear to be typographical artifacts.
  2. [Section 5 and Supplementary F] The paper claims a general framework for multi-representation generation, but Section 5 states that a separate model is required for each representation, and Supplementary F shows that joint training with multiple representations yields poor results. Please soften the 'general' claim or make the per-representation requirement explicit in the abstract and introduction.
  3. [Section 4.1] The definition of P-FID is imprecise: 'calculated between the generated shapes and ground truth shapes on the test dataset' does not specify the feature extractor or the number of samples used. Since P-FID is a distributional metric, details of the point sets and backbone matter for reproducibility.
  4. [Figure 2] The schematic uses several visual tokens (? marks, 'learnable tokens', 'prefix tokens', 'recon. shape tokens') without a legend; the caption should define each token type and the meaning of 'Only in generation' and the alpha blend notation.
  5. [Section 3.2, Eq. (2)] The factorization p(X|T) = product p(x_i | x_<i, T) is standard, but the claim that optimizing this product avoids the exponential complexity of joint-distribution modeling should be justified more carefully, since the condition x_<i still encodes the full prefix; a brief reference to why MAE random shuffling suffices would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: LTM3D is trained with diffusion and reconstruction losses and evaluated on external metrics; the CLAY reimplementation is a benchmarking-validity concern, not a circular derivation.

full rationale

The paper's derivation chain is not circular. The conditional distribution backbone is trained with the diffusion objective in Eq. 3 and the reconstruction module with MSE (Sec. B.4), while all reported metrics (ULIP, CD, EMD, F-Score, P-FID) are computed with externally pretrained or geometry-based measures (Sec. 4.1) that do not appear as terms in the training losses. The claimed state-of-the-art results depend on comparisons in Tables 1, 2, and 4, in which CLAY results are reproduced by the authors because the official code and data are unavailable (Sec. 4.2). That is a threat to the strength of the comparison, not a circularity: the authors do not fit any parameter of LTM3D to the CLAY numbers, and no equation in the paper defines a prediction as the fitted value of its own input. The Prefix Learning and Latent Token Reconstruction modules are learned feed-forward mappings from condition tokens to shape tokens; blending them in Eq. 5 modifies the conditioning of the MAE but does not replace the diffusion-sampled tokens. No self-citation is load-bearing, and no 'uniqueness' or ansatz is imported from the authors' prior work. The appropriate verdict is no significant circularity (score 0).

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The framework introduces no new physical or conceptual entities. It depends on pretrained VAE latent spaces, the MAR modeling paradigm, and the ULIP evaluation metric, all of which are taken as reliable from prior work. The only hand-tuned quantities that directly affect the reported gains are the RGS fusion ratio and fusion step, selected from ablations.

free parameters (2)
  • fusion_ratio_alpha = 0.1
    Chosen via ablation over {0.1, 0.2, 0.3, 0.4} to balance ULIP, CD, EMD, F-Score; the RGS contribution uses this value.
  • fusion_step_threshold = 30
    Chosen via ablation over {10, 20, 30, 40}; at 40 ULIP declines, so 30 is selected. The RGS inference procedure depends on this cutoff.
assumptions (3)
  • domain assumption 3DShape2VecSet VAE provides a faithful latent token space for SDF shape generation
    The framework relies on the pretrained VAE to encode and decode shapes; the quality of generation is bounded by the VAE.
  • domain assumption MAR's factorized diffusion is an effective modeling choice for 3D latent tokens
    The Conditional Distribution Modeling backbone directly adopts MAR's masked autoencoder and DenoiseNet; the paper does not validate this choice for 3D data beyond its own experiments.
  • domain assumption ULIP score measures prompt fidelity for 3D shapes
    The paper uses ULIP as the main faithfulness metric; it is a pretrained model and may not capture all aspects of prompt adherence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LTM3D: Bridging Token Spaces for Conditional 3D Generation with Auto-Regressive Diffusion Framework." pith.science (2026). https://pith.science/paper/6XTUCQNZ

@misc{pith2026250524245,
  author       = {Pith},
  title        = {Pith review of: LTM3D: Bridging Token Spaces for Conditional 3D Generation with Auto-Regressive Diffusion Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6XTUCQNZ}},
  note         = {Machine review of arXiv:2505.24245}
}
read the original abstract

We present LTM3D, a Latent Token space Modeling framework for conditional 3D shape generation that integrates the strengths of diffusion and auto-regressive (AR) models. While diffusion-based methods effectively model continuous latent spaces and AR models excel at capturing inter-token dependencies, combining these paradigms for 3D shape generation remains a challenge. To address this, LTM3D features a Conditional Distribution Modeling backbone, leveraging a masked autoencoder and a diffusion model to enhance token dependency learning. Additionally, we introduce Prefix Learning, which aligns condition tokens with shape latent tokens during generation, improving flexibility across modalities. We further propose a Latent Token Reconstruction module with Reconstruction-Guided Sampling to reduce uncertainty and enhance structural fidelity in generated shapes. Our approach operates in token space, enabling support for multiple 3D representations, including signed distance fields, point clouds, meshes, and 3D Gaussian Splatting. Extensive experiments on image- and text-conditioned shape generation tasks demonstrate that LTM3D outperforms existing methods in prompt fidelity and structural accuracy while offering a generalizable framework for multi-modal, multi-representation 3D generation.

Figures

Figures reproduced from arXiv: 2505.24245 by the authors.

Figure 1
Figure 1. LTM3D is a conditional 3D shape generation framework that supports image and text inputs, as well as multiple 3D data [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our framework LTM3D consists of three components. In Conditional Distribution Modeling, we implement an auto-regressive [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative image-conditioned generation results on ShapeNet. Row (a) displays the input images, row (b) shows the generation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Image-conditioned generation results for different input views on ShapeNet. Row (a) displays the input images, row (b) shows [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative text-conditioned generation results on ShapeNet. Row (a) displays the input text descriptions, row (b) shows the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Image-conditioned generation results on Objaverse. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative image-conditioned generation results on [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Image-conditioned generation results for SDF data on Objaverse. Each pair consists of an input image (left) and the corresponding [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Image-conditioned generation results for SDF data on ShapeNet. Each pair consists of an input image (left) and the corresponding [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Image-conditioned generation results for point cloud, 3DGS, and mesh data on ShapeNet. Each pair includes an input image [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Visualization of joint training on ShapeNet chairs. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 17 canonical work pages

  1. [14]

    Autoregressive image generation without vec- tor quantization.arXiv preprint arXiv:2406.11838, 2024

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization.arXiv preprint arXiv:2406.11838, 2024. 2, 3, 4, 1

    arXiv 2024

  2. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  3. [2]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

  4. [3]

    Shapenet: An information-rich 3d model repository.arXiv preprint arXiv:1512.03012, 2015

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository.arXiv preprint arXiv:1512.03012, 2015. 4

    arXiv 2015

  5. [4]

    Meshxl: Neural coordinate field for generative 3d foundation models.arXiv preprint arXiv:2405.20853,

    Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Yanru Wang, Zhibin Wang, Chi Zhang, et al. Meshxl: Neural coordinate field for generative 3d foundation models.arXiv preprint arXiv:2405.20853,

    arXiv

  6. [5]

    Text-to-3d using gaussian splatting

    Zilong Chen, Feng Wang, Yikai Wang, and Huaping Liu. Text-to-3d using gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21401–21412, 2024. 2

    work page 2024

  7. [6]

    Sdfusion: Multimodal 3d shape completion, reconstruction, and generation

    Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexan- der G Schwing, and Liang-Yan Gui. Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4456–4465, 2023. 1, 2

    work page 2023

  8. [7]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13142–13153, 2023. 4

    2023

Show all 41 references
  1. [8]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 4, 1

    2022

  2. [9]

    Dreamtime: An improved optimiza- tion strategy for text-to-3d content creation.arXiv preprint arXiv:2306.12422, 2023

    Yukun Huang, Jianan Wang, Yukai Shi, Xianbiao Qi, Zheng- Jun Zha, and Lei Zhang. Dreamtime: An improved optimiza- tion strategy for text-to-3d content creation.arXiv preprint arXiv:2306.12422, 2023. 2

    2023 arXiv

  3. [10]

    Shap-e: Generat- ing conditional 3d implicit functions.arXiv preprint arXiv:2305.02463, 2023

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions.arXiv preprint arXiv:2305.02463, 2023. 1

    2023 arXiv

  4. [11]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1,

  5. [12]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 1

    2023

  6. [13]

    Diffusion- sdf: Text-to-shape via voxelized diffusion

    Muheng Li, Yueqi Duan, Jie Zhou, and Jiwen Lu. Diffusion- sdf: Text-to-shape via voxelized diffusion. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12642–12651, 2023. 4

    2023

  7. [15]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2023

  8. [16]

    Flowseq: Non-autoregressive conditional sequence generation with generative flow.arXiv preprint arXiv:1909.02480, 2019

    Xuezhe Ma, Chunting Zhou, Xian Li, Graham Neubig, and Eduard Hovy. Flowseq: Non-autoregressive conditional sequence generation with generative flow.arXiv preprint arXiv:1909.02480, 2019. 2, 4

    1909 arXiv

  9. [17]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 2

    2021

  10. [18]

    Autosdf: Shape priors for 3d comple- tion, reconstruction and generation

    Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shub- ham Tulsiani. Autosdf: Shape priors for 3d comple- tion, reconstruction and generation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 306–315, 2022. 7

    2022

  11. [19]

    Polygen: An autoregressive generative model of 3d meshes

    Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes. InInternational conference on machine learning, pages 7220–7229. PMLR, 2020. 1

    2020

  12. [20]

    Point-e: A system for generat- ing 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751, 2022

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751, 2022. 2, 5, 1

    2022 arXiv

  13. [21]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 3, 8

    2023 arXiv

  14. [22]

    Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022. 2

    2022 arXiv

  15. [23]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  16. [24]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1, 2 9

    2021

  17. [25]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2

    2022

  18. [26]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information ...

    2022

  19. [27]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Ta- tiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  20. [28]

    Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior

    Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen. Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 22819–22829, 2023. 2

    2023

  21. [29]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1

    2023 arXiv

  22. [30]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017. 2

    2017

  23. [31]

    Hd- fusion: Detailed text-to-3d generation leveraging multiple noise estimation

    Jinbo Wu, Xiaobo Gao, Xing Liu, Zhengyang Shen, Chen Zhao, Haocheng Feng, Jingtuo Liu, and Errui Ding. Hd- fusion: Detailed text-to-3d generation leveraging multiple noise estimation. InProceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 3...

    2024

  24. [32]

    Disn: Deep implicit surface network for high-quality single-view 3d reconstruction.Ad- vances in neural information processing systems, 32, 2019

    Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. Disn: Deep implicit surface network for high-quality single-view 3d reconstruction.Ad- vances in neural information processing systems, 32, 2019. 4

    2019

  25. [33]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. InProceedings of the IEEE/CVF conference on...

    2023

  26. [34]

    Points-to-3d: Bridging the gap be- tween sparse points and shape-controllable text-to-3d gener- ation

    Chaohui Yu, Qiang Zhou, Jingliang Li, Zhe Zhang, Zhibin Wang, and Fan Wang. Points-to-3d: Bridging the gap be- tween sparse points and shape-controllable text-to-3d gener- ation. InProceedings of the 31st ACM International Confer- ence on Multimedia, pages 6841–6850, 2023. 2

    2023

  27. [35]

    3dilg: Ir- regular latent grids for 3d generative modeling.Advances in Neural Information Processing Systems, 35:21871–21885,

    Biao Zhang, Matthias Nießner, and Peter Wonka. 3dilg: Ir- regular latent grids for 3d generative modeling.Advances in Neural Information Processing Systems, 35:21871–21885,

  28. [36]

    3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023

    Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023. 1, 2, 3, 4

    2023

  29. [37]

    Gaussiancube: Structuring gaussian splatting using opti- mal transport for 3d generative modeling.arXiv preprint arXiv:2403.19655, 2024

    Bowen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, and Baining Guo. Gaussiancube: Structuring gaussian splatting using opti- mal transport for 3d generative modeling.arXiv preprint arXiv:2403.19655, 2024. 1, 2, 4

    2024 arXiv

  30. [38]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2

    2023

  31. [39]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets.ACM Transactions on Graphics (TOG), 43(4):1–20, 2024

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets.ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 1, 2, 3, 4, 5, 6, 7

    2024

  32. [40]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation.Advances in Neural Information Processing Systems, 36, 2024

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation.Advances in Neural Information Processing Systems, 36, 2024. 2, ...

    2024

  33. [41]

    Hifa: High- fidelity text-to-3d generation with advanced diffusion guid- ance.arXiv preprint arXiv:2305.18766, 2023

    Junzhe Zhu, Peiye Zhuang, and Sanmi Koyejo. Hifa: High- fidelity text-to-3d generation with advanced diffusion guid- ance.arXiv preprint arXiv:2305.18766, 2023. 2 10 LTM3D: Bridging Token Spaces for Conditional 3D Generation with Auto-Regressive Diffusion Framework Supplementa...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.