Pith. sign in

REVIEW 4 major objections 4 minor 93 references

SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-scale 3D VQVAE tokenizer lets an autoregressive transformer generate a 3D object from an image in 0.82 seconds and lets an LLM caption it.

desk verdict Useful integration of next-scale prediction with 3D triplane latents, but the 'surpasses both speed and quality' claim doesn't survive its own Table 1. read the letter →

arxiv 2411.16856 v3 pith:M7AODW3M submitted 2024-11-25 cs.CV

Yongwei Chen , Yushi Lan , Shangchen Zhou , Tengfei Wang , Xingang Pan This is my paper · ORCID
classification cs.CV
keywords autoregressive3Dgenerationmulti-scaleVQVAEnext-scalepredictiontriplanelatentrepresentationtext-to-3Dsingle-image-to-3DcaptioningunderstandingwithLLMs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SAR3D argues that autoregressive prediction can do for 3D objects what next-token and next-scale models did for language and images, if the 3D object is first turned into a compact multi-scale discrete latent. The paper introduces a multi-scale 3D VQVAE that encodes six posed RGB-D renderings into a hierarchical triplane token sequence, then trains a transformer to predict the next scale rather than the next single token. This yields 3D generation from a single image in 0.82 seconds on an A6000 GPU and from text in 1.46 seconds, with quality that the paper reports as surpassing existing feed-forward and diffusion-based generators. The same VQVAE tokens, truncated to their coarse scales, are fed to a fine-tuned LLM, letting it caption 3D models and even caption objects it has just generated. The significance is a single tokenizer serving both fast generation and multimodal understanding.

What carries the argument

The engine of SAR3D is a multi-scale 3D VQVAE whose latent space is a triplane: three axis-aligned feature planes that jointly describe a volume. A 3D object is first rendered from six viewpoints into RGB, depth, and Plücker coordinates (six channels encoding each pixel's ray origin and direction), and a multi-view convolutional encoder maps these ten-channel renderings to a continuous triplane feature map. A residual quantizer then converts that map into ten scales of discrete token maps sharing one codebook, so each new scale adds finer detail on top of the previous ones. A decoder turns the quantized triplane back into rendered images, depths, and normals, and is later fine-tuned on a hybrid mesh representation for clean surface extraction. For generation, a GPT-style transformer predicts these scales causally, conditioned on text-encoder or image-encoder features through cross-attention, using the next-scale prediction schedule from 2D visual autoregressive models, which is what cuts the number of autoregressive steps. For understanding, only the first eight scales (37.5% of the tokens) are projected into a pretrained LLM's embedding space.

What would settle it

Take objects with large self-occlusion or hidden structure, encode each from exactly six canonical views with the released VQVAE, and compare the reconstructed meshes against dense 360-degree scans; large geometric error, or captions that confidently name parts invisible in those views, would show the six-view premise fails. A cheaper check is to randomize the six camera poses and watch whether reconstruction and captioning metrics change, which would indicate the tokenizer depends on the exact viewpoints rather than on full object coverage.

Watch

Extended reading notes

Core claim

The central claim is that a multi-scale tokenization of 3D content turns 3D generation and understanding into one autoregressive modeling problem. By encoding multi-view RGB-D renderings with Plücker camera embeddings into a latent triplane and quantizing it over ten shared-codebook scales, SAR3D replaces the next-token loop with next-scale prediction: at each step the transformer outputs an entire token map at a higher resolution, conditioned on text or image features via cross-attention. The paper reports that this single design achieves sub-second 3D generation, state-of-the-art or competitive generation metrics, and that the same tokenizer, using only the first eight scales, can be aligned with an LLM for detailed 3D captioning and for simultaneous generation and captioning. In the authors' telling, the VQVAE token space is the load-bearing bridge: it is compact enough for fast autoregressive sampling, detailed enough to reconstruct geometry and texture, and structured enough to be consumable by a language model.

Load-bearing premise

The whole pipeline rests on the assumption that six posed RGB-D renderings with per-pixel ray embeddings are enough to determine a 3D object's geometry and appearance, so the quantized triplane keeps the detail needed for both faithful reconstruction and truthful captioning.

Editorial extensions

If this is right

  • Image-conditioned 3D generation completes in 0.82 seconds and text-conditioned generation in 1.46 seconds on one A6000 GPU including mesh extraction, faster than the compared diffusion-based native 3D generators.
  • In the image-conditioned comparison, SAR3D reports the best or near-best scores on FID, KID, MUSIQ, coverage, and MMD, with only LGM's FID slightly lower, while running faster than most baselines.
  • A fine-tuned LLM can caption a 3D object from a single encoding of its VQVAE tokens, using only the first eight of ten scales, and can also caption objects produced by the autoregressive generator, enabling simultaneous generation and understanding.
  • The same tokenizer serves both tasks, so no separate encoder is needed for understanding versus generation, in contrast to approaches that decouple the two.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper does not run: remove views from the six-view input and measure how reconstruction and captioning degrade; if coarse scales alone drive understanding, even two views might caption adequately while still being insufficient for accurate geometry.
  • The success of truncated coarse tokens suggests a general token-budget rule for 3D multimodal models: language understanding needs category and layout structure more than fine detail, so understanding could be made much cheaper than generation.
  • Because the generator and the LLM consume the same discrete token space, the framework points toward a single multimodal model that both emits and describes 3D content, a direction the authors list as future work rather than a demonstrated result.
  • If the next-scale schedule scales like its 2D counterpart, increasing dataset size and model width should further improve 3D quality; the paper states that this scalability is not yet verified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SAR3D, a framework built around a multi-scale 3D VQVAE that tokenizes multi-view RGB-D renderings (with Plücker coordinates) into hierarchical triplane latents. A GPT-style transformer then performs next-scale autoregressive prediction conditioned on an image or text prompt, enabling fast 3D generation. The same tokenizer, using a truncated subset of scales, is connected to a finetuned LLaMA model for 3D captioning and simultaneous generation-and-captioning. The authors report 0.82 s generation on an A6000 GPU and claim superiority over prior 3D generation methods in both speed and quality, supported by quantitative image-to-3D results in Table 1 and by qualitative text-to-3D and captioning results.

Significance. The core idea is a meaningful and potentially useful extension of VAR-style next-scale prediction to 3D: a single multi-scale triplane tokenizer that supports both efficient generation and LLM-based understanding is an attractive design, and the presented pipeline is more coherent than approaches that train separate encoders for generation and understanding. The paper is also candid in parts, notably in the Table 1 note conceding that LGM has better FID. The implementation details are sufficiently concrete to be reproducible, and the qualitative results show appealing 3D consistency. However, the headline claim of dominance in both speed and quality is not supported by the paper's own evidence, and the text-to-3D and captioning evaluations are qualitative only. The value of the work is better described as a favorable speed-quality trade-off plus a unified tokenizer, rather than demonstrated superiority on every axis.

major comments (4)
  1. [Abstract and §5.1 (Table 1)] The claim that SAR3D 'surpasses current 3D generation methods in both speed and quality' is contradicted by Table 1. Splatter-Image has a reported V100 latency of 0.83 s, while SAR3D-NeRF and SAR3D-Flexicubes are listed at 1.64 s and 2.92 s, respectively, so SAR3D is not the fastest method in the table. On quality, LGM achieves FID 19.93 versus 22.55 for SAR3D-NeRF and 27.30 for SAR3D-Flexicubes; the table note itself concedes that LGM is better on FID. No baseline is dominated on both speed and all quality axes. The abstract and contribution bullet should be revised to claim a speed-quality trade-off or a different, precisely stated form of superiority that is consistent with the reported numbers.
  2. [§5.1] The latency comparison is not apples-to-apples. The text states that SAR3D timings are measured on an A6000 GPU and then 'scaled by a factor of 2' to populate the Latency-V100 column, while all baseline timings are measured directly on a Tesla V100. No evidence is given that a constant factor of 2 accurately maps A6000 timings to V100 timings across different models, kernels, and memory profiles. Since the speed claim is central to the paper, please report timings measured on the same GPU for all methods, or validate the scaling factor on at least a subset of the baselines and disclose the variance.
  3. [§5.2 and §5.3] Text-to-3D generation and 3D captioning are evaluated only qualitatively. There are no quantitative metrics for text-to-3D (e.g., CLIP-score, T3Bench, or human preference), no quantitative captioning metrics (e.g., CIDEr or BLEU against Cap3D annotations), and no comparison with existing 3D captioning methods such as Cap3D. The claims that SAR3D 'supports' text-to-3D and that the LLM can 'interpret and caption 3D models comprehensively' are therefore not quantitatively supported. At minimum, the paper should add one quantitative evaluation for text-to-3D and one for captioning, or explicitly scope these sections as qualitative demonstrations.
  4. [§4.3] The truncation choice in Eq. (10), where only the first K−2 scales are used for LLM understanding, is described as an empirical observation, but no ablation or analysis is provided. The number of scales fed to the LLM is a free parameter that directly affects the understanding pipeline, and the paper even notes that it contains only 37.5% of the generation tokens. Please add an ablation over the truncation level, or at least a quantitative comparison showing that K−2 outperforms other choices, so that the design is not justified solely by a single qualitative observation.
minor comments (4)
  1. [§5.1, Table 1 caption] The table caption says 'The latency time is all profiled on Tesla V100 architecture,' but the text says SAR3D was measured on A6000 and then scaled. The caption should be corrected to state that SAR3D times are scaled estimates, or the table should report raw hardware and measured/scaled status for each row.
  2. [Abstract and §5.1] The abstract's '0.82 seconds on an A6000 GPU' does not match the Table 1 Latency-V100 value of 1.64 s for the same model. Please label the hardware and protocol consistently in the abstract, main text, and table so that readers are not misled.
  3. [Supplemental Material, Table S1] The training-data statistics table appears to contain a formatting error: the 'Ours' entry is duplicated as '170K 170K', and Shap-E's '2M - 9M' range is ambiguous. Please correct the table so that each method has one clear dataset size.
  4. [§5.1] The sentence 'Our SAR3D demonstrates strong performance across all metrics' is too strong given that LGM has a better FID. Consider writing 'strong performance on most metrics' or explicitly qualifying the FID exception.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; one minor self-citation (LN3Diff) is architectural and not load-bearing.

full rationale

SAR3D is an empirical systems paper rather than a derivation from first principles. The multi-scale 3D VQVAE is trained with reconstruction losses (Eq. 8 and Eq. 9) on multi-view RGB-D renderings; the autoregressive transformer is trained with cross-entropy next-scale prediction conditioned on CLIP or DINOv2 features; and the LLM is fine-tuned on the VQVAE tokens following PointLLM. Each claimed capability is benchmarked against external baselines (Table 1, Figs. 4 and 6) rather than derived from the same fitted values. The use of the authors' prior LN3Diff for the latent-triplane design is an architectural borrowing, not a load-bearing citation: the triplane representation is also attributed to external Direct3D [80], and reconstruction and generation quality are judged independently. The understanding module is trained and evaluated on the same tokenizer's outputs, which is a closed-loop representation setup; this constrains what can be concluded about general 3D understanding but is not circular in the sense of a prediction reducing to its inputs. The reported speed comparison applies an unvalidated 2x scaling factor from A6000 to V100, and Table 1 does not support the abstract's 'both speed and quality' dominance claim on every axis; these are experimental-rigor and claim-support concerns, not circularity. No equation in the paper reduces to its own input by construction, and no central premise depends on a self-citation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. It relies on several domain assumptions about the sufficiency of multi-view RGB-D input, the fidelity of the triplane latent, the transfer of VAR's 2D recipe to 3D, and the LLM's ability to learn 3D semantics from the truncated tokens. The main hand-chosen design decisions are the input view count and the truncation level for LLM training.

free parameters (2)
  • truncation_level_for_understanding = K-2 (scales r_1,...,r_{K-2})
    Empirically chosen because 'not all scales in R are required for 3D understanding training' (Sec. 4.3); no principled criterion is given.
  • input_view_count = 6
    The VQVAE encodes six posed RGB-D views (Sec. 4.1); this number is fixed without an ablation and directly determines the information available to the tokenizer.
assumptions (4)
  • domain assumption Six multi-view RGB-D renderings with Plücker embeddings are a sufficient representation of a 3D object for tokenization
    Invoked in Sec. 4.1 when defining M = (I, Delta, pi) from 6 views; the VQVAE's reconstruction quality depends on this.
  • domain assumption The multi-scale latent triplane with shared codebook can faithfully encode both geometry and texture
    The whole pipeline in Sec. 4.1 relies on this reconstruction capacity; no quantitative reconstruction error is reported in the main text.
  • ad hoc to paper Next-scale prediction and residual quantization from 2D VAR transfer to 3D triplane latents
    Adopted from VAR (Sec. 3.1 and 4.2) without independent evidence that the 2D inductive biases carry over to 3D.
  • domain assumption A fine-tuned LLM can acquire 3D understanding from truncated scale tokens
    Sec. 4.3; only qualitative captioning results are shown, no quantitative benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE." pith.science (2026). https://pith.science/paper/M7AODW3M

@misc{pith2026241116856,
  author       = {Pith},
  title        = {Pith review of: SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M7AODW3M}},
  note         = {Machine review of arXiv:2411.16856}
}
read the original abstract

Autoregressive models have demonstrated remarkable success across various fields, from large language models (LLMs) to large multimodal models (LMMs) and 2D content generation, moving closer to artificial general intelligence (AGI). Despite these advances, applying autoregressive approaches to 3D object generation and understanding remains largely unexplored. This paper introduces Scale AutoRegressive 3D (SAR3D), a novel framework that leverages a multi-scale 3D vector-quantized variational autoencoder (VQVAE) to tokenize 3D objects for efficient autoregressive generation and detailed understanding. By predicting the next scale in a multi-scale latent representation instead of the next single token, SAR3D reduces generation time significantly, achieving fast 3D object generation in just 0.82 seconds on an A6000 GPU. Additionally, given the tokens enriched with hierarchical 3D-aware information, we finetune a pretrained LLM on them, enabling multimodal comprehension of 3D content. Our experiments show that SAR3D surpasses current 3D generation methods in both speed and quality and allows LLMs to interpret and caption 3D models comprehensively.

Figures

Figures reproduced from arXiv: 2411.16856 by the authors.

Figure 1
Figure 1. Our method, SAR3D, proposes a comprehensive framework for 3D generation and understanding via autoregressive modeling. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Multi-scale VQVAE. Given a 3D model, we leverage multi-view RGB-D(epth) renderings and Plucker embeddings ¨ as the input to our multi-view encoder E. The encoder predicts a continuous feature map that is then quantized by the multi-scale quantizer Q, giving R = (r1, r2, . . . , rK) of latent tri-plane features. Each code of different scales share the same codebook. The triplane decoder then converts the … view at source ↗
Figure 3
Figure 3. Overview of 3D Generation and 3D Understanding. Given a 3D model, our 3D VQVAE encodes it into multi-scale discrete tokens for both 3D generation and understanding. In (a) 3D Generation, text or a single image is encoded by CLIPT or DINOv2, and the encoded condition features are integrated into the decoder-only transformer via cross attention. The transformer then causally predicts each scale of the latent triplane.… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison of Image-conditioned 3D Generation. Here, we compare with the state-of-the-art 3D generative models under different categories. As visualized here, our method achieves superior 3D consistency across views and generates intact objects without dist…
Figure 5
Figure 5. Figure 5: More results of image and text conditioned 3D generation of SAR3D. ”A brown wooden table with a rectangular shape, featuring visible legs and vintage look.” Input Ours “a sword with a red handle.” ”A cute and friendly pink teddy bear with sitting pose.” Point-E Shap-E …
Figure 6
Figure 6. Figure 6: Comparison of Text-conditioned 3D Generation. We present text-conditioned 3D objects generated by SAR3D, displaying two views of each sample.Compared to baseline methods, our approach consistently yields better quality regarding geometry, texture, and text-3D alignment…
Figure 7
Figure 7. Figure 7: Simultaneous 3D Generation and Captioning. Given a single image or text, SAR3D-LLM can generate both a 3D model and a descriptive caption for the model. A sleek and modern blue and white chair with a curved backrest, designed for comfort and support. It features a blac…
Figure 8
Figure 8. Figure 8: 3D Object Captioning. Given a 3D model, SAR3D￾LLM can generate captions that include both category and details. a concise interpretation of the 3D data presented here.”, SAR3D-LLM can generate both the correct category and fine details of the input 3D models. For examp…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

93 extracted references · 57 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems (NeurIPS) , 35: 23716–23736, 2022. 1, 3

    2022

  3. [3]

    Demystifying mmd gans

    Mikołaj Bi ´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. In International Conference on Learning Representations (ICLR), 2018. 7

    2018

  4. [4]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1:8, 2024. 4

    2024

  5. [5]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS) , 33:1877–1901,

    1901

  6. [6]

    Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Mano- lis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu

    Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Mano- lis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information-Rich 3D Model Repository. arXiv preprint arXiv:1512.03012, 2015. 2

    arXiv 2015

  7. [7]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 11315–11325,

  8. [8]

    Lara: Efficient large-baseline radiance fields

    Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, and Andreas Geiger. Lara: Efficient large-baseline radiance fields. In Proceedings of the European Conference on Com- puter Vision (ECCV), pages 338–355. Springer, 2024. 7, 14

    2024

Show all 93 references
  1. [9]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22246–22256, 2023. 2

    2023

  2. [10]

    Comboverse: Compositional 3d assets creation using spatially-aware diffusion guidance

    Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, and Ziwei Liu. Comboverse: Compositional 3d assets creation using spatially-aware diffusion guidance. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 128–146. Springer, 2024. 2

    2024

  3. [11]

    Meshanything: Artist- created mesh generation with autoregressive transformers

    Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Jiaxiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, Guosheng Lin, and Chi Zhang. Meshanything: Artist- created mesh generation with autoregressive transformers. In International Conference on Learning Representations ...

    2025

  4. [12]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023...

    2023

  5. [13]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1–113, 2023. 1

    2023

  6. [14]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems (NeurIPS), 36:35799...

    2023

  7. [15]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  8. [16]

    Palm- e: An embodied multimodal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm- e: An embodied multimodal language model. In Inter- national Conference on Machine Learning (ICML) , pages 8469–8488. PM...

    2023

  9. [17]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12873–12883, 2021. 2

    2021

  10. [18]

    Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following

    Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xi- anzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xi- anzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following. arXiv prepri...

    2023 arXiv

  11. [19]

    OpenLRM: Open-source large reconstruction models

    Zexin He and Tengfei Wang. OpenLRM: Open-source large reconstruction models. https://github.com/ 3DTopia/OpenLRM, 2023. 6, 7, 14

    2023

  12. [20]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017. 7

    2017

  13. [21]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 5

    2021

  14. [22]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS), 2020. 2, 5

    2020

  15. [23]

    3DTopia: Large text-to-3d generation model with hybrid diffusion priors

    Fangzhou Hong, Jiaxiang Tang, Ziang Cao, Min Shi, Tong Wu, Zhaoxi Chen, Tengfei Wang, Liang Pan, Dahua Lin, and Ziwei Liu. 3DTopia: Large text-to-3d generation model with hybrid diffusion priors. arXiv preprint arXiv:2403.02234 ,

    arXiv

  16. [24]

    3d-llm: In- jecting the 3d world into large language models

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: In- jecting the 3d world into large language models. Advances in Neural Information Processing Systems (NeurIPS) , 36: 20482–20494, 2023. 1, 3

    2023

  17. [25]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. In International Conference on Learning Representa- tions (ICLR), 2024. 1, 2, 5, 6, 14

    2024

  18. [26]

    2d gaussian splatting for geometrically ac- curate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically ac- curate radiance fields. In ACM SIGGRAPH Conference Pro- ceedings, pages 1–11, 2024. 8

    2024

  19. [27]

    Shap-E: Generat- ing conditional 3D implicit functions

    Heewoo Jun and Alex Nichol. Shap-E: Generat- ing conditional 3D implicit functions. arXiv preprint arXiv:2305.02463, 2023. 1, 7, 14

    2023 arXiv

  20. [28]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. 4

    2020

  21. [29]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5148–5157, 2021. 7

    2021

  22. [30]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding vari- ational bayes. arXiv, 2013. 2, 4

    2013

  23. [31]

    Nerf-vae: A geometry aware 3d scene generative model

    Adam R Kosiorek, Heiko Strathmann, Daniel Zo- ran, Pol Moreno, Rosalia Schneider, Sona Mokr ´a, and Danilo Jimenez Rezende. Nerf-vae: A geometry aware 3d scene generative model. In International Conference on Ma- chine Learning (ICML), pages 5742–5752. PMLR, 2021. 2

    2021

  24. [32]

    Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation

    Yushi Lan, Fangzhou Hong, Shuai Yang, Shangchen Zhou, Xuyi Meng, Bo Dai, Xingang Pan, and Chen Change Loy. Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 112–130. Springer, 2...

    2024

  25. [33]

    Gaussiananything: Interactive point cloud latent dif- fusion for 3d generation

    Yushi Lan, Shangchen Zhou, Zhaoyang Lyu, Fangzhou Hong, Shuai Yang, Bo Dai, Xingang Pan, and Chen Change Loy. Gaussiananything: Interactive point cloud latent dif- fusion for 3d generation. In International Conference on Learning Representations (ICLR), 2025. 2, 13

    2025

  26. [34]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In CVPR, 2022. 2

    2022

  27. [35]

    CraftsMan: High-fidelity mesh generation with 3D native generation and interactive geometry refiner

    Weiyu Li, Jiarui Liu, Hongyu Yan, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. CraftsMan: High-fidelity mesh generation with 3D native generation and interactive geometry refiner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2025

  28. [36]

    Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining

    Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining. arXiv preprint arXiv:2408.02657, 2024. 2

    2024 arXiv

  29. [37]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems (NeurIPS), 36:34892–34916, 2023. 3

    2023

  30. [38]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems (NeurIPS), 36:22226–22246, 2023. 7, 14

    2023

  31. [39]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9298–9309, 2023. 2

    2023

  32. [40]

    Wonder3d: Sin- gle image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  33. [41]

    Scalable 3d captioning with pretrained models

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin John- son. Scalable 3d captioning with pretrained models. Ad- vances in Neural Information Processing Systems (NeurIPS), 36:75307–75337, 2023. 5

    2023

  34. [42]

    View selec- tion for 3d captioning via diffusion ranking

    Tiange Luo, Justin Johnson, and Honglak Lee. View selec- tion for 3d captioning via diffusion ranking. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 180–197. Springer, 2024. 5

    2024

  35. [43]

    Kosmos-2.5: A multimodal literate model

    Tengchao Lv, Yupan Huang, Jingye Chen, Yuzhong Zhao, Yilin Jia, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, et al. Kosmos-2.5: A multimodal literate model. arXiv preprint arXiv:2309.11419, 2023. 3

    2023 arXiv

  36. [44]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In Proceedings of the European Conference on Com- puter Vision (ECCV), pages 405–421, 2020. 4

    2020

  37. [45]

    Autosdf: Shape priors for 3d comple- tion, reconstruction and generation

    Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shub- ham Tulsiani. Autosdf: Shape priors for 3d comple- tion, reconstruction and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 306–315, 2022. 2

    2022

  38. [46]

    Point-e: A system for generat- ing 3d point clouds from complex prompts

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022. 1, 7

    2022 arXiv

  39. [47]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research , p...

  40. [48]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- vances in Neural Information Processing Systems (Neu...

    2022

  41. [49]

    Scalable diffusion mod- els with transformers

    William Peebles and Saining Xie. Scalable diffusion mod- els with transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 4195–4205, 2023. 4, 5

    2023

  42. [50]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In In- ternational Conference on Learning Representations (ICLR),

  43. [51]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 214–238. Springer, 2024. 3

    2024

  44. [52]

    Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to- 3d

    Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mu- tian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong, Liefeng Bo, and Xiaoguang Han. Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to- 3d. In Proceedings of the IEEE/CVF Conference on Com- ...

    2024

  45. [53]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  46. [54]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Confer- ence on Machine Learning (ICML), pages 8821–8831. Pmlr,

  47. [55]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 1, 4, 5

    2022

  48. [56]

    Pixelcnn++: Improving the pixelcnn with dis- cretized logistic mixture likelihood and other modifications

    Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcnn++: Improving the pixelcnn with dis- cretized logistic mixture likelihood and other modifications. In International Conference on Learning Representations (ICLR), 2017. 2

    2017

  49. [57]

    Flexible isosurface extraction for gradient-based mesh optimization

    Tianchang Shen, Jacob Munkberg, Jon Hasselgren, Kangxue Yin, Zian Wang, Wenzheng Chen, Zan Gojcic, Sanja Fidler, Nicholas Sharp, and Jun Gao. Flexible isosurface extraction for gradient-based mesh optimization. ACM Transactions on Graphics (TOG), 42(4):1–16, 2023. 4

    2023

  50. [58]

    Zero123++: a single image to consistent multi-view dif- fusion base model

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model. In arXiv, 2023. 2

    2023

  51. [59]

    Mvdream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. In International Conference on Learning Represen- tations (ICLR), 2024. 2, 4

    2024

  52. [60]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Ta- tiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  53. [61]

    Light field networks: Neural scene representations with single-evaluation rendering

    Vincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum, and Fredo Durand. Light field networks: Neural scene representations with single-evaluation rendering. Ad- vances in Neural Information Processing Systems (NeurIPS), 34:19313–19325, 2021. 4

    2021

  54. [62]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations (ICLR), 2021. 2

    2021

  55. [63]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1, 2, 6

    2024 arXiv

  56. [64]

    Emu: Generative pretraining in multimodality

    Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Emu: Generative pretraining in multimodality. In International Conference on Learning Representations (ICLR), 2024. 1

    2024

  57. [65]

    Splatter image: Ultra-fast single-view 3d recon- struction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d recon- struction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10208–10217, 2024. 1, 6, 14

    2024

  58. [66]

    Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior

    Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen. Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22819–22829, 2023. 2

    2023

  59. [67]

    Lgm: Large multi-view gaus- sian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaus- sian model for high-resolution 3d content creation. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 1–18. Springer, 2024. 2, 4, 5, 7, 14

    2024

  60. [68]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in Neural In- formation Processing Systems (NeurIPS) , 37:84839–84865,

  61. [69]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1, 2, 3, 5

    2023 arXiv

  62. [70]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 1, 2, 3, 5, 6

    2023 arXiv

  63. [71]

    Lion: Latent point diffu- sion models for 3d shape generation

    Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point diffu- sion models for 3d shape generation. Advances in Neural In- formation Processing Systems (NeurIPS) , 35:10021–10039,

  64. [72]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017. 2, 3, 4

    2017

  65. [73]

    Rodin: A generative model for sculpting 3d digital avatars using diffusion

    Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2023

  66. [74]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 2

    2024 arXiv

  67. [75]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems (NeurIPS), 36:8406–8441, 2023. 2

    2023

  68. [76]

    Crm: Single image to 3d textured mesh with convo- lutional reconstruction model

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convo- lutional reconstruction model. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV), pages 57–74....

    2024

  69. [77]

    Phidias: A gen- erative model for creating 3d content from text, image, and 3d conditions with reference-augmented diffusion

    Zhenwei Wang, Tengfei Wang, Zexin He, Gerhard Petrus Hancke, Ziwei Liu, and Rynson WH Lau. Phidias: A gen- erative model for creating 3d content from text, image, and 3d conditions with reference-augmented diffusion. In Inter- national Conference on Learning Representations (ICLR) ,

  70. [78]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024. 3, 5, 8

    arXiv

  71. [79]

    Multiview compres- sive coding for 3d reconstruction

    Chao-Yuan Wu, Justin Johnson, Jitendra Malik, Christoph Feichtenhofer, and Georgia Gkioxari. Multiview compres- sive coding for 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9065–9075, 2023. 4

    2023

  72. [80]

    Direct3d: Scal- able image-to-3d generation via 3d latent diffusion trans- former

    Shuang Wu, Youtian Lin, Yifei Zeng, Feihu Zhang, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scal- able image-to-3d generation via 3d latent diffusion trans- former. In Advances in Neural Information Processing Sys- tems (NeurIPS), 2024. 4

    2024

  73. [81]

    Latte3d: Large-scale amortized text-to-enhanced3d synthe- sis

    Kevin Xie, Jonathan Lorraine, Tianshi Cao, Jun Gao, James Lucas, Antonio Torralba, Sanja Fidler, and Xiaohui Zeng. Latte3d: Large-scale amortized text-to-enhanced3d synthe- sis. In Proceedings of the European Conference on Com- puter Vision (ECCV), pages 305–322. Springer, 2024. 4

    2024

  74. [82]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

    arXiv

  75. [83]

    Pointllm: Empowering large language models to understand point clouds

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiang- miao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 131–147. Springer, 2024. 1, 3, 5, 6

    2024

  76. [84]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF Conference o...

  77. [85]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. Transac- tions on Machine Learning Research, 2022. 6

    2022

  78. [86]

    Language model beats diffusion-tokenizer is key to visual generation

    Lijun Yu, Jose Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G Hauptmann, et al. Language model beats diffusion-tokenizer is key to visual generation. In International Conference on Learning Repre- s...

    2024

  79. [87]

    An image is worth 32 tokens for reconstruction and generation

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems (NeurIPS) , 37: 128940–128966, 2024. 2

    2024

  80. [88]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19313–19322, 2022. 2, 3

    2022

  81. [89]

    3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023

    Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023. 2

    2023

  82. [90]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 1, 2, 8

    2024

  83. [91]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 586–595, 2018. 4

    2018

  84. [92]

    Multi-HeadSelf Attention Multi-HeadCross Attention ×N Scale, Shift Layer Norm CLIPT Text Scale Scale ⊕ ⊕ (a) Transformer Block (Text condition) FFN𝛾!,𝛽!Scale, Shift 𝛼! Layer Norm𝛾

    Shangchen Zhou, Kelvin Chan, Chongyi Li, and Chen Change Loy. Towards robust blind face restora- tion with codebook lookup transformer. Advances in Neural Information Processing Systems (NeurIPS) , 35: 30599–30611, 2022. 7 Supplemental Materials for SAR3D: Autoregressive 3D Ob...

    2022

  85. [2024]

    1, 2, 3, 4, 6, 8, 13

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.