Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper unifies text-to-3D, image-to-3D, 3D understanding, and 3D editing in one autoregressive model.

desk verdict Solid engineering with a valuable new 3D dataset, but the editing claim is unvalidated and the generation numbers are weaker than the prose admits. read the letter →

arxiv 2506.01853 v1 pith:G55CEGWQ submitted 2025-06-02 cs.CV

Junliang Ye , Zhengyi Wang , Ruowen Zhao , Shenghao Xie , Jun Zhu This is my paper · ORCID
classification cs.CV
keywords 3DnativemultimodallanguagemodelVQVAEdiscretetokenstext-to-3Dgenerationimage-to-3Dobjectcaptioningtext-basedediting3D-Alpacadataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ShapeLLM-Omni claims that a single autoregressive language model can treat 3D as a native token modality, so that text, images, and 3D assets can appear in any order in the input and the model can answer with text or new 3D tokens. The paper reports this as the first unified framework covering text-to-3D, image-to-3D, 3D understanding, and 3D editing in one system. To make that feasible it compresses each 3D mesh into 1024 discrete tokens with an 8192-entry codebook and builds a 2.56M-sample instruction corpus (3.46B tokens) named 3D-Alpaca covering generation, captioning, and editing. The value of the claim is a path to natural-language interactive 3D content creation, where the same model that understands a mesh can also change it or generate a new one from a sentence or a picture.

What carries the argument

The load-bearing object is the 3D VQVAE with its 8192-entry codebook. It encodes a $64^3$ voxel grid into a $16^3$ latent grid, serializes that grid into 4096 tokens, concatenates every four neighboring tokens along the channel dimension to obtain 1024 tokens with 32 channels, and quantizes them into the codebook. Those 1024 tokens are what the LLM learns to read and write, so a mesh is just another token sequence in the same transformer. A second load-bearing mechanism is the 3D-Alpaca corpus; for editing it is built by an image-mediated pipeline that renders an asset, asks ChatGPT-4o to edit the rendered image from a text instruction, and reconstructs the edited image back into a 3D mesh with Trellis, yielding 70k training pairs after filtering.

What would settle it

Take a random sample of the 70k editing pairs, reconstruct each edited mesh, and have raters check two things: whether the requested change is present and whether unchanged parts of the object still match the original. If a substantial fraction of pairs fail either check, the editing supervision is corrupted and the model's editing behavior on held-out prompts would be expected to degrade in proportion.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that 3D can be folded into a multimodal LLM's next-token prediction loop without a task-specific head: a 3D VQVAE turns a $64^{3}$ voxel grid into 1024 discrete tokens, these tokens are added to the LLM vocabulary, and after instruction tuning on 3D-Alpaca the single model handles text-to-3D, image-to-3D, 3D captioning, and instruction-based 3D editing. The same 1024 tokens serve as both the input representation for understanding and the output representation for generation, letting the model read an existing mesh, respond to a prompt, and emit an edited or new mesh in one continuous token stream. The paper reports language benchmarks close to its 7B backbone, captioning scores near a dedicated PointLLM model, and generation quality below but close to Trellis, which it uses as the performance upper bound.

Load-bearing premise

The editing capability stands on the unverified assumption that the auto-generated training pairs—ChatGPT-4o's image edits followed by Trellis 3D reconstruction—actually realize the intended edits while preserving the rest of the object; the paper gives no consistency metric for these pairs and no quantitative 3D-editing evaluation.

Editorial extensions

If this is right

  • A single model can interleave text, images, and 3D tokens, so 3D generation can use multi-turn conversational context instead of a fixed one-shot prompt.
  • Instruction-based 3D editing becomes an autoregressive continuation problem: the model reads the current mesh tokens plus the edit request and emits the edited mesh tokens, supporting iterative changes in one system.
  • The 3D-Alpaca corpus, with 2.56M samples and 3.46B tokens covering generation, captioning, and editing, is a reusable instruction dataset for training other or larger backbones.
  • The 1024-token voxel representation fits 3D within the context windows of existing LLMs while staying accurate enough for downstream reconstruction via a rectified-flow voxel-to-mesh refiner.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the central claim holds, the image-mediated editing pipeline could be reused to mint cheap training pairs for new categories, making 3D-editing supervision scalable.
  • The paper does not explore chains like caption-then-edit; a direct test of the native claim would be whether the model can compose those steps in one session.
  • A testable extension is an ablation of editing-pair count (for example 5k versus 70k pairs) to see whether editing quality is bounded by data quantity or by the fidelity of the automatically generated pairs.
  • Because images stay continuous while 3D is discrete, the architecture is asymmetric, so one prediction is that the model's image-conditioned 3D reasoning should be strong while image generation itself would require a separate image tokenizer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents ShapeLLM-Omni, a Qwen-2.5-VL-Instruct-7B model extended with a discrete 3D token vocabulary from a VQVAE that maps 64^3 voxel grids to 1024 tokens. The model is trained in a fully autoregressive next-token prediction setting on 3D-Alpaca, a newly constructed corpus containing text-to-3D, image-to-3D, 3D captioning, and language-driven 3D editing instances, together with UltraChat for language preservation. The authors report that the model retains language capabilities, achieves generation quality below Trellis but above most baselines on Toys4K metrics, is competitive with PointLLM on captioning, and qualitatively supports 3D editing. The main contributions are the unified framework and the large 3D instruction dataset.

Significance. If the empirical claims are confirmed, this is a useful step toward native 3D multimodal LLMs: the early fusion of discrete 3D tokens with text and images inside a single autoregressive model is a natural and scalable design, and the 3D-Alpaca corpus (2.56M samples, 3.46B tokens) is a substantial community resource. The authors are commendably transparent that Trellis, the model that supplies the tokenizer and decoder, outperforms the proposed system on all generation metrics. However, the paper's headline claim of a four-capability system is only partially supported: the editing leg lacks any quantitative validation, and several quantitative statements are overstated relative to the tables. These issues are local and addressable.

major comments (4)
  1. [§3.4, §4.3] The claimed 3D editing capability is not established. In Section 3.4, steps (5)–(6), each editing pair is produced by rendering a 3D asset to a frontal image, asking ChatGPT-4o to edit the image, and reconstructing the edited image with Trellis; no identity-preservation score, edit-success rate, or reconstruction-consistency metric is reported for the 70k pairs. The only evaluation in Section 4.3 is the qualitative Figure 6, and Appendix A.2 shows dataset examples, not an independent test. Because editing is one of the four central capabilities claimed in Section 3.2 and the paper's own Limitation states that 70k pairs are far too few for ChatGPT-4o-level editing, this claim needs either quantitative validation (e.g., held-out edit alignment and identity metrics, human study) or a substantially weakened formulation.
  2. [§4.2, Table 5; §3.4] No train/test separation is reported for the 3D understanding evaluation. 3D-Alpaca is built from Objaverse-XL/Objaverse plus other 3D datasets, and Table 5 evaluates captioning on PointLLM's curated Objaverse test set. The paper must state explicitly that the test assets were excluded from 3D-Alpaca; without that statement, the captioning numbers in Table 5 may be inflated by train/test overlap. The same clarification is needed for the Toys4K generation evaluation in Table 4, since the paper does not state that Toys4K test prompts/assets are disjoint from the 3D-Alpaca training collection.
  3. [§4.2, Tables 4 and 5] Two quantitative claims in the text are not supported by the reported numbers. First, "our generation results outperform all baseline methods except for Trellis" is contradicted by Table 4: on image-to-3D CLIP score, SAR3D obtains 84.70 while ShapeLLM-Omni obtains 84.5; moreover, Trellis is better than the proposed model on every generation metric, not merely in an unspecified way. Second, Table 5's caption "our model achieves better performance on 3D understanding/caption tasks" is misleading: PointLLM-13B* is higher on Sentence-BERT (50.15 vs. 48.34) and SimCSE (50.83 vs. 49.72), and the text in Section 4.2 calling the model "second only to PointLLM" is therefore not accurate across metrics. These overstatements should be corrected.
  4. [§3.4, Table 2; §4.1 and Appendix B.1] Several dataset and training numbers are internally inconsistent. The Introduction states 62k paired 3D meshes for editing, Section 3.4 reports 70k valid editing samples, and Table 2 reports 420k items for 3D-Edit; the relationship between pairs and dialogue items should be stated. The Introduction also gives 710k text/image-3D pairs while Section 3.4 and Table 2 use 712k. For training, Section 4.1 reports a learning-rate decay from 5e-5 to 5e-6 and 15 epochs, whereas Appendix B.1 reports a constant 1e-5 learning rate and 60k iterations; the schedule and stopping criterion should be reconciled.
minor comments (6)
  1. [Table 3] The SIQA row for Chameleon-7B appears as "5740.4", which is either a formatting error or a corrupted value; the bold/underline markers in this table are also applied inconsistently.
  2. [Table 5] The column header "BLUE-1" should read "BLEU-1", and the duplicate references [22] and [23] should be merged.
  3. [Figure 10] Figure 10 has the caption "Some cases of 3D editing result from our method" but it shows 3D-to-caption results; this caption appears to have been copied from Figure 6 and should be corrected.
  4. [§3.3] The description of channel concatenation ("4096 tokens with 8 channels into 1024 tokens with 32 channels") should specify how neighboring tokens are grouped, for example by reshaping the 16^3 latent grid before concatenation, so the operation is unambiguous.
  5. [Abstract] The abstract contains a sentence fragment: "Finally, by performing instruction-based training of the Qwen-2.5-vl-7B-Instruct model on the 3D-Alpaca dataset." should be completed or merged with the following sentence.
  6. [§3.4] The captioning ground truth is produced by Qwen-2.5-VL-Instruct, the same model family being fine-tuned; this self-distillation choice should be acknowledged as a limitation, since it may bias the teacher-student relationship and the diversity of captions.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: central claims are benchmarked externally; the unvalidated 3D-editing pipeline and Qwen-based caption self-distillation are evidence gaps, not circular reductions.

full rationale

This is an empirical systems paper rather than a derivation chain. The central claims are evaluated against external benchmarks and baselines: language abilities on MMLU/PIQA/GSM8K/SIQA, 3D generation on Toys4K against CRM, SAR3D, 3DTopia-XL, and TRELLIS, and 3D-to-caption on PointLLM's Objaverse benchmark. The 3D VQVAE is trained with its own reconstruction objective and ablated on Chamfer/Hausdorff distance. No 'prediction' in the paper is a fitted parameter renamed as a prediction by construction. The closest concern is that 3D-Alpaca captions are generated by Qwen-2.5-VL-Instruct and the model is initialized from Qwen-2.5-VL-7B-Instruct (Sec. 3.4 and Sec. 4.1), making the captioning task a self-distillation; however, the paper does not present captioning as a derived theorem, and the quantitative caption evaluation uses the external PointLLM benchmark, so this is a data-construction choice rather than a circular reduction. The 3D editing leg uses an image-mediated pipeline (render, ChatGPT-4o edit, TRELLIS reconstruction) with no quantitative consistency or identity metric (Sec. 3.4 and Sec. 4.3); this is a correctness and evidence gap, not circularity, because the model's edited mesh is not identical to the pipeline input by construction. The paper cites the authors' prior works (e.g., LLaMA-Mesh, CRM, ProlificDreamer) contextually in Related Work; none is invoked as a load-bearing uniqueness theorem or as the justification for a forced choice. Overall, no step reduces to its own inputs; score 2 reflects minor non-load-bearing self-citations and the self-distillation, not a circular derivation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central claim rests on existing Trellis components, LLM-generated captions, and ChatGPT-4o-mediated editing pairs. The main free parameters are dataset construction choices and the codebook size, all selected by hand or by small ablations. No new physical entities are introduced; the grey texture latent is the only new latent-code artifact.

free parameters (6)
  • 3D VQVAE codebook size = 8192
    Chosen by ablation in Appendix B.3; larger 16384 does not improve reconstruction (CD 0.0095 vs 0.0094), so 8192 selected.
  • Token concatenation factor = 4
    Neighboring channel tokens concatenated 4-to-1 to reduce 4096 tokens to 1024, following Chameleon-style image tokenization [71].
  • Editing prompts per category = 20
    ChatGPT-4o asked for 20 feasible edits per category, manually filtered to 371 unique prompts in Section 3.4.
  • Assets per editing prompt = 200
    Resource-limited sampling choice in Section 3.4, yielding 70k valid editing pairs.
  • Dialogue templates per task = 25
    25 templates per task; 6 randomly selected per edit instance, producing 420k editing dialogues as shown in Table 2.
  • Generation sampling hyperparameters = top-k=8192, top-p=0.7, temperature=0.7
    Used for all 3D generation evaluations in Section 4.2; not fitted but chosen by hand.
assumptions (5)
  • domain assumption 64^3 voxel grids preserve enough skeletal structure for high-quality mesh reconstruction.
    Section 3.3 states this resolution balances structure and redundancy, citing Trellis; final mesh quality depends on the voxel-to-mesh decoder.
  • domain assumption Qwen-2.5-VL-Instruct captions are high-quality ground truth for 3D captioning and text-to-3D prompts.
    Section 3.4 uses these captions as both prompts and ground-truth targets, without human verification or filtering described.
  • domain assumption ChatGPT-4o image edits lifted through Trellis image-to-3D reconstruction produce valid, identity-preserving 3D editing pairs.
    Section 3.4 steps 5 and 6; no consistency metric is reported for the 70k pairs.
  • domain assumption Trellis pretrained VQVAE and rectified-flow decoder parameters are a suitable base and remain valid after codebook fine-tuning.
    Section 4.1 Stage 1 and Stage 2 plus Section 3.3; the model relies on Trellis for both tokenizer and mesh decoder.
  • domain assumption Toys4K test prompts and images are representative for text-to-3D and image-to-3D evaluation.
    Section 4.2 evaluates on Toys4K without discussing distribution shift from the Objaverse/Trellis training data.
invented entities (1)
  • Grey texture latent
    purpose: Used in the voxel-to-mesh decoder to support generation of non-textured meshes.
    Introduced in Appendix B.1; no external evidence provided that this choice preserves reconstruction quality beyond qualitative examples.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding." pith.science (2026). https://pith.science/paper/G55CEGWQ

@misc{pith2026250601853,
  author       = {Pith},
  title        = {Pith review of: ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G55CEGWQ}},
  note         = {Machine review of arXiv:2506.01853}
}
read the original abstract

Recently, the powerful text-to-image capabilities of ChatGPT-4o have led to growing appreciation for native multimodal large language models. However, its multimodal capabilities remain confined to images and text. Yet beyond images, the ability to understand and generate 3D content is equally crucial. To address this gap, we propose ShapeLLM-Omni-a native 3D large language model capable of understanding and generating 3D assets and text in any sequence. First, we train a 3D vector-quantized variational autoencoder (VQVAE), which maps 3D objects into a discrete latent space to achieve efficient and accurate shape representation and reconstruction. Building upon the 3D-aware discrete tokens, we innovatively construct a large-scale continuous training dataset named 3D-Alpaca, encompassing generation, comprehension, and editing, thus providing rich resources for future research and training. Finally, by performing instruction-based training of the Qwen-2.5-vl-7B-Instruct model on the 3D-Alpaca dataset. Our work provides an effective attempt at extending multimodal models with basic 3D capabilities, which contributes to future research in 3D-native AI. Project page: https://github.com/JAMESYJL/ShapeLLM-Omni

Figures

Figures reproduced from arXiv: 2506.01853 by the authors.

Figure 1
Figure 1. ShapeLLM-Omni inherits Qwen2.5-vl’s strong multimodal capabilities and additionally [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of 3D VQVAE, which can compress voxels into discrete tokens. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Our proposed 3D-Alpaca dataset comprises 3D generation, 3D understanding, and 3D [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Comparisons with other baselines on the image-to-3D task. Our results demonstrate more complete geometry and high-fidelity textures compared to baselines, enabling photorealistic image-to-3D generation. 4.3 Qualitative comparisons 3D Generation To evaluate the effectiv…
Figure 5
Figure 5. Figure 5: Comparisons with other baselines on text-to-3d task. Compared to other methods, our results show better text alignment, with generated 3D shapes accurately reflecting the input descriptions. software-based 3D content creation pipelines, which are usually time-consuming…
Figure 6
Figure 6. Figure 6: Some cases of 3D editing result from our method. Our method enables the editing of 3D assets based on textual instructions while preserving their original identity and visual consistency. 5 Conclusion In this work, we introduce ShapeLLM-Omni, a novel framework that adv…
Figure 7
Figure 7. Figure 7: About how to generate 3d mesh from voxel. The upper part illustrates the process of reconstructing a textured mesh from voxel inputs using a texture transformer [91] and mesh decoder. In contrast, the lower part demonstrates the pipeline for reconstructing a non-textur…
Figure 8
Figure 8. Figure 8: Training Loss Curve and Testing Loss Curve 0ea33b6617174530b97d6b7a92c275fb InstructBLIP UID 3D-LLM PointLLM An appleavatar 3d model A 3D model of a red apple. This is a 3D model of a unique apple, distinctively adorned with a single, vibrant green leaf at the top. Sha…
Figure 9
Figure 9. Figure 9: Qualitative results on Objaverse. B.2 More Qualitative comparisons In [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Some cases of 3D editing result from our method [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Some cases of our 3D-Editing Data 21 [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: More cases of Image-to-3D result from our method. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: More cases of Image-to-3D result from our method. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: More cases of Image-to-3D result from our method. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: More cases of Text-to-3D result from our method. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets

    cs.RO 2026-08 conditional novelty 6.0 of 10

    PhysX-CoT turns single-image 3D asset generation into an explicit, ordered, supervised chain of physical states, beating an output-centric VLM baseline on geometry and physical attributes.

  2. Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A unified 3D multimodal model combines understanding, text-to-3D generation, instruction-guided editing, and part generation in one architecture, trained on an 87M-sample corpus, with claimed state-of-the-art results.

  3. ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    ELSA3D introduces elastic semantic anchoring via sparse anchor tokens and a scale-aware octree tokenizer to unify 3D generation and captioning at reduced computational cost.

Reference graph

Works this paper leans on

108 extracted references · 12 canonical work pages · cited by 3 Pith papers

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    2022

  3. [3]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  5. [5]

    Demystifying mmd gans

    Mikołaj Bi´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. arXiv preprint arXiv:1801.01401, 2018

    arXiv 2018

  6. [6]

    Piqa: Reasoning about physical common- sense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical common- sense in natural language. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020

    2020

  7. [7]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015

    arXiv 2015

  8. [8]

    Pointgpt: Auto-regressively generative pre-training from point clouds.Advances in Neural Information Processing Systems, 36: 29667–29679, 2023

    Guangyan Chen, Meiling Wang, Yi Yang, Kai Yu, Li Yuan, and Yufeng Yue. Pointgpt: Auto-regressively generative pre-training from point clouds.Advances in Neural Information Processing Systems, 36: 29667–29679, 2023

    2023

Show all 108 references
  1. [9]

    Microdreamer: Zero- shot 3d generation in 20 seconds by score-based iterative reconstruction.arXiv e-prints, pages arXiv–2404, 2024

    Luxi Chen, Zhengyi Wang, Chongxuan Li, Tingting Gao, Hang Su, and Jun Zhu. Microdreamer: Zero- shot 3d generation in 20 seconds by score-based iterative reconstruction.arXiv e-prints, pages arXiv–2404, 2024

    2024

  2. [10]

    Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation.arXiv preprint arXiv:2303.13873, 2023

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation.arXiv preprint arXiv:2303.13873, 2023

    2023 arXiv

  3. [11]

    Meshxl: Neural coordinate field for generative 3d foundation models.Advances in Neural Information Processing Systems, 37:97141–97166, 2025

    Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Billzb Wang, Jingyi Yu, Gang Yu, et al. Meshxl: Neural coordinate field for generative 3d foundation models.Advances in Neural Information Processing Systems, 37:97141–97166, 2025

    2025

  4. [12]

    Meshanything: Artist-created mesh generation with autoregressive transformers

    Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Jiaxiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, et al. Meshanything: Artist-created mesh generation with autoregressive transformers. arXiv preprint arXiv:2406.10163, 2024. 12

    2024 arXiv

  5. [13]

    Meshanything v2: Artist-created mesh generation with adjacent mesh tokenization.arXiv preprint arXiv:2408.02555, 2024

    Yiwen Chen, Yikai Wang, Yihao Luo, Zhengyi Wang, Zilong Chen, Jun Zhu, Chi Zhang, and Guosheng Lin. Meshanything v2: Artist-created mesh generation with adjacent mesh tokenization.arXiv preprint arXiv:2408.02555, 2024

    2024 arXiv

  6. [14]

    Sar3d: Autoregressive 3d object generation and understanding via multi-scale 3d vqvae

    Yongwei Chen, Yushi Lan, Shangchen Zhou, Tengfei Wang, and Xingang Pan. Sar3d: Autoregressive 3d object generation and understanding via multi-scale 3d vqvae. InCVPR, 2025

    2025

  7. [15]

    3dtopia-xl: Scaling high-quality 3d asset generation via primitive diffusion.arXiv preprint arXiv:2409.12957, 2024

    Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, et al. 3dtopia-xl: Scaling high-quality 3d asset generation via primitive diffusion.arXiv preprint arXiv:2409.12957, 2024

    2024 arXiv

  8. [16]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer visi...

    2024

  9. [17]

    Text-to-3d using gaussian splatting

    Zilong Chen, Feng Wang, Yikai Wang, and Huaping Liu. Text-to-3d using gaussian splatting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21401– 21412, 2024

    2024

  10. [18]

    V3d: Video diffusion models are effective 3d generators.arXiv preprint arXiv:2403.06738, 2024

    Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang, and Huaping Liu. V3d: Video diffusion models are effective 3d generators.arXiv preprint arXiv:2403.06738, 2024

    2024 arXiv

  11. [19]

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    2024 arXiv

  12. [20]

    Training verifiers to solve math word problems, 2021.URL https://arxiv

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems, 2021.URL https://arxiv. org/abs/2110.14168, 9, 2021

    2021 arXiv

  13. [21]

    Abo: Dataset and benchmarks for real-world 3d object understanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object understanding. InProceedings of the IEEE/CVF conference on comp...

    2022

  14. [23]

    Objaverse-xl: A universe of 10m+ 3d objects.Advances in Neural Information Processing Systems, 36:35799–35813, 2023

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects.Advances in Neural Information Processing Systems, 36:35799–35813, 2023

    2023

  15. [24]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023

    2023 arXiv

  16. [25]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024

    2024 arXiv

  17. [26]

    Fila-video: Spatio-temporal compression for fine-grained long video understanding.arXiv preprint arXiv:2504.20384, 2025

    Yanan Guo, Wenhui Dong, Jun Song, Shiding Zhu, Xuan Zhang, Hanqing Yang, Yingbo Wang, Yang Du, Xianing Chen, and Bo Zheng. Fila-video: Spatio-temporal compression for fine-grained long video understanding.arXiv preprint arXiv:2504.20384, 2025

    2025 arXiv

  18. [27]

    Meshtron: High-fidelity, artist-like 3d mesh generation at scale.arXiv preprint arXiv:2412.09548, 2024

    Zekun Hao, David W Romero, Tsung-Yi Lin, and Ming-Yu Liu. Meshtron: High-fidelity, artist-like 3d mesh generation at scale.arXiv preprint arXiv:2412.09548, 2024

    2024 arXiv

  19. [28]

    Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  20. [29]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  21. [30]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 13

    2020

  22. [31]

    Lrm: Large reconstruction model for single image to 3d.arXiv preprint arXiv:2311.04400, 2023

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d.arXiv preprint arXiv:2311.04400, 2023

    2023 arXiv

  23. [32]

    3d-llm: Injecting the 3d world into large language models.Advances in Neural Information Processing Systems, 36:20482–20494, 2023

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models.Advances in Neural Information Processing Systems, 36:20482–20494, 2023

    2023

  24. [33]

    Spar3d: Stable point-aware reconstruction of 3d objects from single images.arXiv preprint arXiv:2501.04689, 2025

    Zixuan Huang, Mark Boss, Aaryaman Vasishta, James M Rehg, and Varun Jampani. Spar3d: Stable point-aware reconstruction of 3d objects from single images.arXiv preprint arXiv:2501.04689, 2025

    2025 arXiv

  25. [35]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  26. [36]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    2023

  27. [37]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model.arXiv preprint arXiv:2311.06214, 2023

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model.arXiv preprint arXiv:2311.06214, 2023

    2023 arXiv

  28. [38]

    Sweetdreamer: Aligning geometric priors in 2d diffusion for consistent text-to-3d.arxiv:2310.02596, 2023

    Weiyu Li, Rui Chen, Xuelin Chen, and Ping Tan. Sweetdreamer: Aligning geometric priors in 2d diffusion for consistent text-to-3d.arxiv:2310.02596, 2023

    2023 arXiv

  29. [39]

    Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner.arXiv preprint arXiv:2405.14979, 2024

    Weiyu Li, Jiarui Liu, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner.arXiv preprint arXiv:2405.14979, 2024

    2024 arXiv

  30. [40]

    Llama-vid: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. InEuropean Conference on Computer Vision, pages 323–340. Springer, 2024

    2024

  31. [41]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  32. [42]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  33. [43]

    Reconx: Reconstruct any scene from sparse views with video diffusion model, 2024

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Reconx: Reconstruct any scene from sparse views with video diffusion model, 2024. URL https://arxiv.org/abs/2408.16767

    2024 arXiv

  34. [44]

    World model on million-length video and language with blockwise ringattention.arXiv preprint arXiv:2402.08268, 2024

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with blockwise ringattention.arXiv preprint arXiv:2402.08268, 2024

    2024 arXiv

  35. [45]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  36. [46]

    One-2- 3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization.Advances in Neural Information Processing Systems, 36:22226–22246, 2023

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2- 3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization.Advances in Neural Information Processing Systems, 36:22226–22246, 2023

    2023

  37. [47]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. InProceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023

    2023

  38. [48]

    Syncdreamer: Generating multiview-consistent images from a single-view image.arXiv preprint arXiv:2309.03453, 2023

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consistent images from a single-view image.arXiv preprint arXiv:2309.03453, 2023

    2023 arXiv

  39. [49]

    Wonder3d: Single image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single image to 3d using cross-domain diffusion. InProceedings of the IEEE/CVF conference on computer vision and pattern...

    2024

  40. [50]

    Marching cubes: A high resolution 3d surface construction algorithm

    William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. InSeminal graphics: pioneering efforts that shaped the field, pages 347–353. 1998

    1998

  41. [51]

    Video-chatgpt: Towards detailed video understanding via large vision and language models.arXiv preprint arXiv:2306.05424, 2023

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models.arXiv preprint arXiv:2306.05424, 2023

    2023 arXiv

  42. [52]

    Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021

    2021

  43. [53]

    Hierarchical transformers are more efficient language models.arXiv preprint arXiv:2110.13711, 2021

    Piotr Nawrot, Szymon Tworkowski, Michał Tyrolski, Łukasz Kaiser, Yuhuai Wu, Christian Szegedy, and Henryk Michalewski. Hierarchical transformers are more efficient language models.arXiv preprint arXiv:2110.13711, 2021

    2021 arXiv

  44. [54]

    Barron, and Ben Mildenhall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv, 2022

    2022

  45. [55]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. InEuropean Conference on Computer Vision, pages 214–238. Springer, 2024

    2024

  46. [56]

    Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to-3d

    Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mutian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong, Liefeng Bo, and Xiaoguang Han. Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to-3d. InProceedings of the IEEE/CVF conference on computer ...

    2024

  47. [57]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  48. [58]

    Dreambooth3d: Subject-driven text-to- 3d generation

    Amit Raj, Srinivas Kaza, Ben Poole, Michael Niemeyer, Nataniel Ruiz, Ben Mildenhall, Shiran Zada, Kfir Aberman, Michael Rubinstein, Jonathan Barron, et al. Dreambooth3d: Subject-driven text-to- 3d generation. InProceedings of the IEEE/CVF international conference on computer v...

    2023

  49. [59]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  50. [60]

    Socialiqa: Commonsense reasoning about social interactions.arXiv preprint arXiv:1904.09728, 2019

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Commonsense reasoning about social interactions.arXiv preprint arXiv:1904.09728, 2019

    1904 arXiv

  51. [61]

    Zero123++: a single image to consistent multi-view diffusion base model

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view diffusion base model. arXiv preprint arXiv:2310.15110, 2023

    2023 arXiv

  52. [62]

    Mvdream: Multi-view diffusion for 3d generation.arXiv preprint arXiv:2308.16512, 2023

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation.arXiv preprint arXiv:2308.16512, 2023

    2023 arXiv

  53. [63]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern ...

    2024

  54. [64]

    Meta 3d assetgen: Text-to-mesh generation with high-quality geometry, texture, and pbr materials

    Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahendra Kariya, Yanir Kleiman, Emilien Gar- reau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, and David Novotny. Meta 3d assetgen: Text-to-mesh generation with high-quality geometry, texture, and pbr material...

    2024

  55. [65]

    Using shape to categorize: Low-shot learning with an explicit shape bias

    Stefan Stojanov, Anh Thai, and James M Rehg. Using shape to categorize: Low-shot learning with an explicit shape bias. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1798–1808, 2021. 15

    2021

  56. [66]

    Dreamcraft3d: Hierarchical 3d generation with bootstrapped diffusion prior.arXiv preprint arXiv:2310.16818, 2023

    Jingxiang Sun, Bo Zhang, Ruizhi Shao, Lizhen Wang, Wen Liu, Zhenda Xie, and Yebin Liu. Dreamcraft3d: Hierarchical 3d generation with bootstrapped diffusion prior.arXiv preprint arXiv:2310.16818, 2023

    2023 arXiv

  57. [67]

    Rethinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016

    2016

  58. [68]

    Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.arXiv preprint arXiv:2309.16653, 2023

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.arXiv preprint arXiv:2309.16653, 2023

    2023 arXiv

  59. [69]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024

    2024 arXiv

  60. [70]

    Edgerunner: Auto-regressive auto-encoder for artistic mesh generation.arXiv preprint arXiv:2409.18114, 2024

    Jiaxiang Tang, Zhaoshuo Li, Zekun Hao, Xian Liu, Gang Zeng, Ming-Yu Liu, and Qinsheng Zhang. Edgerunner: Auto-regressive auto-encoder for artistic mesh generation.arXiv preprint arXiv:2409.18114, 2024

    2024 arXiv

  61. [71]

    Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024. doi: 10.48550/arXiv.2405.09818. URL https://github.com/ facebookresearch/chameleon

    2024 doi arXiv

  62. [72]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  63. [73]

    Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    2017

  64. [74]

    Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. InEuropean Conference on Computer Vi...

    2024

  65. [75]

    Yeh, and Greg Shakhnarovich

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation.arXiv preprint arXiv:2212.00774, 2022

    2022 arXiv

  66. [76]

    Imagedream: Image-prompt multi-view diffusion for 3d generation.arXiv preprint arXiv:2312.02201, 2023

    Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation.arXiv preprint arXiv:2312.02201, 2023

    2023 arXiv

  67. [77]

    Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction.arXiv preprint arXiv:2311.12024, 2023

    Peng Wang, Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, and Kai Zhang. Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction.arXiv preprint arXiv:2311.12024, 2023

    2023 arXiv

  68. [78]

    Rodin: A generative model for sculpting 3d digital avatars using diffusion

    Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. InProceedings of the IEEE/CVF conference on computer vision and ...

    2023

  69. [79]

    Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    2024 arXiv

  70. [80]

    Animatabledreamer: Text-guided non-rigid 3d model generation and reconstruction with canonical score distillation.arXiv preprint arXiv:2312.03795, 2023

    Xinzhou Wang, Yikai Wang, Junliang Ye, Zhengyi Wang, Fuchun Sun, Pengkun Liu, Ling Wang, Kai Sun, Xintong Wang, and Bin He. Animatabledreamer: Text-guided non-rigid 3d model generation and reconstruction with canonical score distillation.arXiv preprint arXiv:2312.03795, 2023

    2023 arXiv

  71. [81]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  72. [82]

    Llama-mesh: Unifying 3d mesh generation with language models.arXiv preprint arXiv:2411.09595, 2024

    Zhengyi Wang, Jonathan Lorraine, Yikai Wang, Hang Su, Jun Zhu, Sanja Fidler, and Xiaohui Zeng. Llama-mesh: Unifying 3d mesh generation with language models.arXiv preprint arXiv:2411.09595, 2024

    2024 arXiv

  73. [83]

    Crm: Single image to 3d textured mesh with convolutional reconstruction model.arXiv preprint arXiv:2403.05034, 2024

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model.arXiv preprint arXiv:2403.05034, 2024. 16

    2024 arXiv

  74. [84]

    Meshlrm: Large reconstruction model for high-quality meshes.arXiv preprint arXiv:2404.12385, 2024

    Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. Meshlrm: Large reconstruction model for high-quality meshes.arXiv preprint arXiv:2404.12385, 2024

    2024 arXiv

  75. [85]

    Consistent123: Improve consistency for one image to 3d object synthesis.arXiv preprint arXiv:2310.08092, 2023

    Haohan Weng, Tianyu Yang, Jianan Wang, Yu Li, Tong Zhang, CL Chen, and Lei Zhang. Consistent123: Improve consistency for one image to 3d object synthesis.arXiv preprint arXiv:2310.08092, 2023

    2023 arXiv

  76. [86]

    Pivotmesh: Generic 3d mesh generation via pivot vertices guidance.arXiv preprint arXiv:2405.16890, 2024

    Haohan Weng, Yikai Wang, Tong Zhang, CL Chen, and Jun Zhu. Pivotmesh: Generic 3d mesh generation via pivot vertices guidance.arXiv preprint arXiv:2405.16890, 2024

    2024 arXiv

  77. [87]

    Scaling mesh generation via compressive tokenization.arXiv preprint arXiv:2411.07025, 2024

    Haohan Weng, Zibo Zhao, Biwen Lei, Xianghui Yang, Jian Liu, Zeqiang Lai, Zhuo Chen, Yuhong Liu, Jie Jiang, Chunchao Guo, et al. Scaling mesh generation via compressive tokenization.arXiv preprint arXiv:2411.07025, 2024

    2024 arXiv

  78. [88]

    Instructblip: Towards general-purpose vision-language models with instruction tuning [c]

    D Wenliang, L Junnan, L Dongxu, T Anthony Meng Huat, Z Junqi, W Weisheng, L Boyang, F Pascale, and H Steven. Instructblip: Towards general-purpose vision-language models with instruction tuning [c]. Advances in Neural Information Processing Systems, 36, 2023

    2023

  79. [89]

    Unique3d: High-quality and efficient 3d mesh generation from a single image

    Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, Yueqi Duan, and Kaisheng Ma. Unique3d: High-quality and efficient 3d mesh generation from a single image. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  80. [90]

    Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer.arXiv preprint arXiv:2405.14832, 2024

    Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer.arXiv preprint arXiv:2405.14832, 2024

    2024 arXiv

  81. [91]

    Structured 3d latents for scalable and versatile 3d generation.arXiv preprint arXiv:2412.01506, 2024

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation.arXiv preprint arXiv:2412.01506, 2024

    2024 arXiv

  82. [92]

    Show-o: One single transformer to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528, 2024

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528, 2024

    2024 arXiv

  83. [93]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models.arXiv preprint arXiv:2404.07191, 2024

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models.arXiv preprint arXiv:2404.07191, 2024

    2024 arXiv

  84. [94]

    Pointllm: Empow- ering large language models to understand point clouds

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empow- ering large language models to understand point clouds. InEuropean Conference on Computer Vision, pages 131–147. Springer, 2024

    2024

  85. [95]

    Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model.arXiv preprint arXiv:2311.09217, 2023

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model.arXiv preprint arXiv:2311.09217, 2023

    2023 arXiv

  86. [96]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Car- los Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. InProceedings of the IEEE/CVF conference on ...

    2023

  87. [97]

    Hunyuan3d 1.0: A unified framework for text-to-3d and image-to-3d generation.arXiv preprint arXiv:2411.02293, 2024

    Xianghui Yang, Huiwen Shi, Bowen Zhang, Fan Yang, Jiacheng Wang, Hongxu Zhao, Xinhai Liu, Xinzhou Wang, Qingxiang Lin, Jiaao Yu, et al. Hunyuan3d 1.0: A unified framework for text-to-3d and image-to-3d generation.arXiv preprint arXiv:2411.02293, 2024

    2024 arXiv

  88. [98]

    Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 2024

    Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 2024

    2024

  89. [99]

    Hi3dgen: High-fidelity 3d geometry generation from images via normal bridging.arXiv preprint arXiv:2503.22236, 3, 2025

    Chongjie Ye, Yushuang Wu, Ziteng Lu, Jiahao Chang, Xiaoyang Guo, Jiaqing Zhou, Hao Zhao, and Xiaoguang Han. Hi3dgen: High-fidelity 3d geometry generation from images via normal bridging.arXiv preprint arXiv:2503.22236, 3, 2025

    2025 arXiv

  90. [100]

    Dreamreward: Text-to-3d generation with human preference

    Junliang Ye, Fangfu Liu, Qixiu Li, Zhengyi Wang, Yikai Wang, Xinzhou Wang, Yueqi Duan, and Jun Zhu. Dreamreward: Text-to-3d generation with human preference. InEuropean Conference on Computer Vision, pages 259–276. Springer, 2024. 17

    2024

  91. [101]

    Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models

    Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision an...

    2024

  92. [102]

    Geolrm: Geometry-aware large reconstruction model for high-quality 3d gaussian generation.arXiv preprint arXiv:2406.15333, 2024

    Chubin Zhang, Hongliang Song, Yi Wei, Yu Chen, Jiwen Lu, and Yansong Tang. Geolrm: Geometry-aware large reconstruction model for high-quality 3d gaussian generation.arXiv preprint arXiv:2406.15333, 2024

    2024 arXiv

  93. [103]

    Gs-lrm: Large reconstruction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large reconstruction model for 3d gaussian splatting. InEuropean Conference on Computer Vision, pages 1–19. Springer, 2024

    2024

  94. [104]

    Clay: A controllable large-scale generative model for creating high-quality 3d assets

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024

    2024

  95. [105]

    Flexidreamer: single image-to-3d generation with flexicubes.arXiv preprint arXiv:2404.00987, 2024

    Ruowen Zhao, Zhengyi Wang, Yikai Wang, Zihan Zhou, and Jun Zhu. Flexidreamer: single image-to-3d generation with flexicubes.arXiv preprint arXiv:2404.00987, 2024

    2024 arXiv

  96. [106]

    Deepmesh: Auto-regressive artist-mesh creation with reinforcement learning.arXiv preprint arXiv:2503.15265, 2025

    Ruowen Zhao, Junliang Ye, Zhengyi Wang, Guangce Liu, Yiwen Chen, Yikai Wang, and Jun Zhu. Deepmesh: Auto-regressive artist-mesh creation with reinforcement learning.arXiv preprint arXiv:2503.15265, 2025

    2025 arXiv

  97. [107]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation.Advances in neural information processing systems, 36:73969–73982, 2023

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation.Advances in neural information processing systems, 36:73969–73982, 2023

    2023

  98. [108]

    Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039, 2024

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039, 2024

    2024 arXiv

  99. [109]

    Long-lrm: Long-sequence large reconstruction model for wide-coverage gaussian splats.arXiv preprint arXiv:2410.12781, 2024

    Chen Ziwen, Hao Tan, Kai Zhang, Sai Bi, Fujun Luan, Yicong Hong, Li Fuxin, and Zexiang Xu. Long-lrm: Long-sequence large reconstruction model for wide-coverage gaussian splats.arXiv preprint arXiv:2410.12781, 2024

    2024 arXiv

  100. [110]

    Zangetsu

    Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.