REVIEW 27 cited by
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
read the original abstract
This work explores expanding the capabilities of large language models (LLMs) pretrained on text to generate 3D meshes within a unified model. This offers key advantages of (1) leveraging spatial knowledge already embedded in LLMs, derived from textual sources like 3D tutorials, and (2) enabling conversational 3D generation and mesh understanding. A primary challenge is effectively tokenizing 3D mesh data into discrete tokens that LLMs can process seamlessly. To address this, we introduce LLaMA-Mesh, a novel approach that represents the vertex coordinates and face definitions of 3D meshes as plain text, allowing direct integration with LLMs without expanding the vocabulary. We construct a supervised fine-tuning (SFT) dataset enabling pretrained LLMs to (1) generate 3D meshes from text prompts, (2) produce interleaved text and 3D mesh outputs as required, and (3) understand and interpret 3D meshes. Our work is the first to demonstrate that LLMs can be fine-tuned to acquire complex spatial knowledge for 3D mesh generation in a text-based format, effectively unifying the 3D and text modalities. LLaMA-Mesh achieves mesh generation quality on par with models trained from scratch while maintaining strong text generation performance.
Forward citations
Cited by 27 Pith papers
-
One Video, One World: Turning Monocular Video into Physical 4D Scenes
OVOW reconstructs instance-level, simulation-ready 4D mesh scenes from monocular video via a four-stage training-free pipeline and introduces a new benchmark for structured Video-to-4D evaluation.
-
Engine-Native Editable 3D World Reconstruction with Objects and Lighting
A UE5-derived dataset (Lumera-2K) and VLM pipeline parse single-image scenes into oriented object boxes and parametric light tuples, establishing a measurable benchmark for editable, light-aware 3D reconstruction.
-
Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN
Fine-tunes EG3D using a human-preference reward on NeRF density to improve face geometry, achieving 74.4% user preference in pairwise tests with FID rising from 4.09 to 6.66.
-
QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
QuadLink generates anisotropic quad-dominant meshes from point clouds via anchor prediction, centroid-conditioned linking, and quad-first assembly, supporting hybrid n-gon topology.
-
LottieGPT: Tokenizing Vector Animation for Autoregressive Generation
LottieGPT tokenizes Lottie animations into compact sequences and fine-tunes Qwen-VL to autoregressively generate coherent vector animations from natural language or visual prompts, outperforming prior SVG models.
-
MeshTailor: Cutting Seams via Generative Mesh Traversal
MeshTailor is a mesh-native generative model that uses ChainingSeams serialization and a dual-stream transformer with pointer layers to trace coherent seams vertex-by-vertex on 3D surfaces.
-
PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusion
PartDiffuser is a semi-autoregressive discrete diffusion framework that generates high-fidelity 3D meshes from point clouds by combining inter-part autoregression with intra-part parallel diffusion using a part-aware ...
-
GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement
GReFEM shows MLLMs zero-shot isolate load-activated geometric features for volumetric mesh refinement with higher precision than matched-budget geometric heuristics.
-
Nexus: Native Mesh Generation with Diffusion
Nexus replaces autoregressive mesh serialization with two coupled diffusion models — octree vertex generation and a latent topology generator — claiming stronger geometry and perceptual quality on Objaverse and Toys4K.
-
LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flow
Factorizing mesh generation into a vertex flow then a vertex-conditioned topology flow improves fidelity and enables part-wise high-res synthesis and topology-adaptive editing.
-
ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
ELSA3D introduces elastic semantic anchoring via sparse anchor tokens and a scale-aware octree tokenizer to unify 3D generation and captioning at reduced computational cost.
-
Variance Reduction for Expectations with Diffusion Teachers
CARV amortizes upstream diffusion teacher costs over noise resamples with timestep importance sampling and stratified-inverse-CDF sampling, delivering 2-3x effective compute gains in text-to-3D experiments and order-o...
-
PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects
PhysX-Omni unifies simulation-ready 3D asset generation across rigid, deformable, and articulated objects via a new geometry representation, the PhysXVerse dataset, and the PhysX-Bench evaluation suite.
-
QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
QuadLink generates anisotropic quad-dominant meshes from point clouds via a hybrid centroid-conditioned vertex linking model and a Tri-to-Quad data conversion operator.
-
TOPOS: High-Fidelity and Efficient Industry-Grade 3D Head Generation
TOPOS creates high-fidelity 3D heads with fixed industry topology from single images via a specialized VAE with Perceiver Resampler and a rectified flow transformer.
-
Beyond Spatial Compression: Interface-Centric Generative States for Open-World 3D Structure
C2LT-3D factorizes 3D tokenization into canonical local geometry, partition-conditioned context, and relational seam variables to make latent states operational for assembly-level validation and repair in open-world m...
-
UniRecGen: Unifying Multi-View 3D Reconstruction and Generation
UniRecGen unifies reconstruction and generation via shared canonical space and disentangled cooperative learning to produce complete, consistent 3D models from sparse views.
-
Twisted Fiber Bundle Codes over Group Algebras
Singular chain-compatible fiber twists over group algebras can increase CSS encoded dimension k at fixed blocklength n while examples keep distance d unchanged.
-
HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesis
HVG-3D uses a 3D-aware diffusion architecture with ControlNet to synthesize high-fidelity hand-object interaction videos from 3D control signals, achieving state-of-the-art spatial fidelity and temporal coherence on t...
-
LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flow
LATO.2 factorizes mesh generation into a vertex-generation flow and a vertex-conditioned connectivity flow, beating joint-latent and autoregressive baselines on geometric fidelity and connectivity quality.
-
Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image
Sat2City v2 adapts a pretrained native 3D latent model to generate controllable textured 3D city assets from satellite images via geometry flow fine-tuning and anchored texturing on a collected real dataset.
-
Variance Reduction for Expectations with Diffusion Teachers
CARV introduces a hierarchical Monte Carlo estimator with amortized reuse, importance sampling, and stratification that yields 2-3x effective compute gains on diffusion-teacher pipelines while cutting gradient varianc...
-
EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
EVA01 introduces a Mixture-of-Transformers model that natively adds 3D mesh understanding, generation, and multi-turn editing to MLLMs by decoupling understanding and generation experts with shared global self-attention.
-
SynVA: A Modular Toolkit for Vessel Generation and Aneurysm Editing
SynVA toolkit generates realistic vascular meshes and anatomically plausible aneurysms, releasing 50,000 labeled samples for medical vision tasks.
-
CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models
CG-MLLM is a multimodal LLM using a Mixture-of-Transformer architecture with separate TokenAR and BlockAR components integrated with a pre-trained vision-language backbone and 3D VAE to enable 3D captioning and high-f...
-
PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusion
A part-wise semi-autoregressive discrete diffusion model for point-cloud-to-mesh generation that separates global structure from local detail, beating prior SOTA on Objaverse.
-
QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
QuadLink generates anisotropic quad-dominant meshes from point clouds via autoregressive anchor prediction and centroid-conditioned linking, with a Tri-to-Quad data converter and quad-first assembly.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.