Pith. sign in

REVIEW 9 cited by

G3PT: Unleash the power of Autoregressive Modeling in 3D Generation via Cross-scale Querying Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.06322 v1 pith:UZP5NOKI submitted 2024-09-10 cs.CV

Jinzhi Zhang , Feng Xiong , Mu Xu This is my paper · ORCID
classification cs.CV
keywords generationg3ptautoregressivecross-scaledatadifferentlevelsquerying
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Autoregressive transformers have revolutionized generative models in language processing and shown substantial promise in image and video generation. However, these models face significant challenges when extended to 3D generation tasks due to their reliance on next-token prediction to learn token sequences, which is incompatible with the unordered nature of 3D data. Instead of imposing an artificial order on 3D data, in this paper, we introduce G3PT, a scalable coarse-to-fine 3D generative model utilizing a cross-scale querying transformer. The key is to map point-based 3D data into discrete tokens with different levels of detail, naturally establishing a sequential relationship between different levels suitable for autoregressive modeling. Additionally, the cross-scale querying transformer connects tokens globally across different levels of detail without requiring an ordered sequence. Benefiting from this approach, G3PT features a versatile 3D generation pipeline that effortlessly supports diverse conditional structures, enabling the generation of 3D shapes from various types of conditions. Extensive experiments demonstrate that G3PT achieves superior generation quality and generalization ability compared to previous 3D generation methods. Most importantly, for the first time in 3D generation, scaling up G3PT reveals distinct power-law scaling behaviors.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A causal transformer with 3D RoPE generates vector-quantized 3D Gaussian latent grids autoregressively, enabling unconditional synthesis, completion, and open-ended outpainting of indoor scenes.

  2. VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    VesselTok learns compact continuous tokens of large tubular biomedical graphs from centerline points plus a fixed pseudo-radius, enabling reconstruction, generation, and link prediction across anatomies.

  3. Squeeze3D: Your 3D Generation Model is Secretly an Extreme Neural Compressor

    cs.GR 2025-06 conditional novelty 6.0 of 10

    Two small mapping networks connect a frozen 3D encoder to a frozen 3D generator, so the generator decompresses objects from latent codes as small as 3 KB, achieving up to 2187x compression on meshes.

  4. CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Prediction

    cs.RO 2024-12 conditional novelty 6.0 of 10

    A coarse-to-fine autoregressive policy with multi-scale action tokenization matches or beats diffusion policies on robot manipulation benchmarks at roughly 10x lower inference cost.

  5. HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset

    cs.CV 2024-12 conditional novelty 6.0 of 10

    HumanRig provides a large uniform-topology humanoid rigging dataset and an automatic rigging method combining a 2D pose prior with a point transformer and cross-attention, outperforming GNN-based baselines in the repo...

  6. 3D representation in 512-Byte:Variational tokenizer is the key for autoregressive 3D generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    VAT is a variational tokenizer that maps 3D shape features into coarse-to-fine tokens in Gaussian subspaces, enabling autoregressive 3D generation and up to 2000x mesh compression.

  7. Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient

    cs.CV 2024-11 conditional novelty 6.0 of 10

    CoDe speeds up Visual Auto-Regressive image generation by using a 2B model for early coarse scales and a 0.3B model for later fine scales, with 1.7x-2.9x speedup and only a small FID increase.

  8. Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

    cs.LG 2025-05 conditional novelty 5.0 of 10

    ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.

  9. MVPainter: Accurate and Detailed 3D Texture Generation via Multi-View Diffusion with Geometric Control

    cs.CV 2025-05 conditional novelty 5.0 of 10

    MVPainter paints detailed, reference-consistent textures on 3D meshes by conditioning multi-view diffusion on normal and depth maps, reporting the best preference scores among tested open-source texture generators.

Pith tools