Pith. sign in

REVIEW 4 major objections 5 minor 63 references

High-fidelity 3D shape generation can match industrial quality with a tiny fraction of the usual training data by aligning a diffusion model to frozen discriminative 3D foundation priors at train time only.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 03:09 UTC pith:VNPG4XQA

load-bearing objection Solid 3D-specific fix for REPA-style alignment (centroid + Hungarian); the 1.5%-data industrial-parity headline is oversold by teacher-overlapping metrics and mismatched baselines. the 4 major comments →

arxiv 2607.28581 v1 pith:VNPG4XQA submitted 2026-07-30 cs.CV

ROAD: Reciprocal-Objective Alignment of Discriminative Semantics for 3D Shape Generation

Xiao Luo , Mingyang Du , Xin Zhou , Tianrui Feng , Xiwu Chen , Xiaofan Li , Jiangning Zhang , Dingkang Liang This is my paper
classification cs.CV
keywords 3D shape generationdiffusion transformersrepresentation alignment3D foundation modelsHungarian matchingdata efficiencySDF VAE
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current high-fidelity 3D generators learn geometry largely from scratch by scaling model size and data, which is expensive. This paper argues that discriminative 3D foundation models already encode rich semantic and structural understanding of shapes, and that those priors can be transferred into a diffusion transformer so the generator no longer has to rediscover the 3D world. The obstacle is that generative and discriminative latents are not token-aligned: one side tends to hold global surface descriptors while the other holds local patches, and point-cloud order is not fixed. ROAD solves this with a reciprocal-objective alignment used only during training: Holistic Semantic Condensing matches global semantic centroids, and Structural Optimal Alignment uses Hungarian bipartite matching on feature similarity to pair microscopic tokens. The foundation model is dropped at inference, so there is no extra run-time cost. On a pruned Step1X-3D-style backbone trained from scratch on 30k public assets, the method reports competitive Uni3D and ULIP scores against much larger industrial systems trained on far more data, and faster convergence.

Core claim

Transferring frozen discriminative 3D foundation-model priors into a shape diffusion transformer via reciprocal Holistic Semantic Condensing plus Structural Optimal Alignment (bipartite matching on latent similarity) lets the generator reach highly competitive fidelity with roughly 1.5 percent of an industrial baseline's training data and far lower training cost, without using the foundation model at inference.

What carries the argument

Reciprocal-objective alignment (ROAD): Holistic Semantic Condensing (global average-pool cosine alignment of generative and foundation centroids) plus Structural Optimal Alignment (adaptive pooling to equal cardinality, cosine cost matrix, Hungarian matching, then matched cosine loss), added only to the training objective on intermediate DiT features.

Load-bearing premise

That scores from the same family of 3D foundation models used as the alignment teacher, measured on a small held-out set, are enough to claim geometric parity with much larger industrial systems trained on different data and full architectures.

What would settle it

Train the identical pruned backbone on the same 30k data with and without ROAD, then compare human preference or independent geometry metrics (e.g., multi-view consistency, surface error to ground-truth meshes) against a full industrial baseline on a larger, non-overlapping test set that does not rely on Uni3D/ULIP-style scores.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • High-quality image-to-3D generation becomes trainable on a single 8-GPU node with tens of thousands of public assets rather than million-scale private corpora.
  • The foundation model is training-only supervision, so deployed generators keep the same inference cost and architecture as the unaligned baseline.
  • The same HSC+SOA recipe transfers to other generators (CraftsMan, TRELLIS-style sparse latents) without redesigning their tokenizers.
  • 2D or 2.5D teachers underperform native 3D teachers, so future efficiency gains should prioritize stronger 3D discriminative instructors.
  • Faster convergence (reported ~2.4× to baseline peak) means fewer iterations and lower energy cost for academic-scale 3D diffusion training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If semantic bipartite matching is the right bridge for unordered 3D tokens, similar set-matching alignment may cut data needs in other permutation-invariant generative settings (point clouds, meshes, molecules).
  • The paper's own layer probe suggests mid-network double-stream blocks are the sweet spot for prior injection; automated layer search could become a standard add-on for representation-aligned 3D DiTs.
  • Because quality is bounded by the teacher's capacity, open releases of stronger public 3D foundation models would immediately lift the ceiling of this training recipe without changing the generator.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. ROAD proposes reducing the data and compute cost of 3D shape diffusion by transferring priors from a frozen discriminative 3D foundation model (Uni3D) into a diffusion transformer at training time only. The core technical claim is that generative and discriminative 3D latents are semantically–structurally heterogeneous (permutation-invariant global anchors vs. local patches), so element-wise REPA-style alignment fails; the authors instead combine Holistic Semantic Condensing (global centroid cosine alignment) with Structural Optimal Alignment (Hungarian bipartite matching on cosine costs after adaptive pooling). On a pruned Step1X-3D* backbone trained from scratch on 30k Objaverse assets, they report Uni3D/ULIP scores competitive with much larger industrial systems, faster convergence, ablations of HSC/SOA and matching metrics, and transfer to CraftsMan and TRELLIS.

Significance. If the efficiency claim holds under cleaner controls, the work would matter: it reframes high-fidelity 3D generation as prior transfer rather than pure scale, and the set-to-set matching formulation is a concrete, reusable response to 3D token disorder that 2D REPA-style methods do not address. Strengths include a clear diagnosis of heterogeneity (Fig. 4), complementary dual objectives with ablations (Tables 4–5), layer/modality/capacity studies (Fig. 5, Tables 6–7, 9), cross-architecture transfer (Table 3), convergence curves (Fig. 6), and promised code. These make the paper a useful contribution to efficient 3D generation even if the industrial-parity headline needs tightening.

major comments (4)
  1. [Table 1, Fig. 2, §5.1–5.3] Abstract/Table 1/Fig. 2 claim ‘highly competitive’ performance vs Step1X-3D/Hunyuan with ~1.5% data. The comparison confounds (i) architecture (0.65B pruned Step1X-3D* with 6+12 blocks vs 1.3–3.3B full models, §5.1), (ii) data mixture (30k public vs mixed private/public up to ~2M), and (iii) rendering (12 vs 20 views, §5.2). Without a same-backbone full-data or full-architecture control, the headline efficiency/parity claim is not isolated. Please add at least: Step1X-3D* trained on the same 30k without ROAD (already partially present) plus Step1X-3D* or full Step1X-3D on a matched larger public subset, and report wall-clock/GPU-hours under identical hardware assumptions.
  2. [§4.3, §5.1–5.2, Table 1] Training aligns DiT layer-6 features to Uni3D-G (§4.3–4.4, §5.1), and Table 1’s primary metric is Uni3D-Score from the same family. Gains can partly reflect teacher-manifold fitting rather than independent geometric fidelity. ULIP helps but is still embedding similarity on a 160-asset set. Please report non-teacher geometry metrics (e.g., Chamfer/F-score/normal consistency on reconstructed meshes vs GT, or multi-view PSNR/LPIPS against held-out renders) and/or an evaluation encoder disjoint from the alignment teacher. Clarify whether Uni3D-Score is computed on generated meshes re-encoded by Uni3D or on internal latents.
  3. [Table 2, §5.3] Table 2 adapts REPA/REG/SRA by forcing coordinate correspondence after FPS. That is a harsh and possibly unfair transfer of 2D methods; degradation may stem from the forced spatial constraint rather than proving set matching is necessary. A fairer baseline is REPA-style cosine alignment after the same adaptive pooling used in SOA, or soft optimal transport without Hungarian hard assignment. Without that, the claim that 2D alignment ‘fails’ in 3D is only partly supported.
  4. [§5.2–5.4] Evaluation uses 160 unseen assets and only Uni3D/ULIP scores (§5.2). For a claim of industrial-tier geometric fidelity this is thin: no category breakdown, no human preference study, limited qualitative diversity controls, and no texture/PBR metrics despite textured results in figures. Expand the test set and add at least one perceptual or geometric reconstruction protocol so the central claim does not rest solely on two embedding scores.
minor comments (5)
  1. [Eq. (14), §4.3] Eq. (14) sets λ1=λ2=0.5 with no sensitivity sweep; a small grid would strengthen robustness claims.
  2. [§4.3 SOA] SOA pools Z to cardinality M to match Y (after Eq. 10); state clearly how M is chosen relative to Uni3D’s patch count and whether unequal-cardinality matching was tried.
  3. [Fig. 2] Fig. 2(c) GPU-hours comparison needs explicit hardware normalization and sources for competitors’ estimates; ‘estimated’ configs weaken the Pareto plot.
  4. [Throughout] Typo/notation: ‘V AE’ spacing throughout; arXiv ID year 2607 looks placeholder-odd; ensure consistent Step1X-3D* asterisk definition early.
  5. [§6] Limitation paragraph correctly notes dependence on teacher capacity; briefly discuss failure modes when teacher and generator tokenizations diverge more extremely (e.g., pure triplane vs point FM).

Circularity Check

1 steps flagged

No derivation circularity; mild teacher–metric channel on Uni3D-Score is evaluation contamination, not a by-construction reduction.

specific steps
  1. other [§4.3 Eqs. (9)–(14); §5.1 (Uni3D-G teacher); Table 1 Uni3D-Score]
    "we employ the pre-trained Uni3D-G as our discriminative foundation model for alignment... L = L_diff + λ1 L_HSC + λ2 L_SOA... we assess the semantic consistency using standard CLIP-Score metrics, reporting results derived from Uni3D-G and ULIP."

    Not a true by-construction reduction: training aligns intermediate DiT tokens of GT latents to Uni3D-G, while Uni3D-Score scores final generated assets. Still, using the same model family as teacher and primary metric makes Uni3D-Score partially non-independent evidence of the alignment claim. ULIP and other results mitigate this; it is evaluation contamination, not definitional circularity.

full rationale

ROAD is an empirical methods paper: it adds training-time HSC (global cosine on pooled tokens) and SOA (Hungarian matching on cosine costs) losses that pull intermediate DiT features toward a frozen Uni3D-G teacher, then reports generation quality on held-out assets. There is no first-principles derivation, uniqueness theorem, self-cited ansatz, or parameter fit that is renamed as a prediction. Equations (9)–(14) define standard auxiliary alignment losses; they are not algebraically identical to the reported Uni3D/ULIP scores, which are computed on decoded generated shapes under image conditioning. The only soft concern is that Uni3D-G is both the alignment teacher (§4.3, §5.1) and the source of the primary Uni3D-Score in Table 1, so gains on that metric are partly non-independent of the training objective—an evaluation-validity issue, not a circular derivation. ULIP-Score, cross-architecture transfers (CraftsMan/TRELLIS), qualitative figures, and the pruned same-family baseline comparison supply independent content. No self-citation is load-bearing for the method. Score 1 reflects only that mild teacher–metric overlap, not tautological reasoning.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central efficiency claim rests on standard diffusion/flow-matching training, the empirical hypothesis that frozen Uni3D patch tokens are useful teachers for SDF-VAE/DiT latents, hand-chosen loss weights and alignment layer, and evaluation protocols that partially overlap the teacher model. No new physical entities; invented pieces are named loss modules and the pruned baseline configuration.

free parameters (5)
  • lambda_1, lambda_2 (HSC/SOA loss weights) = 0.5, 0.5
    Fixed to 0.5 each in Eq. 14 without a reported sweep in the main text; they directly scale how hard the model is pulled toward the teacher.
  • Alignment layer index = layer 6
    Chosen as 6th double-stream block after a ModelNet40 linear-probe sweep on the same backbone (Fig. 5, §4.4); a discrete hyperparameter selected on auxiliary accuracy.
  • Point sampling density for 3DFM input = 10000 points
    Table 8 sweeps 4096/10000/20000 points; 10000 chosen as best—data-selected operating point.
  • Step1X-3D* width/depth prune = 6 double-stream + 12 single-stream blocks (~0.65B)
    Baseline reduced ~50% (12+24 blocks → 6+12) for efficiency (§5.1); all main ROAD numbers are on this altered architecture, not the industrial 1.3B model.
  • Training set size and view count = 30k assets, 12 views
    30k random Objaverse assets and 12 rendered views (vs baseline 20) are experimental choices that define the efficiency headline.
axioms (5)
  • domain assumption Frozen discriminative 3D foundation-model patch tokens encode transferable geometric/semantic priors useful for supervising generative SDF latents.
    Core premise of §1 and §4; supported empirically here but not derived.
  • ad hoc to paper Cosine distance on L2-normalized projected tokens is an adequate matching cost for set-to-set geometric correspondence (Eqs. 11–13).
    Chosen over coordinate L2 after ablation (Table 5); defines SOA.
  • domain assumption Hungarian one-to-one assignment on equal-cardinality pooled tokens is a valid surrogate for aligning permutation-invariant generative tokens to unordered foundation tokens.
    Imported from detection-style set matching (§3.2, §4.3); assumes equal cardinality after adaptive pooling is harmless.
  • domain assumption Flow-matching / MM-DiT latent diffusion on SDF-VAE tokens is a correct generative backbone objective (Eqs. 7–8).
    Inherited from Step1X-3D and related latent 3D generators (§4.1).
  • domain assumption Uni3D-Score and ULIP-Score on 160 held-out assets proxy generation quality and semantic consistency comparable across methods with different training data.
    Evaluation protocol §5.2; standard in the cited 3D generation literature but load-bearing for the parity claim.
invented entities (3)
  • Holistic Semantic Condensing (HSC) no independent evidence
    purpose: Global average-pool both latent sets and minimize cosine distance between centroids to enforce macroscopic semantic alignment.
    Named module Eq. 9–10; standard pooling+cosine, packaged as half of the reciprocal objective.
  • Structural Optimal Alignment (SOA) no independent evidence
    purpose: Adaptive-pool generative tokens to foundation cardinality, Hungarian-match on cosine cost, supervise matched pairs.
    Named module Eq. 11–13; combination of known algorithms applied as training loss.
  • Step1X-3D* pruned baseline no independent evidence
    purpose: Half-depth DiT trained from scratch as the controlled generative student for ROAD experiments.
    Experimental construct §5.1; not a pre-existing public checkpoint with independent benchmarks in the paper.

pith-pipeline@v1.2.0-daily-grok45 · 23187 in / 4308 out tokens · 85767 ms · 2026-07-31T03:09:12.463495+00:00 · methodology

0 comments
read the original abstract

High-fidelity 3D generation predominantly relies on scaling model capacity and data, which incurs prohibitive computational costs. This paradigm typically requires learning geometry from scratch and overlooks the rich semantic and structural priors already encapsulated in discriminative 3D foundation models. We contend that leveraging the profound understanding of the 3D world possessed by these discriminative models can significantly reduce generative cost. To this end, we propose ROAD, a framework that reduces the training cost of 3D generation by transferring these rich discriminative priors into diffusion transformers. To address the inherent semantic-structural heterogeneity between generative and discriminative latents, we introduce a reciprocal-objective alignment strategy. This method synergizes Holistic Semantic Condensing to enforce global semantic coherence and Structural Optimal Alignment, which is formulated as a bipartite matching problem to rigorously align microscopic geometric details between disparate latent spaces. The 3D foundation model is only used for training-time supervision of alignment and is not used at inference, incurring no additional inference cost. Compared with the industrial baseline Step1X-3D, the proposed ROAD achieves highly competitive generation performance with only 1.5% of the training data and significantly reduces training costs, effectively reducing the computational overhead of high-fidelity 3D generation. Code is available at https://github.com/H-EmbodVis/ROAD.

Figures

Figures reproduced from arXiv: 2607.28581 by Dingkang Liang, Jiangning Zhang, Mingyang Du, Tianrui Feng, Xiaofan Li, Xiao Luo, Xin Zhou, Xiwu Chen.

Figure 1
Figure 1. Figure 1: High-fidelity 3D assets generated by ROAD. These results showcase the capability of our method to synthesize diverse 3D [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Overview of our proposed ROAD framework. Instead of relying on resource-intensive, brute-force scaling, we introduce a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed ROAD. (a) During training, the frozen 3D Foundation Model (3DFM) provides semantic priors to [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Analysis of semantic misalignment. (a) A showcase of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Classification result on ModelNet40. Our model signifi [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Convergence Analysis. Our method achieves the base [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: ROAD is comparable to TRELLIS (an academic benchmark) in geometric fidelity and achieves visual quality comparable to [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Our ROAD surpasses TRELLIS (academic models) in detail fidelity and achieves visual parity with Hunyuan3D (industrial [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualizations across diverse asset categories demon [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 8 linked inside Pith

  1. [1]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InProc. of European Conference on Computer Vision, pages 213–229, 2020. 2

    2020

  2. [2]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recog- nition, pages 16123–16133, 2022. 3

    2022

  3. [3]

    Dora: Sampling and benchmarking for 3d shape varia- tional auto-encoders

    Rui Chen, Jianfeng Zhang, Yixun Liang, Guan Luo, Weiyu Li, Jiarui Liu, Xiu Li, Xiaoxiao Long, Jiashi Feng, and Ping Tan. Dora: Sampling and benchmarking for 3d shape varia- tional auto-encoders. InProc. of IEEE Intl. Conf. on Com- puter Vision and Pattern Recognition, pages 16251–16261,

  4. [4]

    Pra-net: Point relation-aware network for 3d point cloud analysis.IEEE Transactions on Image Processing, 30:4436– 4448, 2021

    Silin Cheng, Xiwu Chen, Xinwei He, Zhe Liu, and Xiang Bai. Pra-net: Point relation-aware network for 3d point cloud analysis.IEEE Transactions on Image Processing, 30:4436– 4448, 2021. 4

    2021

  5. [5]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 13142–13153, 2023. 8

    2023

  6. [6]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. InProc. of Intl. Conf. on Machine Learning, 2024. 5

    2024

  7. [7]

    Tango3d: To- wards alignment for global and local 2d-3d correspondence

    Zebin He, Mingxin Yang, Shuhui Yang, Hanxiao Sun, Xin- tong Han, Chunchao Guo, and Wenhan Luo. Tango3d: To- wards alignment for global and local 2d-3d correspondence. arXiv preprint arXiv:2605.19727, 2026. 4

    Pith/arXiv arXiv 2026

  8. [8]

    Efficientdreamer: High-fidelity and robust 3d cre- ation via orthogonal-view diffusion priors

    Zhipeng Hu, Minda Zhao, Chaoyi Zhao, Xinyue Liang, Lincheng Li, Zeng Zhao, Changjie Fan, Xiaowei Zhou, and Xin Yu. Efficientdreamer: High-fidelity and robust 3d cre- ation via orthogonal-view diffusion priors. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 4949–4958, 2024. 3

    2024

  9. [9]

    Hunyuan3d 2.1: From images to high- fidelity 3d assets with production-ready pbr material.arXiv preprint arXiv:2506.15442, 2025

    Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, et al. Hunyuan3d 2.1: From images to high- fidelity 3d assets with production-ready pbr material.arXiv preprint arXiv:2506.15442, 2025. 2, 3, 8

    Pith/arXiv arXiv 2025

  10. [10]

    No other representation component is needed: Diffusion transformers can provide representation guidance by themselves

    Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, and Jingdong Wang. No other representation component is needed: Diffusion transformers can provide representation guidance by themselves. InProc. of Intl. Conf. on Learning Representations, 2026. 4, 8, 9 13

    2026

  11. [11]

    Shap-e: Generat- ing conditional 3d implicit functions.arXiv preprint arXiv:2305.02463, 2023

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions.arXiv preprint arXiv:2305.02463, 2023. 3

    Pith/arXiv arXiv 2023

  12. [12]

    Hunyuan3d 2.5: Towards high- fidelity 3d assets generation with ultimate details.arXiv preprint arXiv:2506.16504, 2025

    Zeqiang Lai, Yunfei Zhao, Haolin Liu, Zibo Zhao, Qingxi- ang Lin, Huiwen Shi, Xianghui Yang, Mingxin Yang, Shuhui Yang, Yifei Feng, et al. Hunyuan3d 2.5: Towards high- fidelity 3d assets generation with ultimate details.arXiv preprint arXiv:2506.16504, 2025. 3

    Pith/arXiv arXiv 2025

  13. [13]

    Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers

    Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. InProc. of IEEE Intl. Conf. on Computer Vision, 2025. 2, 4

    2025

  14. [14]

    Craftsman3d: High-fidelity mesh generation with 3d native diffusion and interactive geometry refiner

    Weiyu Li, Jiarui Liu, Hongyu Yan, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. Craftsman3d: High-fidelity mesh generation with 3d native diffusion and interactive geometry refiner. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 5307– 5317, 2025. 9

    2025

  15. [15]

    Step1x-3d: Towards high-fidelity and con- trollable generation of textured 3d assets.arXiv preprint arXiv:2505.07747, 2025

    Weiyu Li, Xuanyang Zhang, Zheng Sun, Di Qi, Hao Li, Wei Cheng, Weiwei Cai, Shihao Wu, Jiarui Liu, Zihao Wang, et al. Step1x-3d: Towards high-fidelity and con- trollable generation of textured 3d assets.arXiv preprint arXiv:2505.07747, 2025. 2, 3, 5, 8

    Pith/arXiv arXiv 2025

  16. [16]

    Hsgan: Hierarchical graph learning for point cloud generation.IEEE Transactions on Image Processing, 30:4540–4554, 2021

    Yushi Li and George Baciu. Hsgan: Hierarchical graph learning for point cloud generation.IEEE Transactions on Image Processing, 30:4540–4554, 2021. 3, 7

    2021

  17. [17]

    Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models.IEEE Transactions on Pattern Analysis and Machine Intelligence,

    Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models.IEEE Transactions on Pattern Analysis and Machine Intelligence,

  18. [18]

    Pointmamba: A simple state space model for point cloud analysis

    Dingkang Liang, Xin Zhou, Wei Xu, Xingkui Zhu, Zhikang Zou, Xiaoqing Ye, Xiao Tan, and Xiang Bai. Pointmamba: A simple state space model for point cloud analysis. InProc. of Advances in Neural Information Processing Systems, pages 32653–32677, 2024. 4

    2024

  19. [19]

    Parameter-efficient fine-tuning in spectral domain for point cloud learning.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 2025

    Dingkang Liang, Tianrui Feng, Xin Zhou, Yumeng Zhang, Zhikang Zou, and Xiang Bai. Parameter-efficient fine-tuning in spectral domain for point cloud learning.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 2025. 2

    2025

  20. [20]

    Meta architecture for point cloud analysis

    Haojia Lin, Xiawu Zheng, Lijiang Li, Fei Chao, Shanshan Wang, Yan Wang, Yonghong Tian, and Rongrong Ji. Meta architecture for point cloud analysis. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 17682–17691, 2023. 4

    2023

  21. [21]

    Pufa-gan: A frequency-aware generative adversarial network for 3d point cloud upsampling.IEEE Transactions on Image Processing, 31:7389–7402, 2022

    Hao Liu, Hui Yuan, Junhui Hou, Raouf Hamzaoui, and Wei Gao. Pufa-gan: A frequency-aware generative adversarial network for 3d point cloud upsampling.IEEE Transactions on Image Processing, 31:7389–7402, 2022. 3, 7

    2022

  22. [22]

    Openshape: Scaling up 3d shape representation towards open-world understanding

    Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xu- anlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding. InProc. of Advances in Neural Information Processing Systems, pages 44860–44879, 2023. 2

    2023

  23. [23]

    Point-e: A system for generat- ing 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751, 2022

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751, 2022. 3

    Pith/arXiv arXiv 2022

  24. [24]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2023. 12

    2023

  25. [25]

    Masked autoencoders for point cloud self-supervised learning

    Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud self-supervised learning. InEuropean Conference on Computer Vision, pages 604–621, 2022. 4

    2022

  26. [26]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. InProc. of Intl. Conf. on Learning Representations, 2023. 3

    2023

  27. [27]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 652–660, 2017. 4

    2017

  28. [28]

    Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining

    Zekun Qi, Runpei Dong, Guofan Fan, Zheng Ge, Xiangyu Zhang, Kaisheng Ma, and Li Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. InProc. of Intl. Conf. on Machine Learning, pages 28223–28243, 2023. 4

    2023

  29. [29]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. InProc. of European Conference on Computer Vision, pages 214–238, 2024. 4

    2024

  30. [30]

    Pointnext: Revisiting pointnet++ with improved training and scaling strategies.Proc

    Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies.Proc. of Advances in Neural Information Processing Systems, 35:23192–23204, 2022. 4

    2022

  31. [31]

    Global-local bidi- rectional reasoning for unsupervised representation learning of 3d point clouds

    Yongming Rao, Jiwen Lu, and Jie Zhou. Global-local bidi- rectional reasoning for unsupervised representation learning of 3d point clouds. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 5376–5385, 2020. 4

    2020

  32. [32]

    Mvdream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. InProc. of Intl. Conf. on Learning Representations,

  33. [33]

    3d neural field generation using triplane diffusion

    J Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Wetzstein. 3d neural field generation using triplane diffusion. InProc. of IEEE Intl. Conf. on Com- puter Vision and Pattern Recognition, pages 20875–20886,

  34. [34]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 19615– 19625, 2024. 3

    2024

  35. [35]

    What mat- ters for representation alignment: Global information or spa- tial structure? InProc

    Jaskirat Singh, Xingjian Leng, Zongze Wu, Liang Zheng, Richard Zhang, Eli Shechtman, and Saining Xie. What mat- ters for representation alignment: Global information or spa- tial structure? InProc. of Intl. Conf. on Learning Represen- tations, 2026. 4 14

    2026

  36. [36]

    X-3d: Explicit 3d structure modeling for point cloud recog- nition

    Shuofeng Sun, Yongming Rao, Jiwen Lu, and Haibin Yan. X-3d: Explicit 3d structure modeling for point cloud recog- nition. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 5074–5083, 2024. 4

    2024

  37. [37]

    U-repa: Aligning diffu- sion u-nets to vits.arXiv preprint arXiv:2503.18414, 2025

    Yuchuan Tian, Hanting Chen, Mengyu Zheng, Yuchen Liang, Chao Xu, and Yunhe Wang. U-repa: Aligning diffu- sion u-nets to vits.arXiv preprint arXiv:2503.18414, 2025. 4

    arXiv 2025

  38. [38]

    Triposr: Fast 3d object reconstruction from a single image.arXiv preprint arXiv:2403.02151, 2024

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image.arXiv preprint arXiv:2403.02151, 2024. 3, 8

    Pith/arXiv arXiv 2024

  39. [39]

    Lion: Latent point dif- fusion models for 3d shape generation

    Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point dif- fusion models for 3d shape generation. InProc. of Advances in Neural Information Processing Systems, 2022. 3

    2022

  40. [40]

    Cad: Photorealistic 3d generation via adversarial distillation

    Ziyu Wan, Despoina Paschalidou, Ian Huang, Hongyu Liu, Bokui Shen, Xiaoyu Xiang, Jing Liao, and Leonidas Guibas. Cad: Photorealistic 3d generation via adversarial distillation. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 10194–10207, 2024. 3

    2024

  41. [41]

    Gpsformer: A global perception and local struc- ture fitting-based transformer for point cloud understanding

    Changshuo Wang, Meiqing Wu, Siew-Kei Lam, Xin Ning, Shangshu Yu, Ruiping Wang, Weijun Li, and Thambipillai Srikanthan. Gpsformer: A global perception and local struc- ture fitting-based transformer for point cloud understanding. InProc. of European Conference on Computer Vision, pages 75–92. Springer, 2024. 4

    2024

  42. [42]

    Rethinking masked representation learning for 3d point cloud understanding.IEEE Transactions on Im- age Processing, 34:247–262, 2024

    Chuxin Wang, Yixin Zha, Jianfeng He, Wenfei Yang, and Tianzhu Zhang. Rethinking masked representation learning for 3d point cloud understanding.IEEE Transactions on Im- age Processing, 34:247–262, 2024. 4

    2024

  43. [43]

    Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recogni- tion, pages 12619–12629, 2023. 3

    2023

  44. [44]

    Vggt: Vi- sual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Vi- sual geometry grounded transformer. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 5294–5306, 2025. 12

    2025

  45. [45]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. InProc. of Advances in Neural Information Processing Systems, 2023. 3

    2023

  46. [46]

    Representation entanglement for genera- tion: Training diffusion transformers is much easier than you think

    Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, et al. Representation entanglement for genera- tion: Training diffusion transformers is much easier than you think. InProc. of Advances in Neural Information Process- ing Systems, 2025. 4, 8, 9

    2025

  47. [47]

    Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling.arXiv preprint arXiv:2507.07982, 2025

    Haoyu Wu, Diankun Wu, Tianyu He, Junliang Guo, Yang Ye, Yueqi Duan, and Jiang Bian. Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling.arXiv preprint arXiv:2507.07982, 2025. 4

    Pith/arXiv arXiv 2025

  48. [48]

    Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention

    Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Xun Cao, Philip Torr, et al. Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention. InProc. of Advances in Neural Information Processing Systems, 2025. 8

    2025

  49. [49]

    Point transformer v2: Grouped vector at- tention and partition-based pooling

    Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Heng- shuang Zhao. Point transformer v2: Grouped vector at- tention and partition-based pooling. InProc. of Advances in Neural Information Processing Systems, pages 33330– 33342, 2022. 4

    2022

  50. [50]

    Point transformer v3: Simpler faster stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xi- hui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recog- nition, pages 4840–4851, 2024. 4

    2024

  51. [51]

    Structured 3d latents for scalable and versatile 3d gen- eration

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d gen- eration. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 21469–21480, 2025. 2, 3, 8, 9

    2025

  52. [52]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 1179–1189, 2023. 2, 8

    2023

  53. [53]

    Ulip-2: Towards scal- able multimodal pre-training for 3d understanding

    Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Jun- nan Li, Roberto Mart´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: Towards scal- able multimodal pre-training for 3d understanding. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recog- nition, pages 27091–27101, 2024. 2

    2024

  54. [54]

    Single- view 3d mesh reconstruction for seen and unseen categories

    Xianghui Yang, Guosheng Lin, and Luping Zhou. Single- view 3d mesh reconstruction for seen and unseen categories. IEEE Transactions on Image Processing, 32:3746–3758,

  55. [55]

    Hi3dgen: High-fidelity 3d geometry generation from images via normal bridging

    Chongjie Ye, Yushuang Wu, Ziteng Lu, Jiahao Chang, Xi- aoyang Guo, Jiaqing Zhou, Hao Zhao, and Xiaoguang Han. Hi3dgen: High-fidelity 3d geometry generation from images via normal bridging. InProc. of IEEE Intl. Conf. on Com- puter Vision, 2025. 8

    2025

  56. [56]

    Homugan: A 3d-aware gan with the method of cylindrical spatial-constrained sampling.IEEE Transactions on Image Processing, 34:320–334, 2024

    Haochen Yu, Weixi Gong, Jiansheng Chen, and Huimin Ma. Homugan: A 3d-aware gan with the method of cylindrical spatial-constrained sampling.IEEE Transactions on Image Processing, 34:320–334, 2024. 3

    2024

  57. [57]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. InProc. of Intl. Conf. on Learning Representations, 2025. 4, 8, 9

    2025

  58. [58]

    Pointclip: Point cloud understanding by clip

    Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xu- peng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. InProc. of IEEE Intl. Conf. on Computer Vision and Pattern Recogni- tion, pages 8552–8562, 2022. 4

    2022

  59. [59]

    Hypergraph spectral analysis and processing in 3d point cloud.IEEE Transactions on Image Processing, 30:1193–1206, 2020

    Songyang Zhang, Shuguang Cui, and Zhi Ding. Hypergraph spectral analysis and processing in 3d point cloud.IEEE Transactions on Image Processing, 30:1193–1206, 2020. 4 15

    2020

  60. [60]

    Vide- orepa: Learning physics for video generation through rela- tional alignment with foundation models

    Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Vide- orepa: Learning physics for video generation through rela- tional alignment with foundation models. InProc. of Ad- vances in Neural Information Processing Systems, 2025. 2, 4

    2025

  61. [61]

    Point transformer

    Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. InProc. of IEEE Intl. Conf. on Computer Vision, pages 16259–16268, 2021. 4

    2021

  62. [62]

    Uni3d: Exploring unified 3d representation at scale

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. InProc. of Intl. Conf. on Learning Representations, pages 46766–46782, 2024. 2, 4, 8

    2024

  63. [63]

    Recur- rent diffusion for 3d point cloud generation from a single image.IEEE Transactions on Image Processing, 2025

    Yan Zhou, Dewang Ye, Huaidong Zhang, Xuemiao Xu, Hua- jie Sun, Yewen Xu, Xiangyu Liu, and Yuexia Zhou. Recur- rent diffusion for 3d point cloud generation from a single image.IEEE Transactions on Image Processing, 2025. 3 16

    2025