Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Taming Feed-forward Reconstruction Models as Latent Encoders for 3D Generative Models

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Recycled reconstruction latents beat dedicated 3D VAEs

desk verdict A credible recipe for training text-to-3D directly on frozen reconstruction latents, but the whitening step is never ablated and nothing is released. read the letter →

arxiv 2501.00651 v2 pith:LPCWDWXI submitted 2024-12-31 cs.CV cs.LG

Suttisak Wizadwongsa , Jinfan Zhou , Edward Li , Jeong Joon Park This is my paper · ORCID
classification cs.CVcs.LG
keywords text-to-3Dgenerationlatentdiffusionrectifiedflowtriplanefeed-forwardreconstructionInstantMeshtransformer3Dgenerativemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that off-the-shelf feed-forward reconstruction models can be repurposed as latent encoders for 3D generative modeling, eliminating the need to train dedicated 3D VAEs. The authors use InstantMesh's frozen triplane features as the latent space and show that, after a global whitening standardization, empty-space masking, and a perceptual rendering loss, a rectified-flow transformer can be trained directly on those features. They report state-of-the-art text-to-3D results, with CLIP scores of 27.61 (ViT-B) and 22.21 (ViT-L) on Objaverse, and ShapeNet FIDs between 10.95 and 15.65, beating the dedicated-VAE baseline LN3Diff. If correct, the computational bottleneck of training dataset-specific 3D encoders disappears, and advances in reconstruction models transfer for free into generation.

What carries the argument

The load-bearing object is the triplane latent produced by InstantMesh: three 64x64 feature maps with 80 channels that a small MLP decodes into density and color. To make this space learnable, the paper applies a whitening transform $Z = ED^{-1/2}E^T(X-\mu)$ computed from the dataset mean and covariance, and computes the flow loss back in the original space $X = ED^{1/2}E^T Z + \mu$. A binary empty-space mask with weights 1 and 0.25 concentrates capacity on object regions, and an LPIPS rendering loss on a one-step denoised estimate $\hat{Z}_0 = Z_t - t v_\theta(Z_t,t)$ aligns latent error with perceptual image error. The multi-stream transformer keeps self-attention on a small latent stream, with triplane tokens communicating through cross-attention fuse blocks, so cost scales linearly with triplane size.

What would settle it

Train the identical TriFlow pipeline on the same data but with the empty-space mask removed, or with per-sample rather than dataset-wide standardization, and compare ShapeNet FID against the reported numbers; if the gap over LN3Diff collapses, the claimed benefit comes from the preprocessing recipe rather than from the reconstruction latent itself.

Watch

Extended reading notes

Core claim

The central claim is that the unstructured latent space of a pretrained reconstruction model can be tamed into a generative training space. Concretely, the paper shows that standardizing triplane pixels to zero mean and unit variance via dataset covariance eigendecomposition, weighting the flow loss with a binary mask that down-weights empty space (1 for object pixels, 0.25 for empty), and adding an LPIPS loss on patches rendered from a single-step denoised triplane allow a rectified-flow transformer to learn the distribution. The model, TriFlow, uses a multi-stream transformer with MM-DiT and DiT blocks and cross-attention fuse blocks that keep self-attention on a small latent stream, giving linear scaling in triplane token count. With this recipe, the paper reports superior text-conditioned generation on Objaverse and ShapeNet compared with methods that train their own VAEs.

Load-bearing premise

The whole approach rests on the empirical premise that the frozen reconstruction latent, once whitened and masked, is regular enough for a rectified-flow model to learn directly.

Editorial extensions

If this is right

  • Training text-to-3D models no longer requires a dedicated, dataset-specific VAE; any strong feed-forward reconstruction model with a triplane representation can be reused.
  • Improvements in reconstruction quality directly translate into better generative latents, so generation improves as reconstruction models improve without retraining the generative pipeline's encoder.
  • The staged training (flow loss first, then perceptual rendering loss) offers a practical recipe for high-dimensional latent spaces where channel-wise importance varies.
  • The multi-stream transformer design makes training on 64x64 triplanes feasible on a small GPU cluster, lowering the compute barrier for 3D generation research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same standardization-plus-mask-plus-perceptual-loss recipe could be ported to other reconstruction latents, such as Gaussian splat parameters or LRM-style features, since nothing in the method is specific to InstantMesh's exact decoder.
  • Because generation quality inherits the reconstruction encoder's blind spots, the reported results likely depend on dataset characteristics; textureless or background-similar objects degrade, as the paper itself notes for ShapeNet.
  • A testable extension is to apply the pipeline to a much larger corpus or to natural images, which would show whether the standardized reconstruction latent remains learnable at scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TriFlow, a text-to-3D generator trained directly on triplane latents produced by a frozen feed-forward reconstruction model (InstantMesh). The authors argue that reconstructing triplanes are not immediately usable for diffusion/flow training and introduce three post-processing ingredients: a dataset-level standardization of the latent features (Eq. 1), a spatial weighting mask that down-weights empty triplane regions, and a two-stage training scheme that adds a 2D perceptual rendering loss (LPIPS) on single-step denoised estimates. The generative model is a rectified-flow transformer with a multi-stream architecture designed for linear scaling in the number of triplane tokens. Experiments on Objaverse LVIS and ShapeNet report state-of-the-art text-to-3D alignment (Table 2 CLIP scores up to 27.61 ViT-B and 22.21 ViT-L vs. LN3Diff at 25.75 and 20.29) and large FID/CMMD improvements over LN3Diff on ShapeNet (Tables 3 and 5).

Significance. If the central claim holds, the paper would make a practical contribution: it would show that expensive training of dedicated 3D VAEs can be replaced by reusing off-the-shelf feed-forward reconstruction models as latent encoders, thereby decoupling reconstruction quality from generative training cost. The paper has clear strengths: the design choices for masking and perceptual loss are directly ablated in Table 1, the architecture is described in detail, the comparisons cover both unconditional and text-conditioned generation, and the supplementary includes qualitative stress tests (e.g., colored toilets) that support the prompt-following claims. The main weakness is that the load-bearing 'taming' claim rests on one unablated component: the standardization transform of Eq. 1 is applied in every configuration. In addition, the headline comparisons in Table 2 use baselines trained on somewhat different data, and the purported generality across feed-forward reconstruction models is only tested with InstantMesh. The result is promising and the central direction is sound, but the evidence is one controlled experiment short of supporting the paper's strongest claims.

major comments (4)
  1. [Sec. 3.3, Eq. (1); Table 1] The standardization transform is never ablated. Table 1 varies the architecture, the mask, the initialization, and the training stages, but every configuration uses Eq. (1), so the paper never tests whether raw InstantMesh triplanes are actually unusable for flow training. This matters because Eq. (1) estimates the covariance over all pixels, including the 70–85% empty-space pixels that are later down-weighted by the mask; a covariance dominated by empty-space noise could distort object features, and the whitening could be either essential or harmful. Please add a control that trains the same model on unstandardized triplanes (with and without the mask and LPIPS loss) and report FID, CMMD, precision/recall, and CLIP scores for those configurations. Without this control, the central premise that reconstruction latents require 'taming' is not directly supported.
  2. [Sec. 4.2, Table 2] The headline text-to-3D comparison is confounded by training-data differences. The paper states that LN3Diff and Direct3D were trained on 'a portion of the Objaverse dataset, though with slightly different text captions,' while TriFlow used its own filtered 80k-object subset and captions from 3DTopia and Cap3D. The reported CLIP-score gains could therefore reflect dataset and caption differences rather than method superiority. The paper should either use a shared evaluation set that is controlled for object and caption distribution, or report the comparison in a way that quantifies this confound. At minimum, report standard deviations or confidence intervals over multiple seeds/sampling runs; the current tables report single point estimates for all metrics.
  3. [Sec. 3.2 and Sec. 4] The paper claims the proposed pipeline is general: 'our proposed processing pipelines and training insights are general and not restricted to a particular feed-forward model, and should apply to other latent spaces.' Yet all experiments use InstantMesh as the only encoder. Since the statistical properties of triplane latents may vary significantly across reconstruction models (e.g., LRM, TripoSR, SF3D), the general claim is not supported by the evidence. Please either test at least one additional off-the-shelf reconstruction encoder or explicitly reframe the contribution as specific to InstantMesh triplanes. This is a load-bearing part of the paper's stated significance as a bridge between reconstruction and generation.
  4. [Sec. 3.4, Sec. 3.5, and supplementary Sec. B.3] Several implementation details needed for reproduction are missing. The empty-space mask is said to be obtained by evaluating densities and 'identify and mask regions that are empty versus those containing object details,' but no density threshold or sampling procedure is given. The LPIPS loss in Eq. (4) is introduced without a loss weight, and the two-stage schedule gives no protocol for how the weight is set during Stage II. Since these are all free parameters in the method, please specify them precisely (threshold, number of sampled points, number of mask samples, LPIPS weight, and stage transition schedule).
minor comments (5)
  1. [Sec. 4.3] The heading 'Unconditional tripalne generation' contains a typo: 'tripalne' should be 'triplane.' The same typo appears in the supplementary Table 6 heading ('Uncondtional').
  2. [Sec. 3.3] The phrase 'V AE with KL-reguliarzation' and 'we found that the V AE option' contain inconsistent spacing and a typo. More substantively, the claim that the VAE option incurs 'significant computational overheads' is stated without quantification; a sentence on measured overhead would help.
  3. [Sec. 4.1, Table 1] The caption says the full model 'leads to the best outcome,' but row (g), the InstantMesh reconstruction, is used as a reference, not a generative model. Please make this distinction explicit in the caption or in the main text to avoid readers comparing rows (a–f) with row (g) as if they were directly comparable generative methods.
  4. [Sec. 4.2] The description of Direct3D checkpoints is clear, but the paper does not state how many samples per prompt were used for each model in Table 2 or whether the six rendered views per prompt come from the same camera distribution for all methods. Please add these evaluation details.
  5. [References] Several references are informal web resources (e.g., [4] and [12]). This is acceptable for model descriptions, but a short note on which version/commit of Flux and AuraFlow was used would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning found; the generative model is trained on externally provided triplane latents and evaluated against external baselines.

full rationale

The paper's central contribution is empirical: it trains a rectified-flow transformer on triplanes produced by the pretrained InstantMesh reconstruction model, after an affine standardization (Eq. 1), empty-space masking, and an LPIPS rendering loss. None of these components is defined in terms of the method's own outputs. Eq. 1 is an invertible affine transform computed from dataset statistics, and the loss is explicitly computed in the original feature space after reverting the transform via Eq. 2, so the standardization cannot encode the generative result. The mask is constructed from density evaluations of the frozen renderer, not from the generated triplanes, and is ablated in Table 1 (rows b vs c). The LPIPS loss uses a one-step clean-triplane estimate rendered and compared with ground-truth images, which is an auxiliary training objective rather than a fitted quantity relabeled as a prediction. All headline numbers (Tables 2, 3, 5) compare against external methods LN3Diff and Direct3D, with the dataset, prompts, and rendered views held fixed, so the reported improvements are not forced by construction. The whitening step is not separately ablated, which is a legitimate experimental gap for a correctness review, but a missing control is not circularity under the standards used here. Self-citations are limited to architecture inspirations (AuraFlow, PointInfinity, Flux), none of which supplies a load-bearing premise unique to this paper's claim. The supplementary's admission that textureless ShapeNet objects degrade reconstruction quality is an honest external limitation, not a circular step. The paper is self-contained as an empirical demonstration and contains no step where an output is equivalent to an input by definition.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central result rests on the frozen InstantMesh encoder and on several empirical assumptions about its latent space. The paper reports ablations for the mask and the perceptual loss but not for standardization, the density threshold, or the LPIPS weight, so several load-bearing choices remain unquantified.

free parameters (8)
  • Empty-space mask weighting = 1.0 for object pixels, 0.25 for empty pixels
    Chosen by hand in Sec. 3.4; it controls how much model capacity is spent on empty triplane regions. The ablation in Table 1 shows that removing the mask changes FID by 4.72, so this value materially affects results.
  • Density threshold for empty-space mask = not reported
    Sec. 3.4 says regions are identified by density but does not state the cutoff. The mask is load-bearing for training, so this omitted threshold is a tuned free parameter.
  • LPIPS loss weight = not reported
    Stage II optimizes LRF plus Llpips in Sec. 3.5, but the relative weighting is never stated. Table 1 shows the largest FID jump comes from adding this loss, so the weighting matters.
  • Triplane patch size = 2x2
    Architecture choice in Sec. B.1 for tokenizing triplane data; it determines token count, positional embedding granularity, and effective receptive field.
  • Transformer hidden width = 768
    Architecture choice in Sec. B.1; it sets the capacity of the triplane, latent, and text streams and the compute cost of training.
  • Center crop resolution = 48x48 from 64x64
    Supplementary Sec. B.3 reports cropping to remove empty space for ShapeNet training; this reduces token count and focuses computation on object regions.
  • Sampling steps = 30
    Sec. 4.1 uses 30 steps for all ablations; the number of ODE steps directly trades generation quality for inference time.
  • Guidance scale = 4
    Sec. 4.1 uses a guidance scale of 4 for sampling; this hyperparameter balances text alignment against sample diversity.
assumptions (5)
  • domain assumption InstantMesh provides sufficiently accurate and informative triplane latents for a diverse set of objects.
    Sec. 3.2 selects InstantMesh as the fixed encoder and builds the entire training dataset on its outputs. The paper acknowledges in Sec. 4.3 and Fig. 5 that textureless inputs degrade reconstruction, which then degrades generation.
  • domain assumption A dataset-wide affine standardization makes the latent distribution close enough to a standard normal for rectified flow training.
    Sec. 3.3 assumes this based on the success of KL-regularized VAEs in 2D latent diffusion, but no quantitative check of Gaussianity, covariance stationarity, or per-category drift is provided.
  • domain assumption The density output of the frozen InstantMesh renderer reliably separates object from empty space.
    Sec. 3.4 computes the training mask by sampling triplane features over 3D coordinates and evaluating density through the renderer's MLP. If density estimation is unreliable, the mask misweights the loss.
  • domain assumption A single-step estimate of the clean triplane is a usable target for LPIPS backpropagation.
    Sec. 3.5, Eq. 3 approximates Z0 from one velocity evaluation. At high noise levels this estimate is crude, yet it is used to render patches and propagate perceptual gradients.
  • standard math The rectified flow ODE and velocity-matching objective are valid for training.
    Sec. 3.4 adopts rectified flow from Liu et al. and uses the standard velocity loss. This is accepted background mathematics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Taming Feed-forward Reconstruction Models as Latent Encoders for 3D Generative Models." pith.science (2026). https://pith.science/paper/LPCWDWXI

@misc{pith2026250100651,
  author       = {Pith},
  title        = {Pith review of: Taming Feed-forward Reconstruction Models as Latent Encoders for 3D Generative Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LPCWDWXI}},
  note         = {Machine review of arXiv:2501.00651}
}
read the original abstract

Recent AI-based 3D content creation has largely evolved along two paths: feed-forward image-to-3D reconstruction approaches and 3D generative models trained with 2D or 3D supervision. In this work, we show that existing feed-forward reconstruction methods can serve as effective latent encoders for training 3D generative models, thereby bridging these two paradigms. By reusing powerful pre-trained reconstruction models, we avoid computationally expensive encoder network training and obtain rich 3D latent features for generative modeling for free. However, the latent spaces of reconstruction models are not well-suited for generative modeling due to their unstructured nature. To enable flow-based model training on these latent features, we develop post-processing pipelines, including protocols to standardize the features and spatial weighting to concentrate on important regions. We further incorporate a 2D image space perceptual rendering loss to handle the high-dimensional latent spaces. Finally, we propose a multi-stream transformer-based rectified flow architecture to achieve linear scaling and high-quality text-conditioned 3D generation. Our framework leverages the advancements of feed-forward reconstruction models to enhance the scalability of 3D generative modeling, achieving both high computational efficiency and state-of-the-art performance in text-to-3D generation.

Figures

Figures reproduced from arXiv: 2501.00651 by the authors.

Figure 1
Figure 1. Text-to-3D generation. Our TriFlow model is trained on triplanes from a pretrained feed-forward reconstruction model [55] and can generate a high-quality 3D model in a few seconds. Left column: samples of a model trained on Objaverse [14] LVIS. Right: samples of models fine-tuned on ShapeNet [10] chairs and cars. Abstract Recent AI-based 3D content creation has largely evolved along two paths: feed-forward image-to-… view at source ↗
Figure 2
Figure 2. Overview of our image-to-3D generation pipeline and architecture. Our framework includes two main components: (1) a dataset preparation stage, where single-view or multi-view images are processed through a feed-forward image-to-triplane model to generate triplanes, and (2) TriFlow, a text-conditioned generative model trained on these triplanes using rectified-flow-based loss (LRF ) and perceptual loss (Llpips) compa… view at source ↗
Figure 3
Figure 3. Visualization of triplane features produced by In￾stantMesh [55] and the mask used for the weighted loss during training. Note the severe noise of the features in the empty spaces. control of information flow between streams allows for effi￾cient computation, enabling robust text-conditioned triplane generation at scale. 4. Experiments We begin by training our main model on the Objaverse dataset (LVIS) [13], a large… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Text-conditional generation on ShapeNet. While LN3Diff[32], a leading text-to-3D method, shows poor visual results when rendered from the side or bottom views, our results are of higher quality with fewer artifacts and adhere better to the input prompts unseen during t…
Figure 5
Figure 5. Figure 5: Some ShapeNet objects cause artifacts when rendered [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparisons on Objaverse. Compared with other SOTA text conditional 3D generation methods, we demonstrate a better balance between realistic rendering and text alignment with fewer visual artifacts, evidenced by superior CLIP metrics (Tab. 3). Zoom in for t…
Figure 7
Figure 7. Figure 7: Additional text-to-3D results on Objaverse. We showcase text conditional samples generated by TriFlow. 4 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Varying colors with text prompts. We asked our and baseline models to generate toilets in various unusual colors, including purple, blue, green, yellow, and orange. These colors do not appear in any of the existing toilet assets in the dataset. While the baseline metho…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Framework

    cs.GR 2025-09 conditional novelty 6.0 of 10

    SDF grids reconstruct best, Dual Octrees score best on automatic generation metrics, but users prefer SDF output, and reconstruction plus compression errors make up a large share of generation error.

Reference graph

Works this paper leans on

56 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Ren- derdiffusion: Image diffusion for 3d reconstruction, inpaint- ing and generation

    Titas Anciukevi ˇcius, Zexiang Xu, Matthew Fisher, Paul Hen- derson, Hakan Bilen, Niloy J Mitra, and Paul Guerrero. Ren- derdiffusion: Image diffusion for 3d reconstruction, inpaint- ing and generation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 12608–12618, 2023. 7

    work page 2023

  2. [2]

    Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond

    Mohammadreza Armandpour, Ali Sadeghian, Huangjie Zheng, Amir Sadeghian, and Mingyuan Zhou. Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond. arXiv preprint arXiv:2304.04968, 2023. 3

    arXiv 2023

  3. [3]

    4d-fy: Text-to-4d generation using hybrid score dis- tillation sampling

    Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B Lin- dell. 4d-fy: Text-to-4d generation using hybrid score dis- tillation sampling. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7996–8006, 2024. 2

    work page 2024

  4. [4]

    Flux.1: An advanced generative ai model

    Black Forest Labs. Flux.1: An advanced generative ai model. https://flux1.io/, 2024. Accessed: 2024-08-28. 1

    work page 2024

  5. [5]

    Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement

    Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement. arXiv preprint arXiv:2408.00653, 2024. 2, 3, 1

    arXiv 2024

  6. [6]

    Large scale gan training for high fidelity natural image synthesis

    Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arxiv 2018. arXiv preprint arXiv:1809.11096, 1809. 2

    arXiv 2018

  7. [7]

    Large-vocabulary 3d diffusion model with transformer

    Ziang Cao, Fangzhou Hong, Tong Wu, Liang Pan, and Ziwei Liu. Large-vocabulary 3d diffusion model with transformer. arXiv preprint arXiv:2309.07920, 2023. 2, 3, 7

    arXiv 2023

  8. [8]

    pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis

    Eric R Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein. pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5799–5809, 2021. 2

    work page 2021

Show all 56 references
  1. [9]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer v...

    2022

  2. [10]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 1, 5, 7

    2015 arXiv

  3. [11]

    Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction

    Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, and Hao Su. Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2416–2425, 2023. 2, 3

    2023

  4. [12]

    Introducing auraflow v0.1, an open exploration of large rectified flow models

    Cloneofsimo and Fal Lab. Introducing auraflow v0.1, an open exploration of large rectified flow models. https: //blog.fal.ai/auraflow/ , 2024. Accessed: 2024- 11-14. 5, 1

    2024

  5. [13]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  6. [14]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 1, 3

    2024

  7. [15]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 2

    2021

  8. [16]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  9. [17]

    Get3d: A generative model of high quality 3d tex- tured shapes learned from images

    Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generative model of high quality 3d tex- tured shapes learned from images. Advances In Neural In- formation Processing Systems , 35:31841–31854, 2022. 2, 7

    2022

  10. [18]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 2

    2014

  11. [19]

    3dgen: Triplane latent diffusion for textured mesh generation

    Anchit Gupta, Wenhan Xiong, Yixin Nie, Ian Jones, and Bar- las O˘guz. 3dgen: Triplane latent diffusion for textured mesh generation. arXiv preprint arXiv:2303.05371, 2023. 2, 3

    2023 arXiv

  12. [20]

    CLIPScore: a reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: a reference-free evaluation met- ric for image captioning. In EMNLP, 2021. 7

    2021

  13. [21]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 7

    2017

  14. [22]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2, 4

    2020

  15. [23]

    3dtopia: Large text-to-3d genera- tion model with hybrid diffusion priors

    Fangzhou Hong, Jiaxiang Tang, Ziang Cao, Min Shi, Tong Wu, Zhaoxi Chen, Shuai Yang, Tengfei Wang, Liang Pan, Dahua Lin, et al. 3dtopia: Large text-to-3d genera- tion model with hybrid diffusion priors. arXiv preprint arXiv:2403.02234, 2024. 5

    2024 arXiv

  16. [24]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2, 3

    2023 arXiv

  17. [25]

    Pointinfinity: Resolution- invariant point diffusion models

    Zixuan Huang, Justin Johnson, Shoubhik Debnath, James M Rehg, and Chao-Yuan Wu. Pointinfinity: Resolution- invariant point diffusion models. In Proceedings of the 9 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10050–10060, 2024. 5, 1

    2024

  18. [26]

    Re- thinking fid: Towards a better evaluation metric for image generation

    Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Re- thinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 930...

    2024

  19. [27]

    Scal- ing up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scal- ing up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10124–10134, 2023. 2

    2023

  20. [28]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 2

    2022

  21. [29]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  22. [30]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  23. [31]

    Improved precision and recall met- ric for assessing generative models

    Tuomas Kynk ¨a¨anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall met- ric for assessing generative models. Advances in neural in- formation processing systems, 32, 2019. 2

    2019

  24. [32]

    Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation

    Yushi Lan, Fangzhou Hong, Shuai Yang, Shangchen Zhou, Xuyi Meng, Bo Dai, Xingang Pan, and Chen Change Loy. Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation. In European Conference on Computer Vision , pages 112–130. Springer, 2025. 2, 3, 6, 7, 8, 5

    2025

  25. [33]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 3

    2023 arXiv

  26. [34]

    Direct-3d: Learning direct text-to-3d generation on massive noisy 3d data

    Qihao Liu, Yi Zhang, Song Bai, Adam Kortylewski, and Alan Yuille. Direct-3d: Learning direct text-to-3d generation on massive noisy 3d data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6881–6891, 2024. 2, 3, 6, 7, 8, 5

    2024

  27. [35]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. 4

    2022 arXiv

  28. [36]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 1

    2017 arXiv

  29. [37]

    Scalable 3d captioning with pretrained models

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin John- son. Scalable 3d captioning with pretrained models. Ad- vances in Neural Information Processing Systems, 36, 2024. 5

    2024

  30. [38]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 2

    2021

  31. [39]

    Diffrf: Rendering-guided 3d radiance field diffusion

    Norman M ¨uller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder, and Matthias Nießner. Diffrf: Rendering-guided 3d radiance field diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4328–4338, 2023. 2, 3, 7

    2023

  32. [40]

    Stylesdf: High-resolution 3d-consistent image and geome- try generation

    Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shecht- man, Jeong Joon Park, and Ira Kemelmacher-Shlizerman. Stylesdf: High-resolution 3d-consistent image and geome- try generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13503– 13513...

    2022

  33. [41]

    Benchmark for compositional text- to-image synthesis

    Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Dar- rell, and Anna Rohrbach. Benchmark for compositional text- to-image synthesis. In Proceedings of the Neural Informa- tion Processing Systems Track on Datasets and Benchmarks,

  34. [42]

    Deepsdf: Learning con- tinuous signed distance functions for shape representation

    Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning con- tinuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 165–174, 2019. 2

    2019

  35. [43]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  36. [44]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 2, 3

    2022 arXiv

  37. [45]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Con...

    2021

  38. [46]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  39. [47]

    Graf: Generative radiance fields for 3d-aware im- age synthesis

    Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger. Graf: Generative radiance fields for 3d-aware im- age synthesis. Advances in Neural Information Processing Systems, 33:20154–20166, 2020. 2

    2020

  40. [48]

    3d neural field generation using triplane diffusion

    J Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Wetzstein. 3d neural field generation using triplane diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20875–20886, 2023. 2, 3

    2023

  41. [49]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  42. [50]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  43. [51]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian 10 Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 3

    2024 arXiv

  44. [52]

    Lion: Latent point dif- fusion models for 3d shape generation

    Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point dif- fusion models for 3d shape generation. Advances in Neural Information Processing Systems, 35:10021–10039, 2022. 2

    2022

  45. [53]

    Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12619–12629, 2023. 2

    2023

  46. [54]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36, 2024. 2

    2024

  47. [55]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

    arXiv

  48. [56]

    a pair of comfortable blue jeans

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 5 11 Taming Feed-for...

    2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.