Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A variational autoencoder learns a compact latent space for whole 3D scenes of Gaussian splats, and it is the only tested model that reconstructs unseen scenes.

desk verdict A promising VAE for 3DGS scenes, but 'scene-level' is demonstrated only on salient-region crops; deserves review with conditions. read the letter →

arxiv 2508.01464 v1 pith:DHZD5CVG submitted 2025-08-02 cs.CV

Quankai Gao , Iliyan Georgiev , Tuanfeng Y. Wang , Krishna Kumar Singh , Ulrich Neumann , Jae Shin Yoon This is my paper
classification cs.CV
keywords 3DGaussianSplattingscene-levellatentmodelingvariationalautoencodercanonicaltokenizationscenegenerationdiffusionscalenormalizationsemantic-awarefiltering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that scene-level 3D Gaussian Splatting (3DGS) data can be compressed into a low-dimensional latent space by a purpose-built variational autoencoder, and that this latent space is the missing piece for feedforward 3D scene generation. The proposed model, Can3Tok, tokenizes tens of thousands of Gaussian primitives via cross-attention into canonical tokens and decodes them back into 3D, achieving a test-set failure rate of 2.5% on the DL3DV-10K scene dataset while three existing point-cloud or 3D-convolution VAEs fail on 70–100% of unseen scenes. The paper also argues that scale inconsistency across scenes is a fundamental obstacle that no architecture alone can overcome, and introduces normalization plus semantic-aware filtering as the data preprocessing needed to make large-scale training possible. If correct, these results provide a stable scene-level latent space that text-conditioned or image-conditioned models can be attached to, turning per-scene optimization into feedforward generation.

What carries the argument

The carrying mechanism is Can3Tok, a transformer-based variational autoencoder whose encoder uses a single cross-attention block to compress N Gaussian primitives (with Fourier positional encodings of their centers and of nearest voxel centers) against a learnable canonical query initialized with regular voxel coordinates; eight self-attention blocks refine the tokens, and a VAE bottleneck maps them to mean and log-variance latents. The decoder applies 16 self-attention blocks and then linear layers to regress the 3DGS parameters back into continuous space. Two data-side mechanisms are load-bearing: 3DGS normalization (mean-shifting and sphere-bounding both Gaussians and their associated camera centers) removes cross-scene scale inconsistency, and semantic-aware filtering (text-prompt segmentation on a middle frame followed by K-NN propagation to 40K Gaussians) removes floaters that otherwise wash out high-frequency detail. The learned latent has the same 64x64x4 size as image latent diffusion spaces, which is what makes plugging a diffusion UNet onto it straightforward.

What would settle it

Train Can3Tok on the same DL3DV-10K splits with the same normalization and augmentation but without semantic-aware filtering, using all scene Gaussians, and compare test-set failure rates. If the unfiltered model's failure rate jumps far above 2.5%, the generalization claim depends on the salient-region crop; if it stays comparable, the crop is not essential. As a complementary check, measure the fraction of the scene's spatial extent (for example, the volume of the bounding box of the cropped 40K Gaussians divided by the full scene's bounding-box volume) and show whether the retained region is a small object-like fraction.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that unstructured, unbounded 3D Gaussian scenes are learnable by a VAE if the encoder's tokens are formed with a learnable canonical query anchored in a regular voxel grid and if the scenes are first normalized into a bounded sphere and cleaned of hallucinated regions. With those choices, a transformer VAE with 12-head attention encodes a 40K-Gaussian scene into a 64x64x4 latent, and decoding reconstructs the scene well enough that only 2.5% of test scenes fail, compared with 70–100% failure for PointNet, L3DG, and PointTransformer baselines trained on the same data. The paper presents this as the first demonstration that scene-level 3DGS latent representation learning scales to thousands of real scenes and generalizes to novel inputs, and it shows text-to-3DGS and image-to-3DGS generation as downstream applications of that latent space.

Load-bearing premise

The load-bearing premise is that cropping each scene to its text-segmented 'most salient region' and keeping only 40K nearby Gaussians preserves the properties that make scene-level generation hard, such as backgrounds, layouts, and scale variation, rather than quietly turning the task into salient-object reconstruction.

Editorial extensions

If this is right

  • A diffusion model can be trained directly on Can3Tok latents: the paper shows text-conditioned denoising from a random latent to a coherent 3DGS scene, with the decoder frozen.
  • Image-to-3DGS generation works by regressing an image to the latent and decoding, avoiding per-scene optimization.
  • The 2.5% residual failure rate plus the ablation numbers imply that each component, including canonical query, normalization, filtering, voxel appending, and augmentation, contributes to convergence and detail, so future scene-level 3D generative models will likely need similar preprocessing.
  • Because normalization rescales camera centers along with the scene, rendered views are unchanged and metric scale can be recovered from generated scenes via depth estimation.
  • The 64x64x4 latent geometry aligns with existing latent-diffusion infrastructure, so scene generation can inherit 2D generative tooling rather than requiring a new generative backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The filtering step crops each scene to the text-guided 'most salient region,' so the demonstrated generalization may be for salient-region scenes rather than full scenes with backgrounds and unbounded layouts. A direct test would be training on full normalized scenes without filtering and measuring whether the failure rate stays near 2.5%; the paper's own no-filtering ablation (6.1% failure) sugge
  • If the latent space is as structured as the t-SNE loops suggest, other generative heads, such as flow matching, autoregressive token prediction, or diffusion transformers, could likely replace the UNet demo without retraining the VAE, analogous to how image latents support multiple generators.
  • The method is tied to 3DGS's discrete primitive structure; adapting it to continuous implicit representations would require a different tokenization, so the approach's scope is likely Gaussian-splatting-native pipelines.
  • A testable extension is to vary the number N of Gaussians kept by the filter and measure reconstruction quality; if quality degrades sharply beyond a small N, the latent's capacity for scene-level detail is concentrated in the salient crop.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Can3Tok, a variational autoencoder designed to compress scene-level 3D Gaussian Splatting (3DGS) representations, which consist of tens of thousands of Gaussian primitives, into a low-dimensional latent space. The encoder uses a Perceiver-style cross-attention with a learnable canonical query initialized from a regular voxel grid, followed by self-attention blocks; the decoder maps the latent back to 3DGS parameters. To handle scale inconsistency across scenes, the authors propose a normalization scheme that mean-shifts and rescales Gaussian centers and scales into a bounded sphere. They also introduce semantic-aware filtering, which uses LangSAM on the middle frame to select the "most salient region" and retains 40K Gaussians per scene. Experiments on DL3DV-10K show that Can3Tok reconstructs unseen scenes with a 2.5% failure rate, whereas re-implemented baselines (L3DG, PointNet VAE, PointTransformer) fail at 70-100%, and the paper demonstrates text-to-3DGS and image-to-3DGS applications using the learned latent space.

Significance. If the claims are substantiated, Can3Tok could be a meaningful step toward feedforward 3D scene generation, providing a compact and structured latent space for 3DGS that is compatible with diffusion models. The architecture is simple and computationally efficient, the normalization is well-motivated, and the paper includes ablations and latent-space visualizations. The main strengths are the clear problem formulation, the careful identification of scale inconsistency as a training obstacle, and the demonstration of a working 3DGS VAE at a scale (40K-100K Gaussians) that prior object-level VAEs do not handle. However, the significance of the central claim depends on whether the evaluation truly covers "scene-level" content, which is currently in question because the data filtering step removes backgrounds, layouts, and unbounded-scale structure.

major comments (3)
  1. [Section 3.3 and Section 4.1] The semantic-aware filtering reduces each scene to 40K Gaussians in the "most salient region" via LangSAM on the middle frame and K-NN propagation. Since the paper itself (Section 1) identifies background, layout, and unbounded scale as the core challenges of scene-level 3D, this filtering removes exactly those challenges. Consequently, the failure rates in Table 1 and the reconstruction quality in Fig. 6 are evidence for salient-region crops, not for full scene-level 3DGS. The "w/o data filtering" ablation in Table 2 trains on raw 100K-Gaussian noisy inputs; it does not isolate whether background and layout content are faithfully encoded and decoded. To support the headline claim of being the first scene-level 3DGS VAE, the authors should either re-scope the claims to "salient-region 3DGS" or provide experiments on unfiltered full scenes, e.g., encoding all Gaussians or evaluating background-region reconstruction separately.
  2. [Section 4.2] The three baselines (L3DG, PointNet VAE, PointTransformer) are implemented by the authors, and no hyperparameters or training budgets are reported for them. The claim that only Can3Tok converges and generalizes is therefore sensitive to the fairness of these implementations. The authors should release the baseline code and settings, or compare with official pretrained models where available. At minimum, report the number of parameters, training iterations, and final training loss for each baseline to demonstrate that they received comparable capacity and optimization effort. Without this, the qualitative and quantitative failure of the baselines may reflect under-tuning rather than an inherent limitation of those architectures.
  3. [Section 4.2, Table 1] The primary metric is the L2 distance in 3DGS parameter space, with a failure threshold of 1000.0. This parameter-space metric does not necessarily reflect rendered image quality, which is the intended downstream use. The paper should include rendering-based metrics (PSNR, SSIM, LPIPS) on held-out viewpoints for the reconstructed Gaussians, and justify the failure threshold. Without such metrics, the quantitative superiority of Can3Tok over PointTransformer (L2 30.1 vs 230.7) is hard to interpret, since a large parameter-space error may or may not correspond to visibly worse renderings.
minor comments (5)
  1. [Section 3.2] There is a typo: "and the the decoder reconstructs" should read "and the decoder reconstructs".
  2. [Section 4.1] The symbol N is used both for the raw Gaussian count (100K) and the filtered count (40K), which is confusing; consider using N_raw and N_filtered.
  3. [Section 4.1] The notation "~0.06 sec" should be "approximately 0.06 sec" for clarity.
  4. [Section 4.6] The text-to-3DGS application does not report the number of text prompts used, the size of the diffusion training set, or whether the UNet was trained from scratch; please add these details for reproducibility.
  5. [Appendix, Figures 12 and 13] The captions appear mismatched: Figure 12 is described as "Illustration of image-to-3DGS architecture" but seems to show qualitative examples, while Figure 13 is described as "Qualitative examples" but is referenced as the pipeline figure; please correct the captions and cross-references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Can3Tok's VAE uses a standard reconstruction objective on held-out DL3DV-10K splits, with no fitted parameters, self-citation chains, or uniqueness arguments defining the target result.

full rationale

The derivation chain for the central claim (Can3Tok is the first scene-level 3DGS VAE that generalizes to novel scenes) is empirical and self-contained. The training objective in Eq. (2) is the standard VAE loss: L2 distance between decoded and input Gaussian parameters plus a KL term; there is no fitted constant that is later relabeled as a prediction. Generalization is measured on the DL3DV-10K testing split, disjoint from training, using the same normalization (Eq. 3), semantic-aware filtering, and 40K-Gaussian tokenization at train and test time. The ablations in Table 2 isolate each component (learnable query, normalization, voxel appending, filtering, augmentation) rather than constructing the main result from an ablation. No load-bearing result is imported from a same-author citation, no uniqueness theorem is invoked, and the PerceiverIO/LangSAM/BLIP references are external prior work used as building blocks, not as proof of Can3Tok's effectiveness. The semantic-aware filtering that crops each scene to the 'most salient region' is a legitimate data-distribution choice; it may limit the scope of the 'scene-level' claim (backgrounds, layouts, and unbounded scale are partially removed), but that is a generality/external-validity concern, not circularity: the reconstruction target is not defined in terms of the model output, and the paper does not fit any parameter to the test set. The image-to-3DGS and text-to-3DGS applications train modules to reproduce latents produced by Can3Tok, which is an application pipeline rather than evidence used to establish the VAE's reconstruction claim. Accordingly, no circular step meeting the quoted-evidence standard is present.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central contribution is an empirical architecture, so the ledger lists design hyperparameters chosen by hand, and domain assumptions about 3DGS data. No new physical entities are introduced; the learnable canonical query is a learned parameter tensor, not an independent entity.

free parameters (7)
  • Normalization radius r = unspecified (unit sphere presumably)
    Eq. 3 scales every scene into a sphere of radius r; r is chosen by hand and affects global scale of all scenes.
  • Fourier positional encoding band LB = 51
    Section 4.1 sets LB=51; this controls the frequency content of position features.
  • Voxel resolution V = 40
    Section 4.1 sets V=40 for the voxel grid defining nearest-voxel-center anchors.
  • KL weight lambda = 1e-6
    Eq. 2 balances reconstruction and KL divergence; chosen by hand.
  • Number of Gaussians N after filtering = 40,000
    Section 4.1: each scene is subsampled to N=40K Gaussians, bounding the input size.
  • Canonical query size = 256x768
    Section 4.1: latent query Q has 256 tokens of dimension 768.
  • Latent embedding shape = 64x64x4
    Section 4.1: mean and log-variance are 64x64x4, chosen to match Stable Diffusion latent size.
assumptions (5)
  • standard math Fourier positional encoding improves learning of high-frequency spatial functions in transformer architectures.
    Invoked in Section 3.2 with references [43, 64].
  • domain assumption Scene-level 3DGS reconstructions from COLMAP-based SfM have inconsistent, non-metric global scales and per-Gaussian scales, and this inconsistency prevents VAE training from converging across many scenes.
    Motivates the normalization in Eq. 3 (Section 3.3); the paper states that even powerful VAEs fail without it.
  • domain assumption The most salient region identified by LangSAM on the middle frame of each video contains the cleanest and most relevant Gaussians, and K-NN propagation from a seed Gaussian yields a noise-free training distribution.
    Section 3.3 Semantic-aware Filtering; this is the premise that noisy floaters can be removed without losing scene content.
  • domain assumption The L2 distance between predicted and input 3DGS parameters is a meaningful reconstruction objective for learning a useful latent space.
    Equation 2 trains on this loss; the paper's evaluation metric is the same L2 error, so metric choice directly defines what 'good reconstruction' means.
  • domain assumption A VAE with KL divergence to N(0,I) produces a structured, generatively useful latent space without additional constraints.
    Standard VAE assumption invoked in Eq. 2 and Section 4.4, where the authors admit the structure is not fully understood ('inconclusive').

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians." pith.science (2026). https://pith.science/paper/DHZD5CVG

@misc{pith2026250801464,
  author       = {Pith},
  title        = {Pith review of: Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHZD5CVG}},
  note         = {Machine review of arXiv:2508.01464}
}
read the original abstract

3D generation has made significant progress, however, it still largely remains at the object-level. Feedforward 3D scene-level generation has been rarely explored due to the lack of models capable of scaling-up latent representation learning on 3D scene-level data. Unlike object-level generative models, which are trained on well-labeled 3D data in a bounded canonical space, scene-level generations with 3D scenes represented by 3D Gaussian Splatting (3DGS) are unbounded and exhibit scale inconsistency across different scenes, making unified latent representation learning for generative purposes extremely challenging. In this paper, we introduce Can3Tok, the first 3D scene-level variational autoencoder (VAE) capable of encoding a large number of Gaussian primitives into a low-dimensional latent embedding, which effectively captures both semantic and spatial information of the inputs. Beyond model design, we propose a general pipeline for 3D scene data processing to address scale inconsistency issue. We validate our method on the recent scene-level 3D dataset DL3DV-10K, where we found that only Can3Tok successfully generalizes to novel 3D scenes, while compared methods fail to converge on even a few hundred scene inputs during training and exhibit zero generalization ability during inference. Finally, we demonstrate image-to-3DGS and text-to-3DGS generation as our applications to demonstrate its ability to facilitate downstream generation tasks.

Figures

Figures reproduced from arXiv: 2508.01464 by the authors.

Figure 1
Figure 1. Reconstruction results from the latent space of 3D Gaus [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example of unstructured 3DGS representation of a scene [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Can3Tok processes a batch of per-scene 3D Gaussians, with a batch size of B, where each scene contains the same number of [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Before and after our semantic-aware 3DGS filtering. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Qualitative comparisons between ours and other VAE outputs. Results are not cherry picked as all compared methods show zero [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Given 3DGS reconstruction for various scenes, we ran [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Two examples of text-conditioned generation with a latent diffusion UNet, which is trained on our 3DGS embeddings of scenes [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: A pipeline for text-conditioned latent diffusion model [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Ablation study for with and without nearest voxel coor [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Illustration of image-to-3DGS architecture. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Qualitative examples of our image-to-3DGS applica [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: More generative results of our method with correspond [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Qualitative comparisons of w/ and w/o Can3Tok. [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Qualitative comparisons w/ and w/o normalization. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 19
Figure 19. Figure 19: More qualitative results. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_19.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A causal transformer with 3D RoPE generates vector-quantized 3D Gaussian latent grids autoregressively, enabling unconditional synthesis, completion, and open-ended outpainting of indoor scenes.

Reference graph

Works this paper leans on

94 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    Multilayer perceptrons

    Luis B Almeida. Multilayer perceptrons. In Handbook of Neural Computation, pages C1–2. CRC Press, 2020. 3

    2020

  2. [2]

    Distributions of the kullback–leibler divergence with applications

    Dmitry I Belov and Ronald D Armstrong. Distributions of the kullback–leibler divergence with applications. British Journal of Mathematical and Statistical Psychology , 64(2): 291–309, 2011. 3

    2011

  3. [3]

    Generative and discriminative voxel mod- eling with convolutional neural networks

    Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. Generative and discriminative voxel mod- eling with convolutional neural networks. arXiv preprint arXiv:1608.04236, 2016. 3

    arXiv 2016

  4. [4]

    Pythae: Unifying generative autoencoders in python - a benchmarking use case

    Cl ´ement Chadebec, Louis Vincent, and Stephanie Allasson- niere. Pythae: Unifying generative autoencoders in python - a benchmarking use case. InAdvances in Neural Information Processing Systems, pages 21575–21589. Curran Associates, Inc., 2022. 8, 13

    2022

  5. [5]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 2

    arXiv 2015

  6. [6]

    Meshanything: Artist-created mesh generation with au- toregressive transformers

    Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Ji- axiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, et al. Meshanything: Artist-created mesh generation with au- toregressive transformers. arXiv preprint arXiv:2406.10163,

    arXiv

  7. [7]

    Gaussianpro: 3d gaussian splatting with progressive propagation

    Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, and Xuejin Chen. Gaussianpro: 3d gaussian splatting with progressive propagation. InForty- first International Conference on Machine Learning, 2024. 2

    2024

  8. [8]

    4d spatio-temporal convnets: Minkowski convolutional neural networks

    Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition , pages 3075–3084,

Show all 94 references
  1. [9]

    Luciddreamer: Domain-free gen- eration of 3d gaussian splatting scenes

    Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free gen- eration of 3d gaussian splatting scenes. arXiv preprint arXiv:2311.13384, 2023. 1, 3

    2023 arXiv

  2. [10]

    Spconv: Spatially sparse convolu- tion library

    Spconv Contributors. Spconv: Spatially sparse convolu- tion library. https://github.com/traveller59/ spconv, 2022. 6

    2022

  3. [11]

    Fu, Stefano Ermon, Atri Rudra, and Christopher R´e

    Tri Dao, Daniel Y . Fu, Stefano Ermon, Atri Rudra, and Christopher R´e. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022. 5

    2022

  4. [12]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3

    2010 arXiv

  6. [14]

    Instantsplat: Un- bounded sparse-view pose-free gaussian splatting in 40 sec- onds

    Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang, Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, et al. Instantsplat: Un- bounded sparse-view pose-free gaussian splatting in 40 sec- onds. arXiv preprint arXiv:2403.20309, 2024. 2

    2024 arXiv

  7. [15]

    Scenescape: Text-driven consistent scene generation

    Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. Scenescape: Text-driven consistent scene generation. Ad- vances in Neural Information Processing Systems, 36, 2024. 3

    2024

  8. [16]

    Get3d: A generative model of high quality 3d tex- tured shapes learned from images

    Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generative model of high quality 3d tex- tured shapes learned from images. Advances In Neural In- formation Processing Systems , 35:31841–31854, 2022. 1, 3

    2022

  9. [17]

    Vfusion3d: Learning scalable 3d generative models from video diffusion models

    Junlin Han, Filippos Kokkinos, and Philip Torr. Vfusion3d: Learning scalable 3d generative models from video diffusion models. In European Conference on Computer Vision, pages 333–350. Springer, 2025. 3

    2025

  10. [18]

    Text2room: Extracting textured 3d meshes from 2d text-to-image models

    Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7909–7920, 2023. 3

    2023

  11. [19]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 1, 3

    2023 arXiv

  12. [20]

    Mvd-fusion: Single-view 3d via depth-consistent multi-view generation

    Hanzhe Hu, Zhizhuo Zhou, Varun Jampani, and Shubham Tulsiani. Mvd-fusion: Single-view 3d via depth-consistent multi-view generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9698–9707, 2024. 3

    2024

  13. [21]

    Perceiver io: A general architecture for structured inputs & outputs

    Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Kop- pula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. arXiv preprint arXiv:2107.14795, 2021...

    2021 arXiv

  14. [22]

    Perceiver: General perception with iterative attention

    Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In International confer- ence on machine learning, pages 4651–4664. PMLR, 2021. 4

    2021

  15. [23]

    Leap: Liberate sparse-view 3d modeling from camera poses

    Hanwen Jiang, Zhenyu Jiang, Yue Zhao, and Qixing Huang. Leap: Liberate sparse-view 3d modeling from camera poses. arXiv preprint arXiv:2310.01410, 2023

    2023 arXiv

  16. [24]

    Shap-e: Generat- ing conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 3

    2023 arXiv

  17. [25]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  18. [26]

    9 A hierarchical 3d gaussian representation for real-time ren- dering of very large datasets.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024

    Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. 9 A hierarchical 3d gaussian representation for real-time ren- dering of very large datasets.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024. 2

    2024

  19. [27]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3, 4

    2013 arXiv

  20. [28]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2, 5

    2023

  21. [29]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 8

    2022

  22. [30]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 3

    2023 arXiv

  23. [31]

    Infinitenature-zero: Learning perpetual view generation of natural scenes from single images

    Zhengqi Li, Qianqian Wang, Noah Snavely, and Angjoo Kanazawa. Infinitenature-zero: Learning perpetual view generation of natural scenes from single images. In Eu- ropean Conference on Computer Vision , pages 515–534. Springer, 2022. 3

    2022

  24. [32]

    Analytic-splatting: Anti-aliased 3d gaussian splatting via analytic integration

    Zhihao Liang, Qi Zhang, Wenbo Hu, Ying Feng, Lei Zhu, and Kui Jia. Analytic-splatting: Anti-aliased 3d gaussian splatting via analytic integration. arXiv preprint arXiv:2403.11056, 2024. 2

    2024 arXiv

  25. [33]

    Infinicity: Infinite-scale city synthesis

    Chieh Hubert Lin, Hsin-Ying Lee, Willi Menapace, Menglei Chai, Aliaksandr Siarohin, Ming-Hsuan Yang, and Sergey Tulyakov. Infinicity: Infinite-scale city synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22808–22818, 2023. 3

    2023

  26. [34]

    Vastgaussian: Vast 3d gaussians for large scene reconstruction

    Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, You- liang Yan, et al. Vastgaussian: Vast 3d gaussians for large scene reconstruction. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogn...

    2024

  27. [35]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , p...

    2024

  28. [36]

    Infinite na- ture: Perpetual view generation of natural scenes from a sin- gle image

    Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. Infinite na- ture: Perpetual view generation of natural scenes from a sin- gle image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14458–14467, 2021. 3

    2021

  29. [37]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 3

    2023

  30. [38]

    Scaffold-gs: Structured 3d gaussians for view-adaptive rendering

    Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20654–20664, 2024. 2

    2024

  31. [39]

    Learn- ing disentangled representations of timbre and pitch for mu- sical instrument sounds using gaussian mixture variational autoencoders

    Yin-Jyun Luo, Kat Agres, and Dorien Herremans. Learn- ing disentangled representations of timbre and pitch for mu- sical instrument sounds using gaussian mixture variational autoencoders. arXiv preprint arXiv:1906.08152, 2019. 3

    1906 arXiv

  32. [40]

    Shapes- plat: A large-scale dataset of gaussian splats and their self- supervised pretraining

    Qi Ma, Yue Li, Bin Ren, Nicu Sebe, Ender Konukoglu, Theo Gevers, Luc Van Gool, and Danda Pani Paudel. Shapes- plat: A large-scale dataset of gaussian splats and their self- supervised pretraining. arXiv preprint arXiv:2408.10906 ,

    arXiv

  33. [41]

    Normalized image representation for efficient coding

    Jes ´us Malo. Normalized image representation for efficient coding. In The Thrity-Seventh Asilomar Conference on Sig- nals, Systems & Computers, 2003, pages 1408–1412. IEEE,

    2003

  34. [42]

    Text2mesh: Text-driven neural stylization for meshes

    Oscar Michel, Roi Bar-On, Richard Liu, Sagie Benaim, and Rana Hanocka. Text2mesh: Text-driven neural stylization for meshes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13492– 13502, 2022. 3

    2022

  35. [43]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 4

    2020

  36. [44]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 2, 3, 4

    2021

  37. [45]

    Point-e: A system for generat- ing 3d point clouds from complex prompts

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022. 1, 3

    2022 arXiv

  38. [46]

    Radsplat: Radiance field-informed gaussian splat- ting for robust real-time rendering with 900+ fps

    Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakoto- saona, Michael Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik Kaeser, and Federico Tombari. Radsplat: Radiance field-informed gaussian splat- ting for robust real-time rendering with 900+ fps. ...

    2024 arXiv

  39. [47]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  40. [48]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 1, 3

    2022 arXiv

  41. [49]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. arXiv preprint arXiv:1612.00593, 2016. 3

    2016 arXiv

  42. [50]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660,

  43. [51]

    Point- net++: Deep hierarchical feature learning on point sets in a metric space

    Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Point- net++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413, 2017. 2

    2017 arXiv

  44. [52]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  45. [53]

    L3dg: Latent 3d gaussian diffusion

    Barbara Roessle, Norman M ¨uller, Lorenzo Porzi, Samuel Rota Bul `o, Peter Kontschieder, Angela Dai, and Matthias Nießner. L3dg: Latent 3d gaussian diffusion. arXiv preprint arXiv:2410.13530, 2024. 1, 3, 6

    2024 arXiv

  46. [54]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 6

    2021

  47. [55]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 13

    2022

  48. [56]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. In Conference on Com- puter Vision and Pattern Recognition (CVPR), 2016. 5

    2016

  49. [57]

    Structure- from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure- from-motion revisited. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 4104–4113, 2016. 5

    2016

  50. [58]

    Mvdream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. arXiv preprint arXiv:2308.16512, 2023. 3

    2023 arXiv

  51. [59]

    A tutorial on principal component analysis

    Jonathon Shlens. A tutorial on principal component analysis. arXiv preprint arXiv:1404.1100, 2014. 3

    2014 arXiv

  52. [60]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Ta- tiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  53. [61]

    Flash3d: Feed-forward gener- alisable 3d scene reconstruction from a single image

    Stanislaw Szymanowicz, Eldar Insafutdinov, Chuanxia Zheng, Dylan Campbell, Joao F Henriques, Christian Rup- precht, and Andrea Vedaldi. Flash3d: Feed-forward gener- alisable 3d scene reconstruction from a single image. arXiv preprint arXiv:2406.04343, 2024. 2, 13

    2024 arXiv

  54. [62]

    Splatter image: Ultra-fast single-view 3d recon- struction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d recon- struction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10208– 10217, 2024. 2, 13

    2024

  55. [63]

    Bolt3d: Generating 3d scenes in seconds

    Stanislaw Szymanowicz, Jason Y Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Aleksander Holynski, Ricardo Martin-Brualla, Jonathan T Barron, and Philipp Henzler. Bolt3d: Generating 3d scenes in seconds. arXiv preprint arXiv:2503.14445, 2025. 6

    2025

  56. [64]

    Fourier features let networks learn high frequency functions in low dimen- sional domains

    Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ra- mamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimen- sional domains. Advances in neural information ...

    2020

  57. [65]

    Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation. arXiv preprint arXiv:2309.16653,

    arXiv

  58. [66]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision, pages 1–18. Springer, 2025. 3

    2025

  59. [67]

    Learning spatiotemporal features with 3d convolutional networks

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. InProceedings of the IEEE inter- national conference on computer vision , pages 4489–4497,

  60. [68]

    NV AE: A deep hierarchical variational autoencoder

    Arash Vahdat and Jan Kautz. NV AE: A deep hierarchical variational autoencoder. In Neural Information Processing Systems (NeurIPS), 2020. 3

    2020

  61. [69]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9 (11), 2008. 6, 14

    2008

  62. [70]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 3

    2017

  63. [71]

    Attentive normalization for conditional image generation

    Yi Wang, Ying-Cong Chen, Xiangyu Zhang, Jian Sun, and Jiaya Jia. Attentive normalization for conditional image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5094– 5103, 2020. 5

    2020

  64. [72]

    Multimodal token fusion for vision transformers

    Yikai Wang, Xinghao Chen, Lele Cao, Wenbing Huang, Fuchun Sun, and Yunhe Wang. Multimodal token fusion for vision transformers. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 12186–12195, 2022. 3

    2022

  65. [73]

    Mio: A foundation model on multimodal tokens

    Zekun Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jiashuo Wang, Ning Shi, Siyu Li, Yizhi Li, et al. Mio: A foundation model on multimodal tokens. arXiv preprint arXiv:2409.17692, 2024. 3

    2024

  66. [74]

    Synsin: End-to-end view synthesis from a sin- gle image

    Olivia Wiles, Georgia Gkioxari, Richard Szeliski, and Justin Johnson. Synsin: End-to-end view synthesis from a sin- gle image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7467–7477,

  67. [75]

    Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Ji- ahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model. arXiv preprint arXiv:2311.09217, 2023. 3

    2023 arXiv

  68. [76]

    Gaussian- dreamer: Fast generation from text to 3d gaussian splatting with point cloud priors

    Taoran Yi, Jiemin Fang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussian- dreamer: Fast generation from text to 3d gaussian splatting with point cloud priors. arXiv preprint arXiv:2310.08529 ,

    arXiv

  69. [77]

    Wonderworld: Interactive 3d scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. arXiv preprint arXiv:2406.09394, 2024. 3 11

    2024 arXiv

  70. [78]

    Wonderjourney: Going from anywhere to everywhere

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, De- qing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from anywhere to everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2024

  71. [79]

    Mvimgnet: A large-scale dataset of multi-view images

    Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognit...

    2023

  72. [80]

    Mip-splatting: Alias-free 3d gaussian splat- ting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splat- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 19447–19456,

  73. [81]

    Gaussiancube: Structuring gaussian splatting using opti- mal transport for 3d generative modeling

    Bowen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, and Baining Guo. Gaussiancube: Structuring gaussian splatting using opti- mal transport for 3d generative modeling. arXiv preprint arXiv:2403.19655, 2024. 3

    2024 arXiv

  74. [82]

    Gs-lrm: Large recon- struction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large recon- struction model for 3d gaussian splatting. In European Con- ference on Computer Vision, pages 1–19. Springer, 2024. 2

    2024

  75. [83]

    Point transformer

    Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16259–16268, 2021. 3, 6

    2021

  76. [84]

    Cv-vae: A compatible video vae for latent generative video models

    Sijie Zhao, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Muyao Niu, Xiaoyu Li, Wenbo Hu, and Ying Shan. Cv-vae: A compatible video vae for latent generative video models. arXiv preprint arXiv:2405.20279, 2024. 3

    2024 arXiv

  77. [85]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in Neural Information Processing Systems, 36, 2024. 3,...

    2024

  78. [86]

    Ablation Studies We perform ablation study for the importance of each mod- ule of our method. In Tab. 2 of the main paper, we describe the overall quantitative comparison over different ablation studies. More specifically, we verify the performance by removing each of the foll...

  79. [87]

    w/o Learnable Query : We remove the learnable canon- ical latent query and we replace the cross-attention block with self-attention. We observe that simply replacing it with self-attention fails to converge and is more likely to en- counter out-of-memory issue, even with a bat...

  80. [88]

    Both Tab.2 and Fig

    w/o normalization: we do not apply the normalization of data to the entire 3DGS training dataset. Both Tab.2 and Fig. 16 highlight the severe scale inconsistency issue if a V AE model is trained on raw 3DGS input, which hinders scaling up training across thousands of scenes. A...

  81. [89]

    w/o data filtering: we use raw 3DGS reconstruction re- sults as a training set without data filtering. Fig. 18 implies that by suppressing the significant noise by data filtering, the models better learn the mapping between the latent and inputs in a way that preserves the loc...

  82. [90]

    Instead, we append the positional embed- ding only from 3DGS’s position

    w/o voxel coordinate appending : we turn off the dual positional embedding from 3DGS’s position and its near- est voxel center. Instead, we append the positional embed- ding only from 3DGS’s position. Fig. 17 show the effect of voxel coordinate appending where its to preserve ...

  83. [91]

    w/o voxel data enhancement : we disable data enhance- ment during training

  84. [92]

    19, we demonstrate more results from our Can3Tok with various test scenes

    More Results In Fig. 19, we demonstrate more results from our Can3Tok with various test scenes

  85. [93]

    Ground-Truth 3D Gaussian la- tents

    Application: Image-to-3DGS In this section, we showcase the application of our Can3Tok latent space modeling. Other than text-to-3DGS, our latent features can be used for image-to-3DGS generation. To this end, we use an image encoder ( e.g., [4]) that takes as input 2D images ...

  86. [94]

    More Discussions About speed, our Can3Tok V AE (1.1 s/iters) is 10 times faster than L3DG (11.3 s/iters). This is because our method accelerate self-attention steps by reducing the input dimen- sion with a latent query, while 3D convolution step itself is slower than our self-...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.