Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

GaussFusion: Towards Multimodal 3D Gaussian Pretraining

T0 review · 2 major / 6 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding image and text supervision to 3D Gaussian pretraining

desk verdict Solid incremental work on Gaussian pretraining with one real concern about category leakage from text supervision. read the letter →

arxiv 2607.05906 v1 pith:GLVG73GJ submitted 2026-07-07 cs.CV

Zhixuan You , Jihua Zhu , Yiding Sun , Zihao Guo , Haozhe Cheng , Dongxu Zhang , Lin Chen , Hainan Luo This is my paper · ORCID
classification cs.CV
keywords 3DGaussianSplattingself-supervisedpre-trainingmultimodallearningcross-modalalignmentmaskedautoencoderpointcloudunderstandingvision-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that pre-training methods for 3D Gaussian Splatting representations, which have so far relied solely on reconstructing masked Gaussian attributes, learn local geometry but miss high-level semantics. The authors propose GaussFusion, a framework that adds cross-modal supervision from frozen 2D image and text encoders to the masked Gaussian modeling pipeline. Two learnable query tokens are prepended to the Gaussian token sequence inside a shared Transformer encoder; these tokens aggregate 3D features through standard self-attention and are then projected to match the output of a frozen ViT image encoder and a frozen CLIP text encoder via Smooth L1 loss. The image and text encoders are used only during pre-training and are discarded at inference, so downstream tasks receive a purely 3D model with no extra modal input. To make the masked reconstruction objective compatible with cross-modal alignment, the paper also introduces Gaussian Salience-guided Multi-scale Hole Masking (GSHM), which replaces random per-group masking with spatially continuous, multi-scale masked regions selected by a salience score derived from each Gaussian group's average opacity and geometric-mean scale. The combined system is pre-trained on the ShapeSplat dataset (~52,000 objects) and evaluated by transferring the Gaussian encoder to point-cloud classification on ScanObjectNN and ModelNet40, part segmentation on ShapeNetPart, and few-shot classification on ModelNet40. Under the same Gaussian-representation setting, GaussFusion outperforms the prior Gaussian-MAE baseline by 3.85 percent on the hardest ScanObjectNN split and by 0.61 percent on ModelNet40, with consistent gains across segmentation and few-shot tasks.

What carries the argument

Two learnable alignment tokens (image and text) prepended to a Gaussian Transformer encoder sequence, aligned via Smooth L1 loss to frozen ViT and CLIP outputs; GSHM masking using opacity-scale salience scores, salience-weighted farthest-point sampling for hole centers, and per-hole scale perturbation for multi-scale spatially continuous masks; joint loss combining Gaussian attribute reconstruction (Chamfer Distance for xyz, L1 for other attributes) with image and text alignment losses.

What would settle it

If the alignment tokens learn to predict frozen image/text features through a shortcut that depends only on global object shape (e.g., overall bounding-box proportions visible from a single viewpoint), then removing the alignment tokens at fine-tuning time would yield no improvement over Gaussian-MAE on tasks requiring local geometric discrimination, such as part segmentation on categories with thin structures. A direct test: pre-train with alignment tokens but evaluate part-segmentation IoU per category; if gains concentrate on categories whose parts are distinguishable by global shape alone,

Watch

Extended reading notes

Core claim

The central claim is that a 3D Gaussian encoder can absorb category-level and appearance-level semantics from frozen 2D vision-language models during pre-training through two learnable alignment tokens, and that this cross-modal supervision is more effective when the masking strategy produces spatially coherent, salience-weighted holes rather than random scattered masks. The mechanism carrying the argument is the combination of (1) prepended image/text query tokens that collect 3D context via self-attention and are aligned to external frozen features, and (2) GSHM, which uses opacity-and-scale salience scores with farthest-point sampling to build multi-scale contiguous masked regions adapted

Load-bearing premise

The paper assumes that compressing all visible Gaussian tokens into two single alignment tokens (one for image, one for text) preserves enough 3D geometric information for the alignment loss to teach meaningful structural semantics rather than just category-level labels. If the self-attention aggregation into two tokens is too lossy, the cross-modal supervision may only improve coarse classification without genuinely enriching the local 3D representations that downstream part

Editorial extensions

If this is right

  • If cross-modal alignment tokens can inject semantic priors into a Gaussian encoder without any inference-time cost, the same prepended-token alignment design could extend to other 3D representations such as point clouds, voxels, or neural radiance fields, turning any frozen vision-language model into a zero-cost teacher for 3D pre-training.
  • GSHM's salience-guided masking principle could generalize beyond Gaussians to any representation with non-uniform information density, such as adaptive meshes or sparse voxel octrees, where random masking similarly wastes capacity on low-information regions.
  • The finding that ViT image features and CLIP text features are more complementary than using CLIP for both suggests that diversity of teacher feature spaces matters more than alignment of teacher feature spaces for cross-modal 3D supervision.
  • The gains being largest on the hardest ScanObjectNN split and in few-shot settings imply that multimodal pre-training primarily helps robustness and data efficiency rather than ceiling performance on clean data, which would narrow the practical scope of the method to resource-constrained or noisy deployment scenarios.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes GaussFusion, a multimodal self-supervised pre-training framework for 3D Gaussian Splatting (3DGS) representations. The method extends masked Gaussian modeling (as in Gaussian-MAE) by introducing (1) cross-modal semantic alignment, where learnable image and text query tokens aggregate 3D Gaussian features via self-attention and are aligned to frozen ViT and CLIP text encoder outputs, and (2) Gaussian Salience-guided Multi-scale Hole Masking (GSHM), which constructs spatially continuous masked regions based on opacity and scale salience at multiple scales. Experiments on ScanObjectNN, ModelNet40, ShapeNetPart, and few-shot classification show consistent improvements over Gaussian-MAE, with the largest gain on the PB-T50-RS split (+3.85%). The ablation study (Table 5) isolates the contributions of image supervision, text supervision, and GSHM. The central claim—that integrating image and text supervision with salience-guided masking improves transferability of Gaussian representations—is defensible and supported by the experiments.

Significance. The paper addresses a timely problem: extending 3DGS pre-training beyond pure reconstruction to incorporate semantic supervision from frozen vision-language models. The GSHM masking strategy is a reasonable adaptation for the non-uniform distribution of Gaussian primitives, and the ablation in Table 5 provides useful isolation of each component. The use of learnable query tokens (rather than direct feature injection) for cross-modal alignment is a clean design choice that allows the Gaussian encoder to operate without multimodal inputs at inference time. The experimental gains over Gaussian-MAE are consistent across tasks. However, the comparison is limited to a single direct baseline, and the magnitude of gains on fine-grained tasks (part segmentation: +0.78 mIoU_C) is much smaller than on classification (+3.85% on PB-T50-RS), which raises questions about what is being learned that the paper does not fully address.

major comments (2)
  1. Section 4.1 states that text descriptions are 'composed of category names and templated attribute phrases.' The downstream tasks (ModelNet40, ScanObjectNN) are category classification tasks. If ShapeSplat's object categories overlap with these downstream categories, the text branch is effectively providing category-level supervision during pre-training that directly benefits classification. The paper does not report category overlap between ShapeSplat and the downstream datasets, nor does it include an ablation with non-category text (e.g., attribute-only descriptions). This is load-bearing for the central claim that cross-modal alignment improves 'transferable semantic structure,' because the gains may partly reflect category-label transfer rather than genuine representation learning. An analysis of category overlap and/or an ablation with attribute-only text would substantially clarify
  2. The direct comparison is limited to Gaussian-MAE as the sole Gaussian-representation baseline. While the paper acknowledges a performance gap with point cloud methods (Section 4.2.1), the absence of comparison with other cross-modal 3D methods (e.g., ACT, ReCon, ULIP) adapted to the Gaussian setting, or with SceneSplat, makes it difficult to assess whether the gains come from the specific design choices in Section 3.2 or simply from adding any external semantic supervision. At minimum, the paper should discuss why these baselines were not included and whether the gains are expected to hold relative to them.
minor comments (6)
  1. Section 4.1: The paper does not specify which viewpoints are used for rendering images during pre-training (16 viewpoints are mentioned but not how many are sampled per object per epoch). Clarifying this would aid reproducibility.
  2. Table 3: The improvement on part segmentation is modest (+0.78 mIoU_C) and GaussFusion underperforms Gaussian-MAE on several categories (bag, car, e-phone, laptop, pistol, table). The paper briefly attributes this to 'simple categories' but does not explain which categories are affected or why.
  3. References [16]–[21], [26] appear to be by overlapping author groups and several are dated 2025–2026. These should be verified for availability and properly contextualized, especially if they are concurrent work.
  4. Equation (3): The salience score uses the geometric mean of scale parameters, but it is unclear whether scales are in the original or log space. This should be stated explicitly.
  5. Figure 2: The caption mentions 'frozen image and text encoders' but the figure itself does not clearly indicate which components are frozen vs. trainable. Adding freeze symbols would improve clarity.
  6. Section 3.3, Eq. (17): The reconstruction loss is listed as a sum of five terms, then Eq. (18) rewrites it as L_xyz + L_attr. This is slightly redundant; consider merging.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive feedback. Both major comments raise legitimate concerns about the strength of the evidence supporting our central claim. Below we address each point.

read point-by-point responses

  1. Referee: Category overlap between ShapeSplat and downstream datasets may mean the text branch provides category-level supervision that inflates classification gains. Requests category overlap analysis and/or ablation with attribute-only text.

    Authors: The referee raises a valid and important concern. We acknowledge that ShapeSplat's object categories (drawn from ShapeNetCore and other sources) do overlap with categories in ModelNet40, ScanObjectNN, and ShapeNetPart — for example, chairs, tables, airplanes, and lamps appear in both pre-training and downstream data. Because our text descriptions include category names, the text alignment branch does provide some degree of category-level semantic supervision during pre-training, and it is possible that part of the classification gains reflects category-label transfer rather than purely generalizable representation learning. We agree that the current manuscript does not adequately address this confound. To clarify the contribution of the text branch, we will add the following in the revision: (1) A quantitative analysis of category overlap between ShapeSplat and each downstream dataset, reported in the pre-training setup section. (2) An ablation experiment using attribute-only text descriptions (i.e., templated attribute phrases without category names) as the text supervision target, compared against the current category+attribute text. This will directly isolate whether category-name transfer is the primary driver of the text branch's gains. We note that even if some gains are attributable to category-level transfer, the image-only ablation row in Table 5 (85.89% vs. 84.30% baseline) shows that non-category visual supervision also contributes, and the part segmentation gains (+0.78 mIoU_C), while smaller, are consistent with the text branch providing structural rather than purely categorical information. Nevertheless, we agree the attribute-only ablation is necessary to make the central claim precise, and we will include it. revision: yes

  2. Referee: Direct comparison limited to Gaussian-MAE; no comparison with other cross-modal 3D methods (ACT, ReCon, ULIP) adapted to the Gaussian setting, or with SceneSplat.

    Authors: We agree that the comparison is narrower than ideal and that additional baselines would strengthen the paper. We offer the following clarifications and commit to partial revisions. Regarding SceneSplat: SceneSplat operates at the scene level and uses a different pre-training data regime (scene-level Gaussian splats rather than object-level). A direct comparison on object-level downstream tasks (ScanObjectNN, ModelNet40, ShapeNetPart) would require retraining SceneSplat on object-level ShapeSplat data, which is not its intended setting. We will add a discussion of this distinction and clarify why a direct comparison is not straightforward. Regarding ACT, ReCon, and ULIP: these methods are designed for point cloud inputs and rely on point-cloud-specific architectures (e.g., Point-MAE or Point-BERT backbones) and point-cloud-specific cross-modal alignment objectives. Adapting them to the Gaussian setting would require non-trivial modifications: replacing the point cloud encoder with a Gaussian encoder, redefining the masking and reconstruction targets for Gaussian attributes, and re-engineering the cross-modal alignment to operate over Gaussian tokens rather than point patches. This adaptation is itself a research question rather than a straightforward baseline reproduction. That said, we recognize that the absence of such comparisons limits the reader's ability to assess whether our specific design choices (learnable query tokens, GSHM) are necessary or whether any cross-modal supervision would suffice. To partially address this, we will: (1) Add a discussion in the experiments section explaining why these baselines were not directly included and what modifications would be required. (2) If feasible within the revision period, implement a simplified cross-modal baseline — revision: partial

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the derivation chain uses external frozen models and standard benchmarks; self-citations are not load-bearing for the central claim.

full rationale

The paper's central claim is that integrating image and text supervision into masked Gaussian modeling via cross-modal semantic alignment, combined with GSHM masking, improves transferability on downstream tasks. The derivation chain is self-contained against external benchmarks: (1) The cross-modal alignment (Eqs. 9-20) uses frozen external encoders (ViT [15], CLIP [14]) as supervision targets, not self-cited models. (2) The GSHM masking strategy (Eqs. 3-8) computes salience directly from Gaussian attributes (opacity, scale) without fitting parameters to downstream targets. (3) Evaluation is on standard external datasets (ScanObjectNN, ModelNet40, ShapeNetPart) against an external baseline (Gaussian-MAE [12]). (4) While the paper cites several self-references ([16]-[21], [23], [26], [34], [35]), none are load-bearing for the central methodological derivation or the uniqueness of the approach. The self-citations appear in related work and reference baselines, not as premises that define the conclusion. The skeptic's concern about category overlap between ShapeSplat and downstream datasets is a validity/generalization concern, not a circularity issue — the text descriptions used during pre-training are not the downstream labels themselves, and the improvement is measured on held-out downstream tasks. No step in the derivation chain reduces to its own inputs by construction.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities or postulated objects. The free parameters are standard hyperparameters tuned via ablation. The axioms are domain assumptions about the relevance of 2D semantics and Gaussian attributes to 3D representation learning.

free parameters (6)
  • mask_ratio (rho) = 0.6
    Chosen based on ablation in Fig 6(a) showing best performance at 0.6.
  • salience_mixing_coefficient = 0.7
    Controls the mixture between salience distribution and uniform distribution for sampling weights.
  • multi-scale_jitter_coefficient (gamma) = 0.35
    Controls the degree of scale variation for hole mask quotas.
  • lambda_img = 1.0
    Weight for image alignment loss, tuned via ablation in Fig 6(b).
  • lambda_text = 1.0
    Weight for text alignment loss, tuned via ablation in Fig 6(b).
  • number_of_hole_centers (H) = 4
    Number of spatial masking regions per sample.
assumptions (3)
  • domain assumption Gaussian opacity and scale are valid proxies for visual salience and structural importance.
    Used in Section 3.1, Eq. 3 to compute salience scores for masking. Assumes these attributes correlate with semantic importance.
  • domain assumption Frozen 2D image and text features provide useful semantic supervision for 3D Gaussian representations.
    Underlying premise of Section 3.2, assuming that aligning 3D features with ViT and CLIP outputs improves 3D transferability.
  • domain assumption Learnable query tokens can effectively aggregate 3D contextual information for cross-modal alignment.
    Assumed in Section 3.2, Eq. 9-13, where two tokens collect information from visible Gaussian tokens via self-attention.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GaussFusion: Towards Multimodal 3D Gaussian Pretraining." pith.science (2026). https://pith.science/paper/GLVG73GJ

@misc{pith2026260705906,
  author       = {Pith},
  title        = {Pith review of: GaussFusion: Towards Multimodal 3D Gaussian Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GLVG73GJ}},
  note         = {Machine review of arXiv:2607.05906}
}
read the original abstract

3D Gaussian Splatting provides an explicit representation that jointly models geometry and appearance, serving as a scalable foundation for 3D representation learning. Existing pre-training methods for Gaussian representations, such as masked Gaussian reconstruction, primarily capture local structures but offer limited semantic supervision. In this paper, we propose GaussFusion, a multimodal pre-training framework for 3D Gaussian representations. GaussFusion integrates image and text supervision into masked Gaussian modeling through cross-modal semantic alignment, enabling the Gaussian encoder to learn both visual and language-level semantic information during pre-training. To better adapt masked modeling to the non-uniform distribution of Gaussian primitives, we further propose Gaussian Salience-guided Multi-scale Hole Masking (GSHM). GSHM constructs spatially continuous masked regions based on Gaussian salience. By applying hole masks at multiple scales, GSHM encourages the encoder to capture both fine-grained local patterns and broader structural dependencies. Extensive experiments on downstream tasks demonstrate that GaussFusion improves the transferability of Gaussian representations. Notably, GaussFusion outperforms Gaussian-MAE on ModelNet40 and ScanObjectNN (PB-T50-RS) by 0.61\% and 3.85\%, respectively.

Figures

Figures reproduced from arXiv: 2607.05906 by the authors.

Figure 1
Figure 1. Motivation and overview of GaussFusion. Compared with basic masked Gaussian modeling, GaussFusion introduces image-text semantic alignment and GSHM masking to select salient, spatially co￾herent Gaussian regions, improving local structure learning and semantic transferability. ACT[10], and ReCon[11] leverage cross-modal information to improve the semantic expressiveness of point cloud features. These studies show th… view at source ↗
Figure 2
Figure 2. Overall framework of GaussFusion. GaussFusion generates Gaussian tokens from 3D splats and applies Gaussian Salience-guided Multi-scale Hole Masking (GSHM) to local Gaussian groups. Visible Gaussian tokens are combined with learnable image and text tokens in a multimodal encoder. The decoder reconstructs masked Gaussian groups, while frozen image and text encoders provide cross-modal supervi￾sion for learning transf… view at source ↗
Figure 3
Figure 3. Overview of the proposed GSHM strategy. Gauss￾Fusion builds local patches from 3D Gaussian splats, computes salience scores for group centers, and allocates mask quotas by the salience distribution. Selected centers form multi-scale hole regions, where masked patches are replaced by mask tokens and the remaining patches serve as visible tokens for reconstruction. In masked pre-training for 3D Gaussian representation… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of feature distributions. The t-SNE plots compare the feature embeddings of Gaussian-MAE and GaussFusion after pre-training and after fine-tuning on ModelNet40 and ScanOb￾jectNN. Different colors represent different object categories. ing regions for each…
Figure 5
Figure 5. Figure 5: Qualitative comparison of part segmentation results on ShapeNetPart. Different colors rep￾resent predicted semantic parts. Compared with Gaussian-MAE, GaussFusion produces more spatially coherent predictions and clearer part boundaries, particularly for thin structures…
Figure 6
Figure 6. Figure 6: Ablation study on masking ratio and multimodal loss weights. (a) shows the effect of GSHM masking ratios on ScanObjectNN OBJ-BG. (b) shows the effect of image and text loss weights, with the best accuracy of 88.98% at λimg = λtext = 1. that the image and text encoders …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Parameter-Efficient Fine-Tuning for Spiking Point Cloud Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SpikePEFT matches or beats full fine-tuning on spiking point-cloud benchmarks while updating only ~5% of parameters via neuron-dynamics and silent-state adapters.

Reference graph

Works this paper leans on

53 extracted references · 53 canonical work pages · cited by 1 Pith paper

  1. [1]

    ACM Transactions on Graphics42(4), 1–14 (Aug 2023).https://doi.org/10.1145/3592433

    B. Kerbl, G. Kopanas, T. Leimkühler, G. Drettakis, 3d gaussian splatting for real-time radiance field rendering, ACM Transactions on Graphics 42 (4) (2023) 139:1–139:14. doi:10.1145/3592433

    work page doi:10.1145/3592433 2023

  2. [2]

    C. R. Qi, H. Su, K. Mo, L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660. 25

    work page 2017

  3. [3]

    C. R. Qi, L. Yi, H. Su, L. J. Guibas, Pointnet++: Deep hierarchical feature learning on point sets in a metric space, in: Advances in Neural Information Processing Systems, V ol. 30, 2017

    work page 2017

  4. [4]

    J. L. Schönberger, J.-M. Frahm, Structure-from-motion revisited, in: Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4104–4113

    work page 2016

  5. [5]

    Mildenhall, P

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, R. Ng, NeRF: Representing scenes as neural radiance fields for view synthesis, in: Eu- ropean Conference on Computer Vision (ECCV), Springer, 2020, pp. 405–421

    work page 2020

  6. [6]

    Müller, A

    T. Müller, A. Evans, C. Schied, A. Keller, Instant neural graphics primitives with a multiresolution hash encoding, ACM Transactions on Graphics 41 (4) (2022) 102:1–102:15

    work page 2022

  7. [7]

    X. Yu, L. Tang, Y . Rao, T. Huang, J. Zhou, J. Lu, Point-bert: Pre-training 3d point cloud transformers with masked point modeling, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 19313–19322

    work page 2022

  8. [8]

    Y . Pang, W. Wang, F. E. H. Tay, W. Liu, Y . Tian, L. Yuan, Masked autoencoders for point cloud self-supervised learning, in: European Conference on Computer Vision (ECCV), Springer, 2022, pp. 604–621

    work page 2022

Show all 53 references
  1. [9]

    Afham, I

    M. Afham, I. Dissanayake, D. Dissanayake, A. Dharmasiri, K. Thilakarathna, R. Rodrigo, Crosspoint: Self-supervised cross-modal contrastive learning for 3d point cloud understanding, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp...

    2022

  2. [10]

    R. Dong, Z. Qi, L. Zhang, J. Zhang, J. Sun, Z. Ge, L. Yi, K. Ma, Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d repre- sentation learning?, in: International Conference on Learning Representations (ICLR), 2023. 26

    2023

  3. [11]

    Z. Qi, R. Dong, G. Fan, Z. Ge, X. Zhang, K. Ma, L. Yi, Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining, in: Pro- ceedings of the 40th International Conference on Machine Learning, V ol. 202 of Proceedings of Machine Learning R...

    2023

  4. [12]

    Q. Ma, Y . Li, B. Ren, N. Sebe, E. Konukoglu, T. Gevers, L. Van Gool, D. P. Paudel, A large-scale dataset of gaussian splats and their self-supervised pre- training, in: 2025 International Conference on 3D Vision (3DV), IEEE, 2025, pp. 145–155. doi:10.1109/3DV66043.2025.00019

    2025 doi

  5. [13]

    Y . Li, Q. Ma, R. Yang, H. Li, M. Ma, B. Ren, N. Popovic, N. Sebe, E. Konukoglu, T. Gevers, L. Van Gool, M. R. Oswald, D. P. Paudel, Scenesplat: Gaussian splatting-based scene understanding with vision-language pretraining, in: Pro- ceedings of the IEEE/CVF International Confe...

    2025

  6. [14]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: Proceedings of the 38th International Conference on Machine Learni...

    2021

  7. [15]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Un- terthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby, An image is worth 16x16 words: Transformers for image recogni- tion at scale, in: International Conference on Learning ...

    2021

  8. [16]

    Y . Wang, Y . Sun, Q. Wang, P. Li, C. Lu, D. Zhang, Pointrft: Explicit reinforce- ment fine-tuning for point cloud few-shot learning, in: IEEE International Con- ference on Multimedia and Expo (ICME2026), 2026

    2026

  9. [17]

    Zhang, Y

    D. Zhang, Y . Wang, Y . Sun, H. Xu, P. Fan, J. Zhu, Cmhanet: A cross-modal hybrid attention network for point cloud registration, Neurocomputing (2026). 27

    2026

  10. [18]

    X. Han, Y . Sun, C. Lu, Rethinking regressor in 3d gaussian pretraining, in: Pat- tern Recognit. Comput. Vis., 2026, pp. 177–190

    2026

  11. [19]

    Y . Sun, J. Zhu, H. Cheng, C. Lu, Z. Yang, L. Chen, Y . Wang, Align then adapt: Rethinking parameter-efficient transfer learning in 4d perception, IEEE Trans. Multimedia (2026)

    2026

  12. [20]

    Y . Sun, H. Cheng, C. Lu, Z. Li, M. Wu, H. Lu, J. Zhu, Hyperpoint: Multimodal 3d foundation model in hyperbolic space, Pattern Recognit. 173 (2026) 112800

    2026

  13. [21]

    P. Li, Y . Sun, H. Cheng, Pointdico: Contrastive 3d representation learning guided by diffusion models, arXiv preprint arXiv:2512.08330 (2025)

    2025

  14. [22]

    Zhang, Z

    R. Zhang, Z. Guo, P. Gao, R. Fang, B. Zhao, D. Wang, Y . Qiao, H. Li, Point-m2ae: Multi-scale masked autoencoders for hierarchical point cloud pre- training, in: Advances in Neural Information Processing Systems, V ol. 35, 2022, pp. 27061–27074

    2022

  15. [23]

    Cheng, J

    H. Cheng, J. Zhu, N. Hu, J. Chen, W. Yan, PTM: Torus masking for 3d rep- resentation learning guided by robust and trusted teachers, IEEE Transactions on Circuits and Systems for Video Technology 34 (12) (2024) 12158–12170. doi:10.1109/TCSVT.2024.3430904

    2024 doi

  16. [24]

    J. Zhou, Y . Song, C. Chiu, Y . Xiong, Y . Luo, S. Song, CPG: Contrastive patch- graph learning for 3d point cloud, Pattern Recognition 169 (2026) 111954. doi:10.1016/j.patcog.2025.111954

    2026 doi

  17. [25]

    Liang, X

    D. Liang, X. Zhou, W. Xu, X. Zhu, Z. Zou, X. Ye, X. Tan, X. Bai, Point- mamba: A simple state space model for point cloud analysis, in: Advances in Neural Information Processing Systems, V ol. 37, 2024, pp. 32653–32677. doi:10.52202/079017-1026

    2024 doi

  18. [26]

    Z. Guo, J. Zhu, J. Liu, A. S. Mian, Mantis: Mamba-native tuning is efficient for 3d point cloud foundation models, arXiv preprint arXiv:2605.03438 (2026). doi:10.48550/arXiv.2605.03438. 28

    2026 doi arXiv

  19. [27]

    Guédon, V

    A. Guédon, V . Lepetit, SuGaR: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 5354–5363

    2024

  20. [28]

    J. C. Lee, D. Rho, X. Sun, J. H. Ko, E. Park, Compact 3d gaussian representation for radiance field, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21719–21728

    2024

  21. [29]

    S. Zhou, H. Chang, S. Jiang, Z. Fan, Z. Zhu, D. Xu, P. Chari, S. You, Z. Wang, A. Kadambi, Feature 3dgs: Supercharging 3d gaussian splatting to enable dis- tilled feature fields, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 2...

    2024

  22. [30]

    M. Qin, W. Li, J. Zhou, H. Wang, H. Pfister, Langsplat: 3d language gaussian splatting, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20051–20060

    2024

  23. [31]

    X. Zhu, R. Zhang, B. He, Z. Guo, Z. Zeng, Z. Qin, S. Zhang, P. Gao, Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 2639–2650

    2023

  24. [32]

    Huang, X

    R. Huang, X. Pan, H. Zheng, H. Jiang, Z. Xie, C. Wu, S. Song, G. Huang, Joint representation learning for text and 3d point cloud, Pattern Recognition 147 (2024) 110086. doi:10.1016/j.patcog.2023.110086

    2024 doi

  25. [33]

    Zhang, L

    H. Zhang, L. Yu, G. Wang, S. Tian, Z. Yu, W. Li, X. Ning, Cross-modal knowl- edge transfer for 3d point clouds via graph offset prediction, Pattern Recognition 162 (2025) 111351. doi:10.1016/j.patcog.2025.111351

    2025 doi

  26. [34]

    Cheng, X

    H. Cheng, X. Han, P. Shi, J. Zhu, Z. Li, Multi-trusted cross-modal informa- tion bottleneck for 3d self-supervised representation learning, Knowledge-Based Systems 283 (2024) 111217. doi:10.1016/j.knosys.2023.111217. 29

    2024 doi

  27. [35]

    N. Hu, H. Cheng, Y . Xie, P. Shi, J. Zhu, Hyperbolic image-and-pointcloud con- trastive learning for 3d classification, in: 2024 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), IEEE, 2024, pp. 4973–4979. doi:10.1109/IROS58592.2024.10802543

    2024 doi

  28. [36]

    L. Xue, M. Gao, C. Xing, R. Martín-Martín, J. Wu, C. Xiong, R. Xu, J. C. Niebles, S. Savarese, Ulip: Learning a unified representation of language, im- ages, and point clouds for 3d understanding, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recogn...

    2023

  29. [37]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in Neural Infor- mation Processing Systems, V ol. 30, 2017, pp. 5998–6008

    2017

  30. [38]

    M. A. Uy, Q.-H. Pham, B.-S. Hua, T. Nguyen, S.-K. Yeung, Revisiting point cloud classification: A new benchmark dataset and classification model on real- world data, in: Proceedings of the IEEE/CVF International Conference on Com- puter Vision (ICCV), 2019, pp. 1588–1597

    2019

  31. [39]

    Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, J. Xiao, 3d shapenets: A deep representation for volumetric shapes, in: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1912– 1920

    2015

  32. [40]

    L. Yi, V . G. Kim, D. Ceylan, I.-C. Shen, M. Yan, H. Su, C. Lu, Q. Huang, A. Sheffer, L. Guibas, A scalable active framework for region annotation in 3d shape collections, ACM Transactions on Graphics 35 (6) (2016) 210:1–210:12

    2016

  33. [41]

    Y . Xu, T. Fan, M. Xu, L. Zeng, Y . Qiao, Spidercnn: Deep learning on point sets with parameterized convolutional filters, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 87–102

    2018

  34. [42]

    Y . Wang, Y . Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, J. M. Solomon, Dynamic graph cnn for learning on point clouds, ACM Transactions on Graphics (tog) 38 (5) (2019) 146:1–146:12. doi:10.1145/3326362. 30

    2019 doi

  35. [43]

    Y . Li, R. Bu, M. Sun, W. Wu, X. Di, B. Chen, Pointcnn: Convolution on x- transformed points, in: Advances in Neural Information Processing Systems, V ol. 31, 2018

    2018

  36. [44]

    X. Han, Y . Tang, Z. Wang, X. Li, Mamba3d: Enhancing local features for 3d point cloud analysis via state space model, in: Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 4995–5004. doi:10.1145/3664647.3681173

    2024 doi

  37. [45]

    Atzmon, H

    M. Atzmon, H. Maron, Y . Lipman, Point convolutional neural networks by ex- tension operators, ACM Transactions on Graphics 37 (4) (2018) 71:1–71:12

    2018

  38. [46]

    Komarichev, Z

    A. Komarichev, Z. Zhong, J. Hua, A-cnn: Annularly convolutional neural net- works on point clouds, in: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2019, pp. 7421–7430

    2019

  39. [47]

    X. Yan, C. Zheng, Z. Li, S. Wang, S. Cui, Pointasnl: Robust point clouds pro- cessing using nonlocal neural networks with adaptive sampling, in: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 5589–5598

    2020

  40. [48]

    Q. Wu, Q. Zhang, C. Tan, Y . Zhou, C. Sun, Point-to-spike residual learn- ing for energy-efficient 3d point cloud classification, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 38, 2024, pp. 6092–6099. doi:10.1609/aaai.v38i6.28425

    2024 doi

  41. [49]

    Zhang, H

    T. Zhang, H. Yuan, L. Qi, J. Zhang, Q. Zhou, S. Ji, S. Yan, X. Li, Point cloud mamba: Point cloud learning via state space model, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39, 2025, pp. 10121–10130. doi:10.1609/aaai.v39i10.33098

    2025 doi

  42. [50]

    H. Wang, Q. Liu, X. Yue, J. Lasenby, M. J. Kusner, Unsupervised point cloud pre-training via occlusion completion, in: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), 2021, pp. 9782–9792. 31

    2021

  43. [51]

    H. Bao, L. Dong, S. Piao, F. Wei, BEiT: BERT pre-training of image transform- ers, in: International Conference on Learning Representations (ICLR), 2022

    2022

  44. [52]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2016, pp. 770–778

    2016

  45. [53]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, B. Guo, Swin trans- former: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 10012–10022. 32

    2021

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.