Pith. sign in

REVIEW 2 major objections 5 minor 5 cited by

The paper claims that a single self-supervised point-cloud encoder can be trained jointly on indoor, outdoor, and object data, producing one shared 3D representation that transfers across all of them.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 19:06 UTC pith:XEZNWJIM

load-bearing objection A solid multi-domain pretraining recipe; the 'unified representation space' claim is under-supported but the work merits serious review. the 2 major comments →

arxiv 2603.03283 v2 pith:XEZNWJIM submitted 2026-03-03 cs.CV

Utonia: Toward One Encoder for All Point Clouds

Yujia Zhang , Xiaoyang Wu , Yunhan Yang , Xianzhe Fan , Han Li , Yuechen Zhang , Zehao Huang , Naiyan Wang
show 1 more author
Hengshuang Zhao
This is my paper
classification cs.CV
keywords point cloudself-supervised learningcross-domain pretrainingfoundation modelPoint Transformer V3rotary position embeddingmodality dropoutgranularity rescaling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that point-cloud self-supervised learning remains fragmented by domain not because of the structure of the physical world, but because of a few fixable mismatches: differing coordinate scales, gravity conventions, and module availability. It proposes three domain-agnostic fixes—randomly dropping color/normal channels, rescaling every cloud to a shared perceptual granularity, and applying rotary positional embeddings to granularity-aligned coordinates—so that one Point Transformer V3 encoder can train stably on 250k mixed point clouds plus 1M CAD assets. The result is a unified representation space that transfers to indoor, outdoor, and object benchmarks, and whose features also improve robotic manipulation and spatial reasoning when fed into action and vision-language models. If correct, this points toward a single shared 3D foundation encoder rather than separate models per sensor type.

Core claim

The paper establishes that a single self-supervised point transformer can be pretrained jointly on remote sensing, outdoor LiDAR, indoor RGB-D sequences, object CAD, and video-lifted point clouds, yielding a unified representation space that transfers across domains. The enabling designs are Causal Modality Blinding (randomly masking color and normal channels so the encoder cannot depend on their availability), Perceptual Granularity Rescale (rescaling coordinates so all clouds share a comparable spatial unit before positional encoding), and RoPE on granularity-aligned coordinates (a continuous relative-geometry positional hint in every attention layer). With these, joint pretraining matches

What carries the argument

Three mechanisms carry the argument. Causal Modality Blinding randomly drops entire modality groups or individual points' color/normal channels during pretraining, so the encoder treats auxiliary channels as optional and remains usable when they are absent. Perceptual Granularity Rescale maps each input cloud to a shared 'observing granularity' before positional encoding, while preserving gravity alignment for scenes and applying full SO(3) rotations and ±50% scale jitter for objects. The third piece is 3D rotary positional embedding (RoPE): attaching a parameter-free continuous position signal, computed from granularity-aligned and anisotropically jittered coordinates, to attention queries

Load-bearing premise

The transfer claim rests on knowing each sample's domain so the correct rescale factor and rotation augmentation policy can be chosen; for a sample whose domain is unknown or genuinely novel, there is no principled way to set them.

What would settle it

Compare a model pretrained with object-level full SO(3) rotations against one pretrained with only mild scene-level rotations: the paper's own Table 7a shows ScanObjectNN dropping from 66.9 to 63.0, confirming the policy matters. A sharper test is to feed the encoder a point cloud from an unseen sensor with an unknown scale convention and check whether features remain coherent or cluster by scale.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Joint pretraining across point-cloud domains becomes stable and produces a unified representation space instead of representations that cluster by domain.
  • The same encoder matches or beats separately trained domain-specific models on indoor and outdoor segmentation, object classification, and part segmentation, particularly under full fine-tuning.
  • The encoder remains robust when colors or normals are missing, whereas prior methods degrade sharply when an auxiliary modality is absent.
  • Features transfer beyond perception: conditioning vision-language-action policies with Utonia features improves robotic manipulation success, and fusing them into vision-language models improves spatial grounding and question answering.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The unification is conditional: the recipe assumes the practitioner knows a sample's domain so the right rescale factor and rotation policy can be applied, so a genuinely novel domain with unknown scale or gravity conventions would require guessing those parameters.
  • The gap between linear probing and full fine-tuning on part segmentation suggests the representation carries part-level cues that are present but not linearly decodable; task-conditioned query decoders or global register tokens could make them accessible without fine-tuning.
  • The scaling results imply data diversity and scale matter more than architectural novelty for cross-domain transfer, so even larger and noisier multi-domain mixtures may continue to improve transfer at the cost of fine-grained linear readouts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. Utonia proposes a self-supervised pretraining pipeline for a single Point Transformer V3 encoder jointly trained on indoor, outdoor, object, remote-sensing, and video-lifted point clouds. The recipe has three components: Causal Modality Blinding (randomly dropping colors/normals), Perceptual Granularity Rescale (rescaling coordinates to a shared granularity, with per-domain rotation/scale policy), and RoPE on granularity-aligned coordinates. Evaluation covers linear probing, decoder probing, and full fine-tuning on indoor/outdoor segmentation, object classification and part segmentation, plus robotic manipulation, open-world part segmentation, and VLM spatial reasoning. The paper reports that joint pretraining is stable and generally matches or improves on single-domain baselines, and interprets this as evidence of a unified cross-domain representation space.

Significance. Timely and potentially influential: it tackles domain-fragmented point-cloud SSL with an explicit recipe and unusually broad evaluation across indoor, outdoor, object, robotic, and VLM settings. The paper is honest about baseline comparability (Tab. 4 notes Sonata uses PPT/extra data; Sec. 4 repeats this) and includes useful ablations (Tabs. 2, 7a–f). I do not see a circularity problem in the technical sense: pretraining is self-supervised and downstream numbers are on held-out/standard evaluation splits. The main gap is that the 'unified representation space' claim is stronger than the evidence; most experiments show that a single encoder can be probed or fine-tuned per domain to good accuracy, but that does not by itself establish cross-domain feature alignment.

major comments (2)
  1. [Abstract; Sec. 4; Tabs. 3–6] The central claim that Utonia learns a 'unified representation space' (Abstract; Sec. 4 intro) is not directly tested. All quantitative evaluations train per-domain readouts on target-domain labels—linear probes, decoder probes, or fine-tuning—which only establish per-domain linear/decoder separability and can succeed even if features are domain-segregated. The only direct evidence for cross-domain alignment is qualitative (Figs. 1–2). Please add a quantitative alignment test: a shared linear head on a common label space, cross-domain retrieval on known correspondences (e.g., the toy-car/real-car example), or a domain-confusion/CCA-style metric. Without this, the one-encoder-for-all claim is underdetermined by the reported experiments.
  2. [Sec. 3.2; Tab. 7a] The reported gains are conditional on domain-specific priors. Pretraining uses different scale and rotation budgets per domain (objects: ±50% scale and full SO(3); scenes: ±10% scale and yaw-only rotations), and inference requires choosing the rescale factor for the target granularity (Sec. 3.2). Tab. 7a shows this is not neutral: reducing object rotations to scene-level settings drops ScanObjectNN from 66.9 to 63.0. The paper should state this conditionality explicitly as a limitation, or test robustness when domain labels/priors are unknown or misspecified. Otherwise the 'domain-agnostic designs' wording in the Introduction overstates the method.
minor comments (5)
  1. [Tab. 6 caption] The caption reads 'dropping colors or colors from input data'; the second term should be 'normals'.
  2. [Tab. 2] 'Origin grid size' should be 'Original grid size'.
  3. [Sec. 2 / Sec. 3.2] The 'standard observing granularity' concept is used as an axiom but never formally defined. Please specify how the target granularity and per-domain rescale ranges are selected, since they are among the most important free parameters of the recipe.
  4. [Sec. 4.2; Tabs. 9–10] The robotics and spatial-reasoning results are reported for one configuration each and without variance estimates or multiple seeds. These results are useful as preliminary evidence, but they should be labeled as such or accompanied by at least seed variance and, ideally, release of checkpoints so the community can reproduce them.
  5. [A.1] The sentence 'The only difference in training parameters between Utonia and Concerto/Sonata is that the upcast level...' is misleading because the data mixture, number of stages, and data balance also differ. Please clarify whether this statement refers only to optimization hyperparameters.

Circularity Check

0 steps flagged

No significant circularity: Utonia's claims are empirical and benchmarked on held-out data; the unified-space claim is under-tested but not definitionally forced.

full rationale

Utonia is an empirical pretraining recipe rather than a derivation, so there is no equation-level reduction in which a predicted quantity equals a fitted input by construction. The teacher-student self-distillation objective follows prior work (Sonata/Concerto), but the downstream evaluations use held-out validation/test benchmarks with linear probing, decoder probing, and fine-tuning; the pretraining objective does not use downstream labels, so the reported gains in Tabs. 3-6 are not statistically forced by the method's own construction. The self-citations to Sonata and Concerto are to published, externally evaluated prior work and serve as baselines/recipes, not as an unverified uniqueness theorem that selects the answer. The central 'unified representation space' claim is indeed supported mainly by qualitative evidence (Figs. 1-2, 5-6), and the method relies on per-domain rescale/rotation policies plus a few hyperparameters selected with downstream benchmarks (e.g., ScanObjectNN in Tab. 7a). Those are evidentiary/overfitting concerns about how strongly the transfer claim is established, not circularity: they do not make the final benchmark numbers equal to the inputs or reduce the derivation to a fit. The paper is therefore self-contained in the sense relevant to this circularity analysis, and no load-bearing circular step is present.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 1 invented entities

The central claim rests on a modest set of hand-picked hyperparameters (grid size, per-domain scale/rotation ranges, RoPE base, blinding placement) and on the assumption that a canonical perceptual granularity exists. No new physical entities are introduced; the only invented conceptual object is the 'standard observing granularity.' Most ingredients come from prior SSL and positional-encoding literature.

free parameters (7)
  • Global voxel grid size after perceptual rescaling
    Central to Perceptual Granularity Rescale; fixed grid size chosen via pilot comparison (Tab. 2), exact value not stated in text.
  • Per-domain coordinate rescale ranges = scenes ±10%, objects ±50%
    Domain-specific scale augmentation ranges in Sec. 3.2 / App. A.2; ablation in Tab. 7a shows sensitivity.
  • Per-domain rotation policies = scenes yaw [−π, π], x/y ∈ [−π/64, π/64]; objects full SO(3)
    Gravity-prior erasure policy; ablation Tab. 7a shows reducing object rotations degrades ScanObjectNN.
  • RoPE base B = 10
    Default selected via Tab. 7c; performance stable across B=1..1000.
  • Modality blinding dropout scheme and probability
    Dropout at data loading chosen via Tab. 7d; exact probabilities not stated.
  • Self-distillation upcast level = 0
    Set to 0 for speed; authors claim negligible impact on downstream performance.
  • Pretraining schedule = Stage 1 on curated datasets, Stage 2 100 epochs, batch size 256
    Two-stage schedule adopted to stabilize optimization across domains; chosen empirically.
axioms (6)
  • domain assumption Teacher-student self-distillation recipe from Sonata/Concerto yields reliable pretraining representations.
    Utonia inherits the SSL recipe without re-deriving it (Sec. 3); if this recipe fails on broad mixtures, the central result changes.
  • ad hoc to paper A fixed minimal angular resolution justifies a canonical perceptual granularity across all point-cloud domains.
    Perceptual Granularity Rescale (Sec. 3.2) rests on this intuition; no independent measurement of such a universal resolution is provided.
  • domain assumption Gravity alignment is a granularity-dependent prior: scenes stay upright, objects are rotation-invariant.
    Drives the domain-specific rotation and scale policies in Sec. 3.2 and Fig. 3; if incorrect, transfer could be worse.
  • domain assumption RoPE applied separably per axis to queries and keys is a valid continuous positional hint for point attention.
    Adopted from 1D RoPE and applied in App. A.2 Eq. 5; its benefit across densities and scales is supported only empirically.
  • domain assumption DINOv3-style coordinate jittering and isotropic scaling improve cross-domain generalization.
    Applied in Sec. 3.3 following [29]; not tested against alternative coordinate augmentation schemes.
  • domain assumption The downstream evaluation datasets and protocols faithfully measure representation quality.
    Standard benchmarks used; single-run results without error bars rely on benchmark stability.
invented entities (1)
  • Standard observing granularity no independent evidence
    purpose: Justifies rescaling all point clouds to a common coordinate scale before positional encoding and voxelization.
    Introduced in Sec. 3.2; has no falsifiable handle outside the reported model improvements.

pith-pipeline@v1.3.0-alltime-deepseek · 19544 in / 12156 out tokens · 117598 ms · 2026-08-02T19:06:44.105436+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Utonia: Toward One Encoder for All Point Clouds." pith.science (2026). https://pith.science/paper/XEZNWJIM

@misc{pith2026260303283,
  author       = {Pith},
  title        = {Pith review of: Utonia: Toward One Encoder for All Point Clouds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XEZNWJIM}},
  note         = {Machine review of arXiv:2603.03283}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We dream of a future where point clouds from all domains can come together to shape a single model that benefits them all. Toward this goal, we present Utonia, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos. Despite their distinct sensing geometries, densities, and priors, Utonia learns a consistent representation space that transfers across domains. This unification improves perception capability while revealing intriguing emergent behaviors that arise only when domains are trained jointly. Beyond perception, we observe that Utonia representations can also benefit embodied and multimodal reasoning: conditioning vision-language-action policies on Utonia features improves robotic manipulation, and integrating them into vision-language models yields gains on spatial reasoning. We hope Utonia can serve as a step toward foundation models for sparse 3D data, and support downstream applications in AR/VR, robotics, and autonomous driving.

Figures

Figures reproduced from arXiv: 2603.03283 by Han Li, Hengshuang Zhao, Naiyan Wang, Xianzhe Fan, Xiaoyang Wu, Yuechen Zhang, Yujia Zhang, Yunhan Yang, Zehao Huang.

Figure 1
Figure 1. Figure 1: One encoder for all point clouds. We visualize Utonia features by PCA. City-scale geometry with large coordinate ranges: the smooth, coherent features over terrain and building masses indicate robustness to an extreme extent; Outdoor LiDAR with sparse ring-like scan patterns: features remain organized by geometry rather than scanlines, suggesting reduced reliance on sampling-specific shortcuts. Indoor reco… view at source ↗
Figure 2
Figure 2. Figure 2: Cross-domain semantic similarity. Human perception operates at a fixed angular resolution, resulting in similar percep￾tion granularity between a close small toy car and a far-away real car, which motivates semantic matching at a canonical granular￾ity across domains. Utonia representations exhibit high similarity between the toy car from object CAD and real cars in outdoor scenes, while the previous SOTA … view at source ↗
Figure 3
Figure 3. Figure 3: Gravity priors influence. Scene-level data have a strong z-axis up prior. Utonia steps further to erase such assumptions by including rotation-invariant objects with strong SE(3) augmentations into pretraining datasets. SSL recipes. PTv3 [42] offers a strong foundation, while Sonata [44] and Concerto [55] mitigate geometric shortcuts via perturbations and cross-modal synergy, making large￾scale training st… view at source ↗
Figure 4
Figure 4. Figure 4: Overview of Utonia. Utonia introduces three critical improvements to the point cloud SSL pipeline. Cross-domain data: jointly training on object-centric, indoor, and outdoor point clouds. RoPE-Enhanced Point Transformer V3: Strengthening spatial encoding and cross-domain transferability via RoPE on granularity-aligned coordinates and domain-prior erasure. Broader evaluation: extending beyond standard perce… view at source ↗
Figure 5
Figure 5. Figure 5: Utonia features in cluttered manipulation scenes. Utonia can separate objects from supporting surfaces and remain coherent under occlusion and partial observations, providing geometry-aware cues that are useful for downstream grasping and motion planning. Sonata PCA Utonia PCA Sonata Seg Utonia Seg [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Open-world object segmentation. Utonia produces more coherent, part-aligned feature structure, leading to cleaner segment boundaries and more consistent semantic parts compared to Sonata. object-level semantics, while fine-grained part cues are bet￾ter recovered through task-specific adaptation. These ob￾servations highlight different readout requirements going forward: global registers or a [CLS]-like tok… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Intrinsic and Triangulation-Agnostic Attention: A Simple and Powerful Approach for Learning on Meshes

    cs.GR 2026-07 conditional novelty 6.0

    Mass-weighted FEM attention on intrinsic mesh features is triangulation-agnostic and beats current mesh and point-cloud baselines on several geometry-learning benchmarks.

  2. PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding

    cs.CV 2026-06 unverdicted novelty 6.0

    PAR3D is a part-aware 3D-MLLM framework with ScenePart dataset, Part-Aware 3D Representation Learning, and Hierarchical Segmentation Query Generation to improve part-level 3D scene understanding.

  3. Coarse Semantic Injection for LLM-Conditioned Structured Indoor Prediction

    cs.CV 2026-05 unverdicted novelty 6.0

    Coarse four-group semantic color coding (RGBB) appended to point clouds before tokenization improves LLM-based structured indoor prediction on Structured3D, SpatialLM, and ARKitScenes, especially for openings and furn...

  4. MAG-VLAQ: Multi-modal Aerial-Ground Query Aggregation for Cross-View Place Recognition

    cs.CV 2026-05 unverdicted novelty 6.0

    MAG-VLAQ fuses multi-modal ground and aerial data via ODE-conditioned vector-of-locally-aggregated-queries to nearly double recall@1 on aerial-ground place recognition benchmarks.

  5. Beyond Point-Attached Semantics: Object-Centric Semantic Fields for Generalizable Manipulation

    cs.RO 2026-07 conditional novelty 5.0

    An object-conditioned continuous semantic field queried at explicit 3D locations yields more stable part cues and higher manipulation success than point-attached 2D/3D features.

Reference graph

Works this paper leans on

64 extracted references · 1 canonical work pages · cited by 5 Pith papers

  1. [1]

    Crosspoint: Self- supervised cross-modal contrastive learning for 3d point cloud understanding

    Afham, M., Dissanayake, I., Dissanayake, D., Dharmasiri, A., Thilakarathna, K., and Rodrigo, R. Crosspoint: Self- supervised cross-modal contrastive learning for 3d point cloud understanding. InCVPR, pp. 9902–9912, June 2022. 9

    2022

  2. [2]

    R., Jiang, H., Brilakis, I., Fischer, M., and Savarese, S

    Armeni, I., Sener, O., Zamir, A. R., Jiang, H., Brilakis, I., Fischer, M., and Savarese, S. 3d semantic parsing of large- scale indoor spaces. InCVPR, 2016. 15

    2016

  3. [3]

    ARKitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data

    Baruch, G., Chen, Z., Dehghan, A., Dimry, T., Feigin, Y ., Fu, P., Gebauer, T., Joffe, B., Kurz, D., Schwartz, A., and Shulman, E. ARKitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. In NeurIPSW, 2021. 2, 15

    2021

  4. [4]

    Semantickitti: A dataset for semantic scene understanding of lidar sequences

    Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., and Gall, J. Semantickitti: A dataset for semantic scene understanding of lidar sequences. InICCV,

  5. [5]

    H., V ora, S., Liong, V

    Caesar, H., Bankiti, V ., Lang, A. H., V ora, S., Liong, V . E., Xu, Q., Krishnan, A., Pan, Y ., Baldan, G., and Beijbom, O. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020. 2, 15

    2020

  6. [6]

    Bodex: Scalable and efficient robotic dexterous grasp synthesis using bilevel optimization

    Chen, J., Ke, Y ., and Wang, H. Bodex: Scalable and efficient robotic dexterous grasp synthesis using bilevel optimization. InICRA, pp. 01–08. IEEE, 2025. 14

    2025

  7. [7]

    X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M

    Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. Scannet: Richly-annotated 3d recon- structions of indoor scenes. InCVPR, 2017. 14, 15

    2017

  8. [8]

    Objaverse: A universe of annotated 3d ob- jects

    Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L., Ehsani, K., Kembhavi, A., and Farhadi, A. Objaverse: A universe of annotated 3d ob- jects. InCVPR, pp. 13142–13153, 2023. 14

    2023

  9. [9]

    Graspvla: a grasping foundation model pre-trained on billion-scale syn- thetic action data.arXiv preprint arXiv:2505.03233, 2025

    Deng, S., Yan, M., Wei, S., Ma, H., Yang, Y ., Chen, J., Zhang, Z., Yang, T., Zhang, X., Zhang, W., et al. Graspvla: a grasping foundation model pre-trained on billion-scale syn- thetic action data.arXiv preprint arXiv:2505.03233, 2025. 8

    Pith/arXiv arXiv 2025

  10. [10]

    Any3d-vla: Enhanc- ing vla robustness via diverse point clouds.arXiv preprint arXiv:2602.00807, 2026

    Fan, X., Deng, S., Wu, X., Lu, Y ., Li, Z., Yan, M., Zhang, Y ., Zhang, Z., Wang, H., and Zhao, H. Any3d-vla: Enhanc- ing vla robustness via diverse point clouds.arXiv preprint arXiv:2602.00807, 2026. 14

    Pith/arXiv arXiv 2026

  11. [11]

    Graspnet- 1billion: A large-scale benchmark for general object grasp- ing

    Fang, H.-S., Wang, C., Gou, M., and Lu, C. Graspnet- 1billion: A large-scale benchmark for general object grasp- ing. InCVPR, pp. 11444–11453, 2020. 15

    2020

  12. [12]

    Maniskill2: A unified benchmark for generalizable manipulation skills

    Gu, J., Xiang, F., Li, X., Ling, Z., Liu, X., Mu, T., Tang, Y ., Tao, S., Wei, X., Yao, Y ., Yuan, X., Xie, P., Huang, Z., Chen, R., and Su, H. Maniskill2: A unified benchmark for generalizable manipulation skills. InICLR, 2023. 2

    2023

  13. [13]

    Rotary position embedding for vision transformer

    Heo, B., Park, S., Han, D., and Yun, S. Rotary position embedding for vision transformer. InEuropean Conference on Computer Vision, pp. 289–305. Springer, 2024. 10

    2024

  14. [14]

    Spatio-temporal self-supervised representation learning for 3d point clouds

    Huang, S., Xie, Y ., Zhu, S.-C., and Zhu, Y . Spatio-temporal self-supervised representation learning for 3d point clouds. ICCV, pp. 6515–6525, 2021. URLhttps://api. semanticscholar.org/CorpusID:237371865. 9

    2021

  15. [15]

    K., Chen, L

    Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M. K., Chen, L. Y ., Ellis, K., et al. Droid: A large-scale in-the-wild robot manipulation dataset. InRobotics: Science and Systems XX, Delft, The Netherlands, July 15-19, 2024, 2024. doi: 10.15607/RSS.2024.XX.120. URLhttps://doi.org/ 10.15607/RSS.20...

    doi:10.15607/rss.2024.xx.120 2024

  16. [16]

    C., Lo, W.-Y ., et al

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y ., et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pp. 4015–4026, 2023. 15

    2023

  17. [17]

    Hong kong 3D map.https://3d

    Lands Department. Hong kong 3D map.https://3d. map.gov.hk/, 2026. Accessed: 2026-01-12. 15

    2026

  18. [18]

    Y ., Li, Z., Shi, G., Feng, J., and Kang, B

    Lin, H., Chen, S., Liew, J., Chen, D. Y ., Li, Z., Shi, G., Feng, J., and Kang, B. Depth anything 3: Recovering the visual space from any views.CoRR, abs/2511.10647, 2025. doi: 10.48550/ARXIV .2511.10647. URLhttps://doi. org/10.48550/arXiv.2511.10647. 14

    Pith/arXiv arXiv doi:10.48550/arxiv.2511.10647 2025

  19. [19]

    Partslip: Low-shot part segmentation for 3d point clouds via pretrained image-language models

    Liu, M., Zhu, Y ., Cai, H., Han, S., Ling, Z., Porikli, F., and Su, H. Partslip: Low-shot part segmentation for 3d point clouds via pretrained image-language models. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 21736–21746, 2023. 15

    2023

  20. [20]

    Scalable 3d captioning with pretrained models.NeurIPS, 36:75307– 75337, 2023

    Luo, T., Rockwell, C., Lee, H., and Johnson, J. Scalable 3d captioning with pretrained models.NeurIPS, 36:75307– 75337, 2023. 15

    2023

  21. [21]

    P3-sam: Native 3d part segmentation.arXiv preprint arXiv:2509.06784, 2025

    Ma, C., Li, Y ., Yan, X., Xu, J., Yang, Y ., Wang, C., Zhao, Z., Guo, Y ., Chen, Z., and Guo, C. P3-sam: Native 3d part segmentation.arXiv preprint arXiv:2509.06784, 2025. 9, 15

    arXiv 2025

  22. [22]

    L., Singh, R., Guo, Y ., Mazhar, H., et al

    Mittal, M., Yu, C., Yu, Q., Liu, J., Rudin, N., Hoeller, D., Yuan, J. L., Singh, R., Guo, Y ., Mazhar, H., et al. Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics and Automation Letters, 8(6): 3740–3747, 2023. 14

    2023

  23. [23]

    X., Yi, L., Tripathi, S., Guibas, L

    Mo, K., Zhu, S., Chang, A. X., Yi, L., Tripathi, S., Guibas, L. J., and Su, H. Partnet: A large-scale benchmark for fine- grained and hierarchical part-level 3d object understanding. InCVPR, pp. 909–918, 2019. 14

    2019

  24. [24]

    E., Liu, W., Tian, Y ., and Yuan, L

    Pang, Y ., Wang, W., Tay, F. E., Liu, W., Tian, Y ., and Yuan, L. Masked autoencoders for point cloud self-supervised learning. InECCV, 2022. 2, 9

    2022

  25. [25]

    Contrast with reconstruct: Contrastive 3d representa- tion learning guided by generative pretraining

    Qi, Z., Dong, R., Fan, G., Ge, Z., Zhang, X., Ma, K., and Yi, L. Contrast with reconstruct: Contrastive 3d representa- tion learning guided by generative pretraining. InInterna- tional Conference on Machine Learning, pp. 28223–28243. PMLR, 2023. 9 11 Utonia: Toward One Encoder for All Point Clouds

    2023

  26. [26]

    K., Gokaslan, A., Wijmans, E., Maksymets, O., Clegg, A., Turner, J

    Ramakrishnan, S. K., Gokaslan, A., Wijmans, E., Maksymets, O., Clegg, A., Turner, J. M., Undersander, E., Galuba, W., Westbury, A., Chang, A. X., Savva, M., Zhao, Y ., and Batra, D. Habitat-matterport 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AI. In NeurIPS, 2021. 15

    2021

  27. [27]

    Global-local bidirec- tional reasoning for unsupervised representation learn- ing of 3d point clouds.CVPR, pp

    Rao, Y ., Lu, J., and Zhou, J. Global-local bidirec- tional reasoning for unsupervised representation learn- ing of 3d point clouds.CVPR, pp. 5375–5384,

  28. [28]

    and Sievers, B

    Sauder, J. and Sievers, B. Self-supervised deep learning on point clouds by reconstructing space. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch ´e-Buc, F., Fox, E., and Garnett, R. (eds.),NeurIPS, volume 32. Curran Associates, Inc., 2019. URLhttps://proceedings. neurips.cc/paper_files/paper/2019/file/ 993edc98ca87f7e08494eec37fa836f7-Paper. pdf. 9

    2019

  29. [29]

    V ., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V ., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al

    Sim ´eoni, O., V o, H. V ., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V ., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al. Dinov3.arXiv preprint arXiv:2508.10104, 2025. 6, 10, 14

    Pith/arXiv arXiv 2025

  30. [30]

    Roformer: Enhanced transformer with rotary position em- bedding.Neurocomputing, 568:127063, 2024

    Su, J., Ahmed, M., Lu, Y ., Pan, S., Bo, W., and Liu, Y . Roformer: Enhanced transformer with rotary position em- bedding.Neurocomputing, 568:127063, 2024. 10

    2024

  31. [31]

    Scalability in perception for autonomous driving: Waymo open dataset

    Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Pat- naik, V ., Tsui, P., Guo, J., Zhou, Y ., Chai, Y ., Caine, B., et al. Scalability in perception for autonomous driving: Waymo open dataset. InCVPR, 2020. 2, 14, 15

    2020

  32. [32]

    Sundaralingam, B., Hari, S. K. S., Fishman, A., Garrett, C., Van Wyk, K., Blukis, V ., Millane, A., Oleynikova, H., Handa, A., Ramos, F., et al. Curobo: Parallelized collision- free robot motion generation. InICRA, pp. 8112–8119. IEEE, 2023. 14

    2023

  33. [33]

    Team, Q. et al. Qwen2 technical report.CoRR, abs/2407.10671, 2024. doi: 10.48550/ARXIV .2407.10671. URLhttps://doi.org/10.48550/arXiv.2407. 10671. 15

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.10671 2024

  34. [34]

    Geomae: Masked geometric target prediction for self-supervised point cloud pre-training

    Tian, X., Ran, H., Wang, Y ., and Zhao, H. Geomae: Masked geometric target prediction for self-supervised point cloud pre-training. InIEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR 2023, V ancouver , BC, Canada, June 17-24, 2023, pp. 13570–13580. IEEE, 2023. doi: 10.1109/CVPR52729.2023.01304. URLhttps:// doi.org/10.1109/CVPR52729.2...

    arXiv 2023

  35. [35]

    Mujoco: A physics engine for model-based control

    Todorov, E., Erez, T., and Tassa, Y . Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ inter- national conference on intelligent robots and systems, pp. 5026–5033. IEEE, 2012. 14

    2012

  36. [36]

    Llama: Open and efficient foundation lan- guage models.arXiv preprint arXiv:2302.13971, 2023

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi `ere, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation lan- guage models.arXiv preprint arXiv:2302.13971, 2023. 10

    Pith/arXiv arXiv 2023

  37. [37]

    Wang, H., Liu, Q., Yue, X., Lasenby, J., and Kusner, M. J. Unsupervised point cloud pre-training via occlusion com- pletion. InInternational Conference on Computer Vision, ICCV, 2021. 9

    2021

  38. [38]

    Sample-adaptive augmentation for point cloud recog- nition against real-world corruptions

    Wang, J., Ding, L., Xu, T., Dong, S., Xu, X., Bai, L., and Li, J. Sample-adaptive augmentation for point cloud recog- nition against real-world corruptions. InICCV, pp. 14330– 14339, October 2023. 15

    2023

  39. [39]

    Vggt: Visual geometry grounded trans- former

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., and Novotny, D. Vggt: Visual geometry grounded trans- former. InCVPR, 2025. 14

    2025

  40. [40]

    Point trans- former v2: Grouped vector attention and partition-based pooling

    Wu, X., Lao, Y ., Jiang, L., Liu, X., and Zhao, H. Point trans- former v2: Grouped vector attention and partition-based pooling. InNeurIPS, 2022. 10

    2022

  41. [41]

    Masked scene con- trast: A scalable framework for unsupervised 3d representa- tion learning

    Wu, X., Wen, X., Liu, X., and Zhao, H. Masked scene con- trast: A scalable framework for unsupervised 3d representa- tion learning. InCVPR, 2023. 2, 5

    2023

  42. [42]

    Point transformer v3: Simpler, faster, stronger

    Wu, X., Jiang, L., Wang, P.-S., Liu, Z., Liu, X., Qiao, Y ., Ouyang, W., He, T., and Zhao, H. Point transformer v3: Simpler, faster, stronger. InCVPR, 2024. 3, 4, 5, 6, 10

    2024

  43. [43]

    Towards large-scale 3d representation learning with multi-dataset point prompt training

    Wu, X., Tian, Z., Wen, X., Peng, B., Liu, X., Yu, K., and Zhao, H. Towards large-scale 3d representation learning with multi-dataset point prompt training. InCVPR, 2024. 5, 6

    2024

  44. [44]

    Sonata: Self-supervised learning of reliable point representations

    Wu, X., DeTone, D., Frost, D., Shen, T., Xie, C., Yang, N., Engel, J., Newcombe, R., Zhao, H., and Straub, J. Sonata: Self-supervised learning of reliable point representations. In CVPR, 2025. 2, 3, 4, 5, 6, 10

    2025

  45. [45]

    R., Guibas, L., and Litany, O

    Xie, S., Gu, J., Guo, D., Qi, C. R., Guibas, L., and Litany, O. Pointcontrast: Unsupervised pre-training for 3d point cloud understanding. InECCV, 2020. 9

    2020

  46. [46]

    W., Han, R., Fei-Fei, L., and Xie, S

    Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., and Xie, S. Thinking in space: How multimodal large language models see, remember, and recall spaces. InCVPR, pp. 10632–10643, 2025. 2

    2025

  47. [47]

    Cambrian- s: Towards spatial supersensing in video.arXiv preprint arXiv:2511.04670, 2025

    Yang, S., Yang, J., Huang, P., Brown, E., Yang, Z., Yu, Y ., Tong, S., Zheng, Z., Xu, Y ., Wang, M., et al. Cambrian- s: Towards spatial supersensing in video.arXiv preprint arXiv:2511.04670, 2025. 2

    Pith/arXiv arXiv 2025

  48. [48]

    Y ., Cao, Y .-P., and Liu, X

    Yang, Y ., Huang, Y ., Guo, Y .-C., Lu, L., Wu, X., Lam, E. Y ., Cao, Y .-P., and Liu, X. Sampart3d: Segment any part in 3d objects.arXiv preprint arXiv:2411.07184, 2024. 15

    Pith/arXiv arXiv 2024

  49. [49]

    Scan- net++: A high-fidelity dataset of 3d indoor scenes

    Yeshwanth, C., Liu, Y .-C., Nießner, M., and Dai, A. Scan- net++: A high-fidelity dataset of 3d indoor scenes. InICCV,

  50. [50]

    D., Rup- precht, C., and Schindler, K

    Yue, Y ., Robert, D., Wang, J., Hong, S., Wegner, J. D., Rup- precht, C., and Schindler, K. Litept: Lighter yet stronger point transformer, 2025. URLhttps://arxiv.org/ abs/2512.13689. 10

    arXiv 2025

  51. [51]

    Point cloud mixture-of-domain-experts model for 3d self-supervised learning

    Zha, Y ., Dai, T., Guo, H., Wang, Y ., Chen, B., Chen, K., and Xia, S.-T. Point cloud mixture-of-domain-experts model for 3d self-supervised learning. InIJCAI, pp. 2332–2340, 09

  52. [52]

    Sig- moid loss for language image pre-training

    Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. Sig- moid loss for language image pre-training. InICCV, pp. 11975–11986, 2023. 15

    2023

  53. [53]

    Point-m2ae: Multi-scale masked autoencoders for hierarchical point cloud pre-training

    Zhang, R., Guo, Z., Gao, P., Fang, R., Zhao, B., Wang, D., Qiao, Y ., and Li, H. Point-m2ae: Multi-scale masked autoencoders for hierarchical point cloud pre-training. In NeurIPS, 2022. 2, 6, 9

    2022

  54. [54]

    Llava-video: Video instruction tuning with synthetic data.TMLR, 2025, 2025

    Zhang, Y ., Wu, J., Li, W., Li, B., Ma, Z., Liu, Z., and Li, C. Llava-video: Video instruction tuning with synthetic data.TMLR, 2025, 2025. URLhttps://openreview. net/forum?id=EElFGvt39K. 15

    2025

  55. [55]

    Concerto: Joint 2d-3d self-supervised learning emerges spatial representations

    Zhang, Y ., Wu, X., Lao, Y ., Wang, C., Tian, Z., Wang, N., and Zhao, H. Concerto: Joint 2d-3d self-supervised learning emerges spatial representations. InNeurIPS, 2025. 2, 3, 4, 5, 6, 10, 14

    2025

  56. [56]

    Self- supervised pretraining of 3d features on any point-cloud

    Zhang, Z., Girdhar, R., Joulin, A., and Misra, I. Self- supervised pretraining of 3d features on any point-cloud. In ICCV, pp. 10252–10263, 2021. 10

    2021

  57. [57]

    Zhang, Z., Bai, M., and Li, E. L. Self-supervised pretrain- ing for large-scale point clouds. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), NeurIPS, volume 35, pp. 37806–37821. Curran Associates, Inc., 2022. 10

    2022

  58. [58]

    Point transformer

    Zhao, H., Jiang, L., Jia, J., Torr, P., and Koltun, V . Point transformer. InICCV, 2021. 10

    2021

  59. [59]

    Video-3d llm: Learn- ing position-aware video representation for 3d scene under- standing

    Zheng, D., Huang, S., and Wang, L. Video-3d llm: Learn- ing position-aware video representation for 3d scene under- standing. InCVPR, pp. 8995–9006, 2025. 9, 15

    2025

  60. [60]

    Structured3d: A large photo-realistic dataset for structured 3d modeling

    Zheng, J., Zhang, J., Li, J., Tang, R., Gao, S., and Zhou, Z. Structured3d: A large photo-realistic dataset for structured 3d modeling. InECCV, 2020. 14, 15

    2020

  61. [61]

    Open3d: A modern library for 3d data processing.arXiv preprint arXiv:1801.09847, 2018

    Zhou, Q.-Y ., Park, J., and Koltun, V . Open3d: A modern library for 3d data processing.arXiv preprint arXiv:1801.09847, 2018. 14

    Pith/arXiv arXiv 2018

  62. [62]

    Stereo magnification: Learning view synthesis using multi- plane images

    Zhou, T., Tucker, R., Flynn, J., Fyffe, G., and Snavely, N. Stereo magnification: Learning view synthesis using multi- plane images. InSIGGRAPH, 2018. 15 13 Utonia: Toward One Encoder for All Point Clouds Appendix A. Additional Implementation A.1. Data Preparation and Training Details Utonia is pretrained on a diverse mixture of datasets sum- marized in T...

    2018

  63. [2020]

    URLhttps://api.semanticscholar.org/ CorpusID:214713787. 9

  64. [2025]

    10 12 Utonia: Toward One Encoder for All Point Clouds

    doi: 10.24963/ijcai.2025/260. 10 12 Utonia: Toward One Encoder for All Point Clouds

    work page doi:10.24963/ijcai.2025/260 2025