Pith. sign in

REVIEW 4 major objections 6 minor 47 references

OpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read OpenUrban3D claims the first open-vocabulary semantic segmentation of large-scale urban point clouds from raw geometry alone, with no aligned images, pre-trained 3D segmentors, or manual labels.

desk verdict A plausible annotation-free open-vocab pipeline for urban point clouds, but the headline numbers rest on undisclosed prompts and duplicated baseline tables; fixable, but not yet reliable. read the letter →

arxiv 2509.10842 v1 pith:AHHA6QMU submitted 2025-09-13 cs.CV

Chongyu Wang , Kunlei Jing , Jihua Zhu , Di Wang This is my paper · ORCID
classification cs.CV
keywords open-vocabularysemanticsegmentationurbanpointcloudsknowledgedistillationvision-languagemodelsmulti-viewprojectionzero-shotSensatSUM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that open-vocabulary semantic segmentation of large-scale urban point clouds can be achieved from raw geometry alone, with no aligned photographs, no pre-trained 3D segmentation networks, and no manual labels. Its pipeline, OpenUrban3D, renders the point cloud from dozens of virtual viewpoints, extracts mask-level vision-language features from those renderings, fuses them with a sample-balanced scheme, and distills them into a 3D backbone that can be queried by arbitrary text at inference. The stakes are practical: urban scans are plentiful, while aligned imagery and dense annotations are scarce, so a method that removes that dependency would let planners, disaster-response teams, and digital-twin systems query raw scans in natural language. The paper reports 39.6% mIoU on SensatUrban and 75.4% mIoU on SUM, surpassing open-vocabulary baselines and, on SUM, the fully supervised PTV3.

What carries the argument

The load-bearing mechanism is a multi-view, multi-granularity projection module that places global and local virtual cameras around the scene's bounding box, renders the point cloud into RGB images with depth maps, and feeds those renders to a frozen mask-based vision-language model (ODISE). Mask features are back-projected onto 3D points using pinhole camera geometry plus Z-buffer depth validation, then fused across views by sample-balanced fusion, which down-samples over-represented large masks to prevent the distillation from being dominated by big objects. The fused features act as a teacher for a MinkUNet 3D backbone trained by minimizing $1 - \cos(F^{3D}, \text{stopgrad}(F^{2D}))$, aligning 3D features with CLIP's text space. At inference, the fused feature is $F^{fusion} = \alpha F^{3D} + (1-\alpha) F^{2D}$ with $\alpha$ near 0.1. An optional LLM first parses free-form commands into concrete class names, which are then embedded by CLIP.

What would settle it

Run OpenUrban3D on SensatUrban using only the benchmark's original class names as queries (e.g., 'wall', 'rail', 'bike') and compare the resulting mIoU against the paper's 39.6%; the paper's own ablation predicts about 30.7%, so a replication landing near the lower figure, or showing large mIoU swings under small prompt paraphrases, would falsify the claim of robust open-vocabulary generalization.

Watch

Extended reading notes

Core claim

On its own terms, OpenUrban3D's central discovery is that a 'segment-first, recognize-later' pipeline, which relies on pre-trained 3D instance segmentors and aligned RGB sequences, can be replaced by a rendering-and-distillation loop that works with raw point clouds only. The paper claims that rendering the point cloud from multi-scale virtual cameras, back-projecting mask-level vision-language features with Z-buffer occlusion checks, and distilling them into a MinkUNet backbone yields 3D features aligned with CLIP text space. At inference, a weighted fusion of 2D and 3D features, with the 3D weight kept small, assigns each point the class whose text embedding has the highest cosine similarity. The reported results are 39.6% mIoU and 84.7% OA on SensatUrban, and 75.4% mIoU and 90.5% OA on SUM, with the SUM figure exceeding the fully supervised PTV3's 74.0% mIoU.

Load-bearing premise

The headline numbers depend on a manually tuned 'Processed Prompt' for each benchmark class whose exact wording is not given; with the original class-name prompts, reported SensatUrban mIoU drops from 39.6% to 30.7%.

Editorial extensions

If this is right

  • Large-scale urban datasets that have discarded their original aerial or mobile images become usable for open-vocabulary segmentation, since only the point cloud is needed.
  • On SUM, the method surpasses the fully supervised PTV3, suggesting that web-scale vision-language knowledge distilled into a 3D backbone can compete with dense manual labels when categories are well defined.
  • The same trained model can be queried with novel text concepts at inference without retraining, since prediction is a cosine similarity against CLIP text embeddings.
  • The fusion study indicates 3D geometry's main contribution is boundary refinement rather than semantic content, which implies hybrid 2D-semantic and 3D-geometric designs are the right direction for urban point cloud understanding.
  • The prompt ablation shows performance swings by roughly 9 mIoU points between original and processed prompts, so open-vocabulary accuracy in practice depends on prompt precision as much as on the model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is automatic prompt optimization: if an LLM proposes candidate class descriptions and a small labeled probe set (or the model's own confidence) scores them, much of the 30.7-to-39.6 prompt gap might be recovered without manual tuning.
  • The small gap between 2D-visible mIoU (40.1) and full-scene mIoU (39.6) on SensatUrban suggests the distilled 3D branch carries occlusion-covered points well; pushing the same pipeline into denser street canyons or under bridges would test that robustness harder.
  • Because the method renders colorized point clouds, scenes with poor or missing colorization (night scans, shadow-heavy alleyways, sensor failure) may see degraded distillation; replacing color with geometric cues such as normals or height could make the pipeline color-agnostic.
  • The method's reliance on rendered RGB means its ceiling is tied to rendering quality and to the frozen 2D feature extractor; coupling it with diffusion-based or neural rendering might improve mask quality on low-density scans.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes OpenUrban3D, a framework for open-vocabulary semantic segmentation of large-scale urban point clouds that uses only raw geometry, without aligned RGB images, pre-trained 3D segmentors, or manual annotations. Virtual multi-view, multi-granularity images are rendered from the point cloud; a frozen vision-language model (ODISE image encoder with CLIP text encoder) extracts mask-level features that are back-projected and fused with a sample-balanced strategy; these 2D features are distilled into a MinkUNet 3D backbone, and at inference the 2D and 3D features are weight-averaged and compared with CLIP text embeddings. A large language model can optionally parse free-form user queries into category lists. Experiments on SensatUrban and SUM report mIoU of 39.6% and 75.4%, respectively, outperforming open-vocabulary baselines and, on SUM, the fully supervised PTV3. Ablations study the projection hyperparameters, sample-balanced fusion, fusion strategy, fusion weight, and text prompts.

Significance. If the reported results are reproducible, OpenUrban3D would be a practically valuable contribution: it is the first 3D open-vocabulary segmentation pipeline for large-scale urban point clouds that does not require aligned imagery or 3D annotations, and its cross-scene generalization claim is well motivated. The paper also ships a detailed architectural description and several ablation studies, which is commendable. However, the central quantitative claims are not yet fully supported: the headline results depend on undisclosed hand-crafted text prompts and unreported hyperparameter values, and there is an apparent duplication of baseline numbers across the two benchmark tables. These issues are fixable within the manuscript's scope, but they must be addressed before the performance claims can be accepted.

major comments (4)
  1. [IV-B, Tables I and II] The fully supervised baseline rows for PointNet, PointNet++, SPGraph, and SparseConv are numerically identical in Table I (SensatUrban) and Table II (SUM): the same mIoU values (23.7, 32.9, 37.2, 42.6), the same OA values (80.7, 84.3, 76.9, 85.2), and the same per-class IoU entries appear in both tables even though the two datasets have different class sets (13 classes on SensatUrban versus 7 classes on SUM). This duplication is implausible and undermines the credibility of the comparison against fully supervised methods, including the claim that OpenUrban3D surpasses PTV3 on SUM. Please correct the tables, explain the discrepancy, or provide the original baseline log files for verification.
  2. [IV-D.5, Table IV] The headline SensatUrban result of 39.6% mIoU is obtained only with a 'Processed Prompt' that is never specified anywhere in the paper. Table IV shows that using the original class-name prompt drops mIoU to 30.7%. Since the model's only text interface is cosine similarity against CLIP text embeddings, this 9-point mIoU swing means the main comparison is between a tuned prompt and the baselines, not a reproducible zero-shot prediction from arbitrary natural language. Please provide the exact processed prompt strings for both SensatUrban and SUM, report results for both the original and processed prompts on both datasets, and state clearly how these prompts were selected; ideally, prompt selection should not use the test-set metrics directly.
  3. [IV-D.1 and Figures 11/12] The specific values of K, A, R, and the fusion weight alpha used for the main results in Tables I, II, and IV are not reported. Section IV-D.1 and Table V show that mIoU on SUM varies from 72.3% to 77.2% as K, A, and R change, and Figures 11 and 12 show a clear performance peak in alpha, but the text only says the optimum is 'near 0.1' without giving the exact value. Without these numbers, the reported results cannot be reproduced or attributed to the architecture rather than to the chosen operating point. Please report the exact K, A, R, and alpha used for every result table, and consider reporting a small sensitivity range for alpha.
  4. [IV-C and IV-D.3, Tables III and VI] There is an inconsistency in which points are evaluated in different tables. The 'Fusion' rows in Table VI (SensatUrban 40.1 mIoU, SUM 78.5 mIoU) are numerically identical to the 'OpenUrban3D (2D-3D vis)' rows in Table III, indicating that Table VI evaluates only the 2D-visible subset, but this is not stated. The complete-model numbers in Table III are 39.6% mIoU on SensatUrban and 75.4% mIoU on SUM, which differ by 1.5 and 3.1 points respectively. This ambiguity directly affects the comparison with PTV3 (74.0% mIoU) and must be clarified: state explicitly which point subset is used in each table and report the full-scene numbers consistently in all ablation tables.
minor comments (6)
  1. [III.G] The example output in Section III.G reads '[sidewall, road]'; the intended class is presumably 'sidewalk', and the typo should be corrected.
  2. [I] In the introduction, 'an 3D open-vocabulary' should be 'a 3D open-vocabulary'.
  3. [IV-D.1, Table V caption] The caption of Table V does not state the dataset or the feature combination used for the reported mIoU values; specify that these are SUM results and whether they use the full 2D-3D fused features or a different configuration.
  4. [III.G and IV] The LLM-powered text query module is described in Section III.G but never evaluated. At least one quantitative or qualitative example of a free-form LLM-parsed query should be included to substantiate the claimed 'arbitrary natural language' interface.
  5. [IV-D.2] In the SBFF ablation, the mIoU gain is only 0.3 percentage points (27.9% to 28.2%); describing this as 'compelling evidence' is overstated. The per-class gains are meaningful, but the wording should match the modest overall improvement.
  6. [II.C] The related-work claim that Suzuki et al. is 'impractical for occlusion-heavy urban scenes' is cited to a paper on human part segmentation; a more directly relevant urban-scene reference would strengthen the argument.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline SensatUrban mIoU is a tuned-prompt result: the undisclosed 'Processed Prompt' yields 39.6% vs 30.7% with original class names, so the open-vocabulary claim is partly fitted.

  1. fitted input called prediction [Section IV-D5 (Impact of Prompts on Model Performance), Table IV]
    "The experimental results reveal the critical role of a prompt's semantic precision. When using a relatively ambiguous original prompt, for instance, using "wall" to refer to all wall-like structures (including fences and barriers), the model achieved an mIoU of only 30.7%. ... However, when we switched to an optimized prompt with more specific and unambiguous semantics (denoted as "OpenUrban3D (2D-3D) w/ Processed Prompt"), the model's mIoU significantly increased to 39.6%."

    The headline SensatUrban result (39.6% mIoU, 84.7% OA) is identical to the 'Processed Prompt' row in Table IV, yet that prompt is optimized for the benchmark classes and never disclosed. The same pipeline with the original class-name prompt drops to 30.7% mIoU and 72.5% OA, a 9-point swing caused solely by prompt wording. Since inference is only cosine similarity to CLIP text embeddings (Eq. 18), the reported open-vocabulary performance is a tuned-prompt configuration, not a zero-shot prediction from arbitrary natural language. The paper's central claim of 'arbitrary text queries' is therefore evaluated with a fitted input, so the headline comparison against baselines partly measures prompt engineering rather than the architecture's open-vocabulary capability.

full rationale

OpenUrban3D is not built on a claimed mathematical derivation; it is an empirical pipeline whose components (multi-view multi-granularity projection, mask-level VLM feature extraction, back-projection, sample-balanced fusion, and distillation) are independent and mostly self-contained. The distillation loss (Eq. 16) forces F3D into the CLIP space of F2D by design, which is the intended mechanism rather than a hidden circularity. I found no load-bearing self-citation chain and no uniqueness theorem imported from the authors' own prior work; references to OpenScene, ODISE, and CLIP are external. The one serious fitted-input issue is the text prompt: Table IV shows the SensatUrban headline 39.6% mIoU is obtained only with a 'Processed Prompt' that is optimized for the benchmark classes and never specified; replacing it with the original class-name prompt drops mIoU to 30.7% and OA from 84.7% to 72.5%. Because inference is cosine similarity to CLIP text embeddings, this 9-point swing is driven entirely by prompt wording. Thus the central 'arbitrary natural language' / zero-shot claim is evaluated with a tuned input, and the reported improvement over open-vocabulary baselines is partly a property of the hidden prompt, not of the architecture alone. The fusion weight alpha and projection hyperparameters are also selected on benchmark metrics, though this is ordinary model selection; the undisclosed processed prompt is the load-bearing circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. Its free parameters are the projection and fusion hyperparameters plus the unstated prompt template, all tuned on benchmark metrics.

free parameters (6)
  • Fusion weight alpha = ~0.1 (peak in Fig. 11/12)
    Balances 3D vs 2D features in Eq. 17; tuned on benchmark metrics, not reported exactly.
  • View granularity K = 4 (optimal from Table V)
    Number of local anchor grid lines; ablation shows K=4 best; final setting not stated.
  • Angular sampling interval A = not stated for main results
    Controls number of rendered views; smaller A improves S_R and mIoU in Table V but increases cost.
  • Camera trajectory radius R = not stated for main results
    Controls occlusion; smaller R yields better mIoU in Table V; final value missing.
  • Text prompt template ('Processed Prompt') = unspecified
    Switching from original to processed prompts raises SensatUrban mIoU from 30.7% to 39.6% (Table IV); the processed text is not given.
  • top-k in SBFF threshold = k unspecified
    k in Eq. 13 for the per-view sampling threshold is not specified.
assumptions (4)
  • domain assumption Rendered point-cloud views are semantically interpretable by the frozen 2D VLM (ODISE) despite lacking photographic texture and being generated from virtual cameras.
    The whole pipeline rests on back-projecting ODISE mask features from rendered views; Section III-C.2.
  • domain assumption ODISE features and CLIP text embeddings are aligned in a shared space for arbitrary urban categories.
    Used at inference for cosine similarity (Eq. 18); adopted from prior work, not verified here.
  • standard math Depth-based occlusion checking with Z-buffer correctly assigns features to visible points.
    Standard pinhole projection; described in Section III-C.2.
  • domain assumption Color information in the input point clouds is reliable enough for rendering.
    The authors themselves note uneven illumination and shadows can harm 3D learning (Section V).

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds." pith.science (2026). https://pith.science/paper/AHHA6QMU

@misc{pith2026250910842,
  author       = {Pith},
  title        = {Pith review of: OpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AHHA6QMU}},
  note         = {Machine review of arXiv:2509.10842}
}
read the original abstract

Open-vocabulary semantic segmentation enables models to recognize and segment objects from arbitrary natural language descriptions, offering the flexibility to handle novel, fine-grained, or functionally defined categories beyond fixed label sets. While this capability is crucial for large-scale urban point clouds that support applications such as digital twins, smart city management, and urban analytics, it remains largely unexplored in this domain. The main obstacles are the frequent absence of high-quality, well-aligned multi-view imagery in large-scale urban point cloud datasets and the poor generalization of existing three-dimensional (3D) segmentation pipelines across diverse urban environments with substantial variation in geometry, scale, and appearance. To address these challenges, we present OpenUrban3D, the first 3D open-vocabulary semantic segmentation framework for large-scale urban scenes that operates without aligned multi-view images, pre-trained point cloud segmentation networks, or manual annotations. Our approach generates robust semantic features directly from raw point clouds through multi-view, multi-granularity rendering, mask-level vision-language feature extraction, and sample-balanced fusion, followed by distillation into a 3D backbone model. This design enables zero-shot segmentation for arbitrary text queries while capturing both semantic richness and geometric priors. Extensive experiments on large-scale urban benchmarks, including SensatUrban and SUM, show that OpenUrban3D achieves significant improvements in both segmentation accuracy and cross-scene generalization over existing methods, demonstrating its potential as a flexible and scalable solution for 3D urban scene understanding.

Figures

Figures reproduced from arXiv: 2509.10842 by the authors.

Figure 1
Figure 1. Visualization of open-vocabulary semantic understand [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall architecture of the proposed OpenUrban3D framework [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the Multi-view Multi-granularity Projection module. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Example visualization from the SensatUrban dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Example visualization from the SUM dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative visualization results from OpenUrban3D on the SensatUrban dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Qualitative visualization results from OpenUrban3D on the SUM dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Limitations of 3D-only features in distinguishing geo [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 10
Figure 10. Figure 10: Comparison of different feature combinations for [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: The impact of the weighted fusion coefficient [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: The impact of the weighted fusion coefficient [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Impact of text prompts on model performance. [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 25 canonical work pages

  1. [1]

    Urban digital twins for smart cities and citizens: The case study of herrenberg, germany,

    F. Dembski, U. W ¨ossner, M. Letzgus, M. Ruddat, and C. Yamu, “Urban digital twins for smart cities and citizens: The case study of herrenberg, germany,”Sustainability, vol. 12, no. 6, p. 2307, 2020

    work page 2020

  2. [2]

    Lidar boosts 3d ecological observations and modelings: A review and perspective,

    Q. Guo, Y . Su, T. Hu, H. Guan, S. Jin, J. Zhang, X. Zhao, K. Xu, D. Wei, M. Kellyet al., “Lidar boosts 3d ecological observations and modelings: A review and perspective,”IEEE Geoscience and Remote Sensing Magazine, vol. 9, no. 1, pp. 232–257, 2020

    work page 2020

  3. [3]

    Towards sustainable smart cities: A review of trends, architectures, components, and open challenges in smart cities,

    B. N. Silva, M. Khan, and K. Han, “Towards sustainable smart cities: A review of trends, architectures, components, and open challenges in smart cities,”Sustainable cities and society, vol. 38, pp. 697–713, 2018

    2018

  4. [4]

    Point cloud modeling as a bridge between landscape design and planning,

    P. R. Urech, M. A. Dissegna, C. Girot, and A. Gr ˆet-Regamey, “Point cloud modeling as a bridge between landscape design and planning,” Landscape and Urban Planning, vol. 203, p. 103903, 2020

    work page 2020

  5. [5]

    Lidar—a technology to assist with smart cities and climate change resilience: A case study in an urban metropolis,

    R. Garnett and M. D. Adams, “Lidar—a technology to assist with smart cities and climate change resilience: A case study in an urban metropolis,”ISPRS International Journal of Geo-Information, vol. 7, no. 5, p. 161, 2018

    work page 2018

  6. [6]

    Deep learning for lidar point clouds in autonomous driving: A review,

    Y . Li, L. Ma, Z. Zhong, F. Liu, M. A. Chapman, D. Cao, and J. Li, “Deep learning for lidar point clouds in autonomous driving: A review,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 8, pp. 3412–3432, 2020

    2020

  7. [7]

    Deep learning for 3d point clouds: A survey,

    Y . Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3d point clouds: A survey,”IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 12, pp. 4338–4364, 2020

    2020

  8. [8]

    Sqn: Weakly-supervised semantic segmentation of large- scale 3d point clouds,

    Q. Hu, B. Yang, G. Fang, Y . Guo, A. Leonardis, N. Trigoni, and A. Markham, “Sqn: Weakly-supervised semantic segmentation of large- scale 3d point clouds,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 600–619

    work page 2022

Show all 47 references
  1. [9]

    3d semantic segmen- tation with submanifold sparse convolutional networks,

    B. Graham, M. Engelcke, and L. Van Der Maaten, “3d semantic segmen- tation with submanifold sparse convolutional networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9224–9232

    2018

  2. [10]

    Pointnet: Deep learning on point sets for 3d classification and segmentation,

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660

    2017

  3. [11]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,”Advances in neural information processing systems, vol. 30, 2017

    2017

  4. [12]

    Point transformer v3: Simpler faster stronger,

    X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y . Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler faster stronger,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 4840–4851

    2024

  5. [13]

    Randla-net: Efficient semantic segmentation of large- scale point clouds,

    Q. Hu, B. Yang, L. Xie, S. Rosa, Y . Guo, Z. Wang, N. Trigoni, and A. Markham, “Randla-net: Efficient semantic segmentation of large- scale point clouds,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 108–11 117

    2020

  6. [14]

    Language-driven semantic segmentation,

    B. Li, K. Q. Weinberger, S. Belongie, V . Koltun, and R. Ranftl, “Language-driven semantic segmentation,”arXiv preprint arXiv:2201.03546, 2022

    2022 arXiv

  7. [15]

    Scaling open-vocabulary image segmentation with image-level labels,

    G. Ghiasi, X. Gu, Y . Cui, and T.-Y . Lin, “Scaling open-vocabulary image segmentation with image-level labels,” inEuropean conference on computer vision. Springer, 2022, pp. 540–557

    2022

  8. [16]

    Maskclip: Masked self-distillation advances contrastive language-image pretraining,

    X. Dong, J. Bao, Y . Zheng, T. Zhang, D. Chen, H. Yang, M. Zeng, W. Zhang, L. Yuan, D. Chenet al., “Maskclip: Masked self-distillation advances contrastive language-image pretraining,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, p...

    2023

  9. [17]

    Open-vocabulary semantic segmentation with mask-adapted clip,

    F. Liang, B. Wu, X. Dai, K. Li, Y . Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu, “Open-vocabulary semantic segmentation with mask-adapted clip,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 7061–7070

    2023

  10. [18]

    Open- vocabulary panoptic segmentation with text-to-image diffusion models,

    J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, and S. De Mello, “Open- vocabulary panoptic segmentation with text-to-image diffusion models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 2955–2966

    2023

  11. [19]

    Openscene: 3d scene understanding with open vocabularies,

    S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, T. Funkhouseret al., “Openscene: 3d scene understanding with open vocabularies,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 815–824

    2023

  12. [20]

    Regionplc: Regional point-language contrastive learning for open-world 3d scene understand- ing,

    J. Yang, R. Ding, W. Deng, Z. Wang, and X. Qi, “Regionplc: Regional point-language contrastive learning for open-world 3d scene understand- ing,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 19 823–19 832

    2024

  13. [21]

    Pla: Language- driven open-vocabulary 3d scene understanding,

    R. Ding, J. Yang, C. Xue, W. Zhang, S. Bai, and X. Qi, “Pla: Language- driven open-vocabulary 3d scene understanding,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 7010–7019

    2023

  14. [22]

    Towards semantic segmentation of urban-scale 3d point clouds: A dataset, benchmarks and challenges,

    Q. Hu, B. Yang, S. Khalid, W. Xiao, N. Trigoni, and A. Markham, “Towards semantic segmentation of urban-scale 3d point clouds: A dataset, benchmarks and challenges,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4977– 4987

    2021

  15. [23]

    Sum: A benchmark dataset of semantic urban meshes,

    W. Gao, L. Nan, B. Boom, and H. Ledoux, “Sum: A benchmark dataset of semantic urban meshes,”ISPRS Journal of Photogrammetry and Remote Sensing, vol. 179, pp. 108–120, 2021

    2021

  16. [24]

    The hessigheim 3d (h3d) benchmark on semantic segmentation of high-resolution 3d point clouds and textured meshes from uav lidar and multi-view-stereo,

    M. K ¨olle, D. Laupheimer, S. Schmohl, N. Haala, F. Rottensteiner, J. D. Wegner, and H. Ledoux, “The hessigheim 3d (h3d) benchmark on semantic segmentation of high-resolution 3d point clouds and textured meshes from uav lidar and multi-view-stereo,”ISPRS Open Journal of Photog...

    2021

  17. [25]

    Toronto- 3d: A large-scale mobile lidar dataset for semantic segmentation of urban roadways,

    W. Tan, N. Qin, L. Ma, Y . Li, J. Du, G. Cai, K. Yang, and J. Li, “Toronto- 3d: A large-scale mobile lidar dataset for semantic segmentation of urban roadways,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 202–203

    2020

  18. [26]

    Openmask3d: Open-vocabulary 3d instance segmenta- tion,

    A. Takmaz, E. Fedele, R. W. Sumner, M. Pollefeys, F. Tombari, and F. Engelmann, “Openmask3d: Open-vocabulary 3d instance segmenta- tion,”arXiv preprint arXiv:2306.13631, 2023

    2023 arXiv

  19. [27]

    Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance,

    P. Nguyen, T. D. Ngo, E. Kalogerakis, C. Gan, A. Tran, C. Pham, and K. Nguyen, “Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 4018–4028

    2024

  20. [28]

    Mask3d: Mask transformer for 3d semantic instance segmentation,

    J. Schult, F. Engelmann, A. Hermans, O. Litany, S. Tang, and B. Leibe, “Mask3d: Mask transformer for 3d semantic instance segmentation,” arXiv preprint arXiv:2210.03105, 2022

    2022 arXiv

  21. [29]

    Kpconv: Flexible and deformable convolution for point clouds,

    H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” inProceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6411–6420

    2019

  22. [30]

    4d spatio-temporal convnets: Minkowski convolutional neural networks,

    C. Choy, J. Gwak, and S. Savarese, “4d spatio-temporal convnets: Minkowski convolutional neural networks,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 3075–3084

    2019

  23. [31]

    Point transformer,

    H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V . Koltun, “Point transformer,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16 259–16 268

    2021

  24. [32]

    Point transformer v2: Grouped vector attention and partition-based pooling,

    X. Wu, Y . Lao, L. Jiang, X. Liu, and H. Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,” inNeurIPS, 2022

    2022

  25. [33]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PmLR, 2021, pp. 8748–8763

    2021

  26. [34]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: a visual language model for few-shot learning,”Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  27. [35]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34 892– 34 916, 2023

    2023

  28. [36]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4015–4026

    2023

  29. [37]

    Pointclip: Point cloud understanding by clip,

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y . Qiao, P. Gao, and H. Li, “Pointclip: Point cloud understanding by clip,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 8552–8562

    2022

  30. [38]

    Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning,

    X. Zhu, R. Zhang, B. He, Z. Guo, Z. Zeng, Z. Qin, S. Zhang, and P. Gao, “Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 2639–2650

    2023

  31. [39]

    Clip2point: Transfer clip to point cloud classification with 15 image-depth pre-training,

    T. Huang, B. Dong, Y . Yang, X. Huang, R. W. Lau, W. Ouyang, and W. Zuo, “Clip2point: Transfer clip to point cloud classification with 15 image-depth pre-training,” inProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, 2023, pp. 22 157–22 167

    2023

  32. [40]

    Clip2scene: Towards label-efficient 3d scene understanding by clip,

    R. Chen, Y . Liu, L. Kong, X. Zhu, Y . Ma, Y . Li, Y . Hou, Y . Qiao, and W. Wang, “Clip2scene: Towards label-efficient 3d scene understanding by clip,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7020–7030

    2023

  33. [41]

    Opengraph: Open-vocabulary hierarchical 3d graph representation in large-scale outdoor environments,

    Y . Deng, J. Wang, J. Zhao, X. Tian, G. Chen, Y . Yang, and Y . Yue, “Opengraph: Open-vocabulary hierarchical 3d graph representation in large-scale outdoor environments,”IEEE Robotics and Automation Let- ters, 2024

    2024

  34. [42]

    Openins3d: Snap and lookup for 3d open-vocabulary instance segmen- tation,

    Z. Huang, X. Wu, X. Chen, H. Zhao, L. Zhu, and J. Lasenby, “Openins3d: Snap and lookup for 3d open-vocabulary instance segmen- tation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 169–185

    2024

  35. [43]

    Open-vocabulary semantic part segmentation of 3d human,

    K. Suzuki, B. Du, G. Krishnan, K. Chen, R. B. Li, and T. Nguyen, “Open-vocabulary semantic part segmentation of 3d human,”arXiv preprint arXiv:2502.19782, 2025

    2025 arXiv

  36. [44]

    Large-scale point cloud semantic segmentation with superpoint graphs,

    L. Landrieu and M. Simonovsky, “Large-scale point cloud semantic segmentation with superpoint graphs,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4558– 4567

    2018

  37. [45]

    Large-scale point cloud semantic segmentation via local perception and global descriptor vector,

    Z. Zeng, Y . Xu, Z. Xie, W. Tang, J. Wan, and W. Wu, “Large-scale point cloud semantic segmentation via local perception and global descriptor vector,”Expert Systems with Applications, vol. 246, p. 123269, 2024

    2024

  38. [46]

    Pointnat: large- scale point cloud semantic segmentation via neighbor aggregation with transformer,

    Z. Zeng, H. Qiu, J. Zhou, Z. Dong, J. Xiao, and B. Li, “Pointnat: large- scale point cloud semantic segmentation via neighbor aggregation with transformer,”IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–18, 2024

    2024

  39. [47]

    Eyenet++: A multi-scale and multi-density approach for outdoor 3d semantic segmentation inspired by the human visual field,

    S. Yoo, Y . Jeong, M. M. Sheikholeslami, and G. Sohn, “Eyenet++: A multi-scale and multi-density approach for outdoor 3d semantic segmentation inspired by the human visual field,”IEEE Transactions on Geoscience and Remote Sensing, 2025

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.