Pith. sign in

REVIEW 4 major objections 5 minor 60 references

All in One: Visual-Description-Guided Unified Point Cloud Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that offline LLM-generated class descriptions and internet-sourced reference images, embedded with CLIP and used as queries in a unified mask decoder, sharpen semantic, instance, and panoptic segmentation of 3D point…

desk verdict Genuinely new way to inject LLM text and internet images into unified 3D segmentation, with real but modest gains, though the causal story needs controls and the SOTA claim is too strong. read the letter →

arxiv 2507.05211 v2 pith:PEMR7XRV submitted 2025-07-07 cs.CV cs.AI

Zongyan Han , Mohamed El Amine Boudjoghra , Jiahua Dong , Jinhong Wang , Rao Muhammad Anwer This is my paper · ORCID
classification cs.CVcs.AI
keywords 3Dpointcloudsegmentationunifiedpanopticvision-languagemodelsCLIPlargelanguagemultimodalqueriescontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VDG-Uni3DSeg sets out to show that a unified 3D point-cloud segmenter—one model doing semantic, instance, and panoptic segmentation—performs better when its mask decoder is given offline multimodal references: textual descriptions of each class written by an LLM, and reference images collected from the web, both encoded with CLIP. The framework attaches these as extra queries in the decoder, aligns point features to them through a Semantic-Visual Contrastive Loss, and augments the backbone with a sparse-attention Spatial Enhancement Module. On S3DIS, ScanNet, and ScanNet200 it reports consistent gains over the OneFormer3D baseline, the largest being a 4.1-point panoptic-quality jump and a 9.6-point thing-quality jump on S3DIS Area-5. If the claim holds, rich semantic priors can be injected into 3D perception from text and web images alone, with no paired image-point-cloud data needed at inference.

What carries the argument

The load-bearing object is the set of Multi-Modal Reference Queries: for every class, $K$ LLM-generated textual descriptions and $L$ internet-sourced reference images, embedded by CLIP's text and image encoders and projected into the 3D feature space. These queries enter the OneFormer3D-style mask decoder alongside instance queries; after several fusion blocks they yield two semantic masks, one per modality, which are max-pooled per class and summed. The Semantic-Visual Contrastive Loss meanwhile anchors point features to the correct description and image embeddings, and the Spatial Enhancement Module applies sparse attention over randomly sampled point subsets to capture scene-wide spatial relations cheaply. Together these pieces carry the argument that offline multimodal knowledge supplies the fine-grained discrimination that a pure 3D backbone lacks.

What would settle it

Train the same pipeline but replace the LLM descriptions with bare class names and the internet images with irrelevant pictures from another domain, holding all other settings fixed; if semantic mIoU, instance mAP, and panoptic PQ stay at the reported levels instead of falling toward OneFormer3D's numbers, then the multimodal queries are not the cause of the gain. A complementary check would rerun the S3DIS ablation while varying the number of CLIP-selected reference images per class and watch whether the PQ curve tracks image relevance.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that the bottleneck in unified 3D segmentation is a lack of rich class-level cues, and that this bottleneck can be broken by multimodal queries. For each class, an LLM (LLaMA 3.1) generates descriptions covering color, shape, and type, and a search engine supplies reference images; CLIP turns both into embeddings that act as task-specific queries in a transformer mask decoder. The decoder fuses description-based and image-based semantic predictions, while a contrastive loss pulls point features toward their class's description and image embeddings and away from others, and a sparse-attention module spreads scene-wide context. The result, the paper reports, is state-of-the-art semantic, instance, and panoptic performance across S3DIS, ScanNet, and ScanNet200, including fine-grained gains on ScanNet200's 198 object classes.

Load-bearing premise

The whole framework rests on the assumption that CLIP embeddings of LLM-written descriptions and web-scraped images are semantically aligned with the classes that actually appear in 3D indoor scenes, and that this alignment survives the learned projection into point-feature space via the contrastive loss.

Editorial extensions

If this is right

  • On S3DIS Area-5, the model raises panoptic PQ from 62.2 to 66.3 and thing-PQ from 58.4 to 68.0 over OneFormer3D, and six-fold cross-validation shows analogous gains.
  • On ScanNet validation, mAP50 rises by 1.3 points and mIoU by 0.4 points over the baseline, with the hidden test set also improving.
  • On ScanNet200, the method reaches state-of-the-art instance and panoptic performance and remains competitive on semantic mIoU despite the 198-class fine-grained setting.
  • Ablations attribute the gains to all three additions: removing the SVC loss costs 2.6 mAP25 and 6.9 PQth, while removing either description or image queries also degrades results.
  • Because every multimodal cue is generated offline, the inference-time pipeline needs no paired images, captioning models, or extra vision-language modules.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is what the paper claims, the same offline-query recipe should transfer to any closed-set 3D perception task with a fixed class list, such as object detection or part segmentation, turning CLIP's text-image space into a cheap source of class priors.
  • The appendix's own comparison (CLIP-selected web images beat random ones) indicates the ceiling is set by reference relevance; a natural stress test is to measure how sensitive the gains are to the number, source, and lexical phrasing of the LLM descriptions, especially on fine-grained classes.
  • Because the queries are offline and fixed, the framework could in principle re-segment a scene under a new class taxonomy by swapping the reference set at inference time, nudging the closed-set system toward open-vocabulary behavior without retraining the 3D encoder.
  • A head-to-head against paired-image distillation (e.g., using the RGB-D frames in ScanNet itself as references) would quantify how much of the improvement comes from unpaired web images versus genuinely aligned 2D-3D supervision.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes VDG-Uni3DSeg, a unified 3D point cloud segmentation framework that augments a OneFormer3D-style mask decoder with offline multimodal reference queries: LLM-generated textual descriptions and internet-sourced reference images are encoded with CLIP and fed as additional semantic queries, alongside a new Semantic-Visual Contrastive (SVC) loss and a Spatial Enhancement Module based on sparse attention. The method is evaluated on S3DIS, ScanNet, and ScanNet200 for semantic, instance, and panoptic segmentation. The authors claim state-of-the-art results and consistent improvements over OneFormer3D across all three datasets.

Significance. If the claims are validated, the paper would make a useful practical contribution: it shows that closed-set 3D segmentation can be improved by offline multimodal knowledge (LLM text and unpaired internet images) without requiring paired image-point cloud data at inference. The architecture is clearly described, the components are ablated, and the authors promise code release. However, the current evidence does not yet establish the load-bearing causal claim that the semantic content of the descriptions/images drives the improvement: several metrics in the paper's own tables contradict 'consistent improvements,' the gains are small on fine-grained classes, and the random-query control for images shows that mIoU is nearly unchanged while only PQ improves with CLIP-selected images. These issues are addressable with additional controlled experiments, multiple seeds, and a more tempered presentation.

major comments (4)
  1. [§4.2, Tables 1–3] The abstract and Section 5 claim 'consistent improvements' and 'state-of-the-art' performance, but several reported metrics decrease relative to the OneFormer3D baseline: S3DIS 6-fold PQst drops from 74.5 to 72.7 (Table 1), ScanNet validation PQth drops from 70.5 to 70.0 (Table 2), ScanNet hidden test mAP25 drops from 89.6 to 88.0 (Table 2), and ScanNet200 PQst drops from 78.6 to 77.9 (Table 3). These contradictions undermine the central claim of consistent gains. Please report multiple seeds with mean and standard deviation (or another statistical test) and either reconcile the claim with the drops or temper the conclusion to specify which metrics actually improve.
  2. [§3.4, Eq. (9), and Appendix Table R6] The causal claim that 'the semantic content of LLM descriptions and internet images drives the improvement' is not supported by the current ablations. The SVC loss aligns point features to the same CLIP embeddings that are used as decoder queries, so the auxiliary loss and the additional learned projections may contribute even if the query content is semantically meaningless. The only non-semantic control in the paper is for images (Appendix Table R6), where two random image sets raise S3DIS mIoU from 69.8 to 71.3–71.7, essentially matching the 71.5 of the CLIP-selected images; only PQ (66.3 vs. 62.0/63.5) clearly benefits from the semantic selection. No control is provided for random text descriptions or for randomly initialized projected queries of the same capacity. Please add such controls and report whether the mIoU gain over the baseline persists when query content is random.
  3. [§4.1, Implementation Details; §3.3; Eq. (10); Appendix Table R5] Several hyperparameters required for reproducibility are omitted: the loss weights lambda_1, lambda_2, and lambda_3 in Eq. (10); the number of fusion layers B in Section 3.3; the SEM sample size used for the main results (Appendix Table R5 reports a sweep but does not state which value was used in Tables 1–3); and the number of reference images per class L (the text says 'collect 20 images per class and select the top-5' only in the appendix). Please state all these values explicitly in the main text or a reproducibility table.
  4. [§4.2, Table 3 and surrounding text] The ScanNet200 results do not support the claim of 'state-of-the-art performance in all instance, semantic, and panoptic segmentation.' The gains over OneFormer3D are small (mAP +0.5, PQ +0.3, PQth +0.4, mAP50 unchanged at 40.0, PQst -0.7) and there are no error bars; without multiple seeds or per-class breakdowns, these differences are within plausible seed noise. Additionally, the text says 'slightly lower semantic mIoU' while the table reports a higher mIoU (29.7 vs. 29.0); this inconsistency should be corrected.
minor comments (5)
  1. [References] Several references are duplicated: [3] and [4] are the same paper (Hierarchical aggregation), [5] and [6] are the same (Per-pixel classification), [7] and [8] are the same (Masked-attention mask transformer), and [18] and [19] are the same (3D-SIS). Please deduplicate.
  2. [§3.3, Eq. (6) and 'Multi-Modal Segmentation Ensemble'] The notation is inconsistent: Eq. (6) defines the semantic masks as M_t and M_o, but the ensemble paragraph refers to M_m^d and M_m^o; please use a single consistent notation for description and image predictions.
  3. [Eq. (9)] The expectation notation in Eq. (9) is nonstandard; the subscript E_{(x̃_i, q+)} over a log-softmax is unclear. Please rewrite the SVC loss with an explicit sum over sampled points and positives.
  4. [Figure 2] The caption contains redundant repeated labels ('LLM Models', 'Defined Types') that make the figure harder to read; please clean up the caption and the diagram text.
  5. [Section 6, Acknowledgment] The acknowledgment thanking 'all the reviewers' is inappropriate in a submitted manuscript and should be removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are measured on held-out benchmark splits using fixed CLIP/LLM inputs, so the central claim is not equivalent to the method's own inputs by construction.

full rationale

The claimed derivation chain is: fixed CLIP and LLM encoders convert class names, LLM-generated descriptions, and internet reference images into query embeddings; these queries enter the mask decoder (Eqs. 4-6) and the Semantic-Visual Contrastive loss (Eq. 9); the model is trained with supervision on S3DIS, ScanNet, and ScanNet200 and evaluated on validation/test splits (Sec. 4.2). None of these steps defines the reported metric in terms of the method's inputs. The benchmark labels and metrics are external, the CLIP and LLM weights are pretrained and used as fixed inputs, and no parameter is fitted to a test set and then renamed as a prediction. The only author-overlapping citation, Ref. [1], is used as background on spatial and semantic supervision for 3D instance segmentation and is not load-bearing for the main contribution. The Semantic-Visual Contrastive loss does use the same CLIP embeddings as decoder queries and as contrastive anchors, which is self-referential in design, but this does not make the benchmark numbers equivalent to an input by construction; the scores still measure generalization to held-out annotated scenes. Concerns that random image sets also raise S3DIS mIoU and that several metrics trail OneFormer3D are legitimate questions about causal attribution and statistical robustness, but they are not circularity under the stated hard rules. No specific equation-level reduction or fitted-input-called-prediction step can be exhibited, so no circularity is flagged.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that CLIP embeddings of LLM text and internet images transfer to 3D point features, and on several unreported hyperparameters. The paper introduces no new entities or physical quantities.

free parameters (6)
  • temperature tau (SVC loss) = 1.0
    Section 3.4 states tau is empirically set to 1.0.
  • loss weights lambda1, lambda2, lambda3 = not reported
    Section 3.4, Eq. 10 defines these weights but their values are never given in the paper.
  • number of fusion layers B = not reported
    Section 3.3 uses B fusion layers in the mask decoder but B is never specified.
  • SEM sample size = not reported (appendix tests 16/64/128)
    Section 3.1 uses a randomly sampled subset Si; the chosen size for main results is not stated, although Appendix Table R5 implies larger sizes help.
  • number of descriptions per class K = 10
    Section 4.1 states 'For each class, we generate 10 descriptions using LLaMA 3.1'.
  • number of reference images per class L = 5
    Appendix A: collect 20 images per class and select the top-5 by CLIP similarity to the class name.
assumptions (4)
  • domain assumption CLIP text and image embeddings are semantically aligned with 3D point cloud classes for the target domains.
    The entire method rests on transferring CLIP embeddings to point features via queries and the SVC loss (Eq. 4, 5, 9); no evidence is provided that CLIP's 2D embedding space separates fine-grained 3D scene classes.
  • domain assumption LLM-generated descriptions (LLaMA 3.1) capture visual attributes useful for segmentation and are consistent across classes.
    Section 3.2 assumes the LLM produces K=10 useful descriptions per class; the paper does not show the descriptions or evaluate their quality.
  • domain assumption Internet-sourced reference images from Bing are representative of the classes appearing in S3DIS, ScanNet, and ScanNet200.
    Appendix A describes collecting 20 images per class and selecting top-5 by CLIP similarity; no image list or domain gap analysis is provided.
  • domain assumption The voxelization and sparse 3D U-Net features from OneFormer3D provide a suitable point feature space for multimodal query fusion.
    The method builds entirely on the OneFormer3D backbone and decoder (Sections 3.1-3.3), assuming its feature space is compatible with CLIP projections.

how reviews work

0 comments
Cite this review

Pith. "Pith review of All in One: Visual-Description-Guided Unified Point Cloud Segmentation." pith.science (2026). https://pith.science/paper/PEMR7XRV

@misc{pith2026250705211,
  author       = {Pith},
  title        = {Pith review of: All in One: Visual-Description-Guided Unified Point Cloud Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PEMR7XRV}},
  note         = {Machine review of arXiv:2507.05211}
}
read the original abstract

Unified segmentation of 3D point clouds is crucial for scene understanding, but is hindered by its sparse structure, limited annotations, and the challenge of distinguishing fine-grained object classes in complex environments. Existing methods often struggle to capture rich semantic and contextual information due to limited supervision and a lack of diverse multimodal cues, leading to suboptimal differentiation of classes and instances. To address these challenges, we propose VDG-Uni3DSeg, a novel framework that integrates pre-trained vision-language models (e.g., CLIP) and large language models (LLMs) to enhance 3D segmentation. By leveraging LLM-generated textual descriptions and reference images from the internet, our method incorporates rich multimodal cues, facilitating fine-grained class and instance separation. We further design a Semantic-Visual Contrastive Loss to align point features with multimodal queries and a Spatial Enhanced Module to model scene-wide relationships efficiently. Operating within a closed-set paradigm that utilizes multimodal knowledge generated offline, VDG-Uni3DSeg achieves state-of-the-art results in semantic, instance, and panoptic segmentation, offering a scalable and practical solution for 3D understanding. Our code is available at https://github.com/Hanzy1996/VDG-Uni3DSeg.

Figures

Figures reproduced from arXiv: 2507.05211 by the authors.

Figure 1
Figure 1. Comparison between existing multi-modal 3D segmen [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the VDG-Uni3DSeg framework. First, we utilize a 3D encoder to extract point-wise features from the input point cloud. Then, the Spatial Enhancement Module refines these features by capturing local spatial relationships. Next, class-specific description queries are generated using LLMs, while image queries are constructed from reference images retrieved from the internet. Both description and image qu… view at source ↗
Figure 3
Figure 3. Qualitative results of semantic segmentation on ScanNet. Compared to OneFormer3D, the Description and Image Prediction [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The qualitative results of the instance segmentation on ScanNet. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: The qualitative results of the panoptic segmentation on ScanNet. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 47 canonical work pages

  1. [1]

    3d instance segmentation via enhanced spatial and semantic supervision

    Salwa Al Khatib, Mohamed El Amine Boudjoghra, Jean La- houd, and Fahad Shahbaz Khan. 3d instance segmentation via enhanced spatial and semantic supervision. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 541–550, 2023. 2

    work page 2023

  2. [2]

    3d seman- tic parsing of large-scale indoor spaces

    Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioan- nis Brilakis, Martin Fischer, and Silvio Savarese. 3d seman- tic parsing of large-scale indoor spaces. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 1534–1543, 2016. 5

    work page 2016

  3. [4]

    Hierarchical aggregation for 3d instance segmentation

    Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, and Xinggang Wang. Hierarchical aggregation for 3d instance segmentation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 15467–15476,

  4. [6]

    Per- pixel classification is not all you need for semantic segmen- tation

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per- pixel classification is not all you need for semantic segmen- tation. Advances in Neural Information Processing Systems, 34:17864–17875, 2021. 2

    work page 2021

  5. [7]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 2

    2022

  6. [8]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1290–1299, 2022. 2

    work page 2022

  7. [9]

    Generating long sequences with sparse transformers

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509, 2019. 3

    arXiv 1904

  8. [11]

    4d spatio-temporal convnets: Minkowski convolutional neural networks

    Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3075–3084,

Show all 60 references
  1. [12]

    MMDetection3D: Open- MMLab next-generation platform for general 3D object detection

    MMDetection3D Contributors. MMDetection3D: Open- MMLab next-generation platform for general 3D object detection. https://github.com/open- mmlab/ mmdetection3d, 2020. 5

    2020

  2. [13]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 5, 6

    2017

  3. [14]

    Pla: Language-driven open- vocabulary 3d scene understanding

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Pla: Language-driven open- vocabulary 3d scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7010–7019, 2023. 2

    2023

  4. [15]

    3d-mpa: Multi-proposal ag- gregation for 3d semantic instance segmentation

    Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, and Matthias Nießner. 3d-mpa: Multi-proposal ag- gregation for 3d semantic instance segmentation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9031–9040, 2020. 2

    2020

  5. [16]

    Occuseg: Occupancy-aware 3d instance segmentation

    Lei Han, Tian Zheng, Lan Xu, and Lu Fang. Occuseg: Occupancy-aware 3d instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2940–2949, 2020. 2

    2020

  6. [17]

    Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution

    Tong He, Chunhua Shen, and Anton Van Den Hen- gel. Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 354–363, 2021. 2, 6, 7

    2021

  7. [19]

    3d-sis: 3d se- mantic instance segmentation of rgb-d scans

    Ji Hou, Angela Dai, and Matthias Nießner. 3d-sis: 3d se- mantic instance segmentation of rgb-d scans. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4421–4430, 2019. 2

    2019

  8. [20]

    Odin: A single model for 2d and 3d perception

    Ayush Jain, Pushkal Katara, Nikolaos Gkanatsios, Adam W Harley, Gabriel Sarch, Kriti Aggarwal, Vishrav Chaudhary, and Katerina Fragkiadaki. Odin: A single model for 2d and 3d perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  9. [21]

    Oneformer: One transformer to rule universal image segmentation

    Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. Oneformer: One transformer to rule universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2989–2998, 2023. 3, 5

    2023

  10. [22]

    Pointgroup: Dual-set point grouping for 3d instance segmentation

    Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point grouping for 3d instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and Pattern recognition, pages 4867–4876, 2020. 6, 7

    2020

  11. [23]

    Pointgroup: Dual-set point grouping for 3d instance segmentation

    Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point grouping for 3d instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and Pattern recognition, pages 4867–4876, 2020. 2

    2020

  12. [24]

    Panoptic segmentation

    Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Doll ´ar. Panoptic segmentation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9404–9413, 2019. 5

    2019

  13. [25]

    Top-down beats bottom-up in 3d instance segmentation

    Maksim Kolodiazhnyi, Danila Rukhovich, Anna V orontsova, and Anton Konushin. Top-down beats bottom-up in 3d instance segmentation. arXiv preprint arXiv:2302.02871 ,

    arXiv

  14. [26]

    Oneformer3d: One transformer for unified point cloud segmentation

    Maxim Kolodiazhnyi, Anna V orontsova, Anton Konushin, and Danila Rukhovich. Oneformer3d: One transformer for unified point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  15. [27]

    Oneformer3d: One transformer for unified point cloud segmentation

    Maxim Kolodiazhnyi, Anna V orontsova, Anton Konushin, and Danila Rukhovich. Oneformer3d: One transformer for unified point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20943–20953, 2024. 1, 3, 4, 5, 6, 7, 8, 12

    2024

  16. [28]

    Virtual multi-view fusion for 3d semantic segmentation

    Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, and Caroline Pantofaru. Virtual multi-view fusion for 3d semantic segmentation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part...

    2020

  17. [29]

    3d instance segmentation via multi-task met- ric learning

    Jean Lahoud, Bernard Ghanem, Marc Pollefeys, and Mar- tin R Oswald. 3d instance segmentation via multi-task met- ric learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9256–9266, 2019. 2

    2019

  18. [30]

    Seggcn: Effi- cient 3d point cloud segmentation with fuzzy spherical ker- nel

    Huan Lei, Naveed Akhtar, and Ajmal Mian. Seggcn: Effi- cient 3d point cloud segmentation with fuzzy spherical ker- nel. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 11611–11620,

  19. [31]

    Instance segmentation in 3d scenes using semantic superpoint tree networks

    Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, and Kui Jia. Instance segmentation in 3d scenes using semantic superpoint tree networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2783– 2792, 2021. 2

    2021

  20. [32]

    Instance segmentation in 3d scenes using semantic superpoint tree networks

    Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, and Kui Jia. Instance segmentation in 3d scenes using semantic superpoint tree networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2783– 2792, 2021. 6, 7

    2021

  21. [33]

    Meta architecture for point cloud analysis

    Haojia Lin, Xiawu Zheng, Lijiang Li, Fei Chao, Shanshan Wang, Yan Wang, Yonghong Tian, and Rongrong Ji. Meta architecture for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17682–17691, 2023. 2, 7

    2023

  22. [34]

    Learning gaussian instance seg- mentation in point clouds

    Shih-Hung Liu, Shang-Yi Yu, Shao-Chi Wu, Hwann-Tzong Chen, and Tyng-Luh Liu. Learning gaussian instance seg- mentation in point clouds. arXiv preprint arXiv:2007.09860,

    2007 arXiv

  23. [35]

    Panopticfusion: Online volumetric semantic mapping at the level of stuff and things

    Gaku Narita, Takashi Seno, Tomoya Ishikawa, and Yohsuke Kaji. Panopticfusion: Online volumetric semantic mapping at the level of stuff and things. In 2019 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pages 4205–4212. IEEE, 2019. 2, 7

    2019

  24. [36]

    Isbnet: a 3d point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution

    Tuan Duc Ngo, Binh-Son Hua, and Khoi Nguyen. Isbnet: a 3d point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13550–13559, 2023. 7

    2023

  25. [37]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 815–824, 2023. 12

    2023

  26. [38]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017. 2, 6

    2017

  27. [39]

    Pointnext: Revisiting pointnet++ with improved training and scaling strategies

    Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Informa- tion Processing Systems, 35:23192–23204, 2022. 2, 6, 7

    2022

  28. [40]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  29. [41]

    Learn- ing multi-view aggregation in the wild for large-scale 3d se- mantic segmentation

    Damien Robert, Bruno Vallet, and Loic Landrieu. Learn- ing multi-view aggregation in the wild for large-scale 3d se- mantic segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5575–5584, 2022. 12

    2022

  30. [42]

    Language- grounded indoor 3d semantic segmentation in the wild

    David Rozenberszki, Or Litany, and Angela Dai. Language- grounded indoor 3d semantic segmentation in the wild. In Proceedings of the European Conference on Computer Vi- sion (ECCV), 2022. 5, 6

    2022

  31. [43]

    Language- grounded indoor 3d semantic segmentation in the wild

    David Rozenberszki, Or Litany, and Angela Dai. Language- grounded indoor 3d semantic segmentation in the wild. In European Conference on Computer Vision, pages 125–141. Springer, 2022. 7

    2022

  32. [44]

    Mask3D: Mask Trans- former for 3D Semantic Instance Segmentation

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3D: Mask Trans- former for 3D Semantic Instance Segmentation. In Inter- national Conference on Robotics and Automation (ICRA) ,

  33. [45]

    Mask3d: Mask trans- former for 3d semantic instance segmentation

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask trans- former for 3d semantic instance segmentation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 8216–8223. IEEE, 2023. 2, 6, 7

    2023

  34. [46]

    Superpoint transformer for 3d scene instance segmentation

    Jiahao Sun, Chunmei Qing, Junpeng Tan, and Xiangmin Xu. Superpoint transformer for 3d scene instance segmentation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, 2023. 2

    2023

  35. [47]

    Superpoint transformer for 3d scene instance segmentation

    Jiahao Sun, Chunmei Qing, Junpeng Tan, and Xiangmin Xu. Superpoint transformer for 3d scene instance segmentation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2393–2401, 2023. 6, 7

    2023

  36. [48]

    Neuralbf: Neural bilateral filtering for top- down instance segmentation on point clouds

    Weiwei Sun, Daniel Rebain, Renjie Liao, Vladimir Tankovich, Soroosh Yazdani, Kwang Moo Yi, and Andrea Tagliasacchi. Neuralbf: Neural bilateral filtering for top- down instance segmentation on point clouds. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of C...

    2023

  37. [49]

    Kpconv: Flexible and deformable convolution for point clouds

    Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Franc ¸ois Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6411–6420, 2019. 2, 6

    2019

  38. [50]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1, 5

    2023 arXiv

  39. [51]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 2

    2017

  40. [52]

    Softgroup for 3d instance segmentation on point clouds

    Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo. Softgroup for 3d instance segmentation on point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2708– 2717, 2022. 6, 7

    2022

  41. [53]

    Sgpn: Similarity group proposal network for 3d point cloud instance segmentation

    Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neu- mann. Sgpn: Similarity group proposal network for 3d point cloud instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2569–2578, 2018. 2

    2018

  42. [54]

    Scenegraphfusion: Incremen- tal 3d scene graph prediction from rgb-d sequences

    Shun-Cheng Wu, Johanna Wald, Keisuke Tateno, Nassir Navab, and Federico Tombari. Scenegraphfusion: Incremen- tal 3d scene graph prediction from rgb-d sequences. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7515–7525, 2021. 2, 7

    2021

  43. [55]

    Point transformer v2: Grouped vector atten- tion and partition-based pooling

    Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Heng- shuang Zhao. Point transformer v2: Grouped vector atten- tion and partition-based pooling. Advances in Neural Infor- mation Processing Systems, 35:33330–33342, 2022. 2, 6, 7

    2022

  44. [56]

    3d instances as 1d kernels

    Yizheng Wu, Min Shi, Shuaiyuan Du, Hao Lu, Zhiguo Cao, and Weicai Zhong. 3d instances as 1d kernels. In European Conference on Computer Vision , pages 235–252. Springer,

  45. [57]

    3d open-vocabulary panoptic segmentation with 2d-3d vision- language distillation

    Zihao Xiao, Longlong Jing, Shangxuan Wu, Alex Zihao Zhu, Jingwei Ji, Chiyu Max Jiang, Wei-Chih Hung, Thomas Funkhouser, Weicheng Kuo, Anelia Angelova, et al. 3d open-vocabulary panoptic segmentation with 2d-3d vision- language distillation. In European Conference on Computer V...

    2025

  46. [58]

    Paconv: Position adaptive convolution with dy- namic kernel assembling on point clouds

    Mutian Xu, Runyu Ding, Hengshuang Zhao, and Xiao- juan Qi. Paconv: Position adaptive convolution with dy- namic kernel assembling on point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3173–3182, 2021. 6

    2021

  47. [59]

    Learning ob- ject bounding boxes for 3d instance segmentation on point clouds

    Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning ob- ject bounding boxes for 3d instance segmentation on point clouds. Advances in neural information processing systems, 32, 2019. 2

    2019

  48. [60]

    Tupper-map: Temporal and uni- fied panoptic perception for 3d metric-semantic mapping

    Zhiliu Yang and Chen Liu. Tupper-map: Temporal and uni- fied panoptic perception for 3d metric-semantic mapping. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1094–1101. IEEE, 2021. 2, 7

    2021

  49. [61]

    Gspn: Generative shape proposal network for 3d instance segmentation in point cloud

    Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3947–3956, 2019. 2

    2019

  50. [62]

    Point cloud instance seg- mentation using probabilistic embeddings

    Biao Zhang and Peter Wonka. Point cloud instance seg- mentation using probabilistic embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 8883–8892, 2021. 2

    2021

  51. [63]

    Point transformer

    Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16259–16268, 2021. 2, 6, 7

    2021

  52. [64]

    Divide and conquer: 3d point cloud instance segmentation with point-wise binarization

    Weiguang Zhao, Yuyao Yan, Chaolong Yang, Jianan Ye, Xi Yang, and Kaizhu Huang. Divide and conquer: 3d point cloud instance segmentation with point-wise binarization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 562–571, 2023. 6, 7 Appendix ...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.