Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

Details Matter for Indoor Open-vocabulary 3D Instance Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A paper argues that combining existing building blocks with careful engineering — not a new architecture — produces state-of-the-art open-vocabulary 3D instance segmentation on indoor benchmarks.

desk verdict Solid systems recipe with large, mostly credible gains; the 3D-only SOTA claim rests on an unmatched Top-K comparison and needs a rerun. read the letter →

arxiv 2507.23134 v1 pith:JC2KM4PE submitted 2025-07-30 cs.CV

Sanghun Jung , Jingjing Zheng , Ke Zhang , Nan Qiao , Albert Y. C. Chen , Lu Xia , Chi Liu , Yuyin Sun
show 5 more authors
Xiao Zeng Hsiang-Wei Huang Byron Boots Min Sun Cheng-Hao Kuo
This is my paper
classification cs.CV
keywords open-vocabulary3Dinstancesegmentationtwo-stageproposalgenerationAlpha-CLIPStandardizedMaximumSimilarityframe-wisesuperpointIoUtrackingiterativemergingandremovalScanNet200S3DIS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary 3D instance segmentation asks a system to locate every object matching an arbitrary text query in a scanned indoor scene, including object categories it was never trained on. This paper's thesis is that the scattered ideas in existing methods — 2D foundation-model grounding, tracking-based 3D aggregation, and CLIP-style classification — are complementary, and that a carefully staged recipe combining them sets a new benchmark. Its pipeline generates proposals from both images and point clouds, removes overlapping and partial masks through iterative merging and refinement, and classifies with mask-aware Alpha-CLIP followed by a standardized maximum similarity filter. The reported result is state-of-the-art accuracy on ScanNet200 (32.7 mAP under the Top-K protocol, versus 23.7 for the prior best) and S3DIS (31.3 versus 28.9 mAP), with the largest gains on rare tail classes. The practical message is that fine-grained engineering details — overlap removal in 2D, frame-wise tracking, and score normalization — determine most of the performance.

What carries the argument

The argument runs on a chain of four interacting mechanisms. Frame-wise superpoint-level intersection over union (sIOU) tracking matches each new 2D observation against the per-frame instances already in a tracklet, considering only co-visible superpoints; this absorbs noisy detections that tracklet-wise matching would propagate. Iterative merging and removal then fuses tracklets whose proposal IoU exceeds the merge threshold, refines each merged mask with a multi-view consensus rate, and deletes smaller proposals whose inclusion rate in another proposal exceeds the inclusion threshold, curbing overlapped and partial masks. At classification, Alpha-CLIP injects the object mask as an alpha channel so visual features focus on the object rather than its surroundings, and the Standardized Maximum Similarity (SMS) score converts each query's raw CLIP similarities into scene-relative scores, rejecting proposals whose best match is not confident relative to the rest of the scene. Together these pieces do the work of turning noisy 2D foundation-model outputs into clean, classifiable 3D instances.

What would settle it

Re-run the ScanNet200 Top-K comparisons with every method capped at the same budget (for example K=300 for 3D-only systems) and re-run the ablations on the full 312-scene validation set; if a baseline then reaches or exceeds 32.7 mAP, the claimed margin disappears.

Watch

Extended reading notes

Core claim

The paper's central claim is that the two-stage paradigm — generate class-agnostic 3D proposals, then classify them with a vision-language model — is already the right architecture, and what separates a strong system from a weak one is the details at each stage. On the proposal side it combines image-based proposals (Grounded SAM masks lifted to 3D superpoints and aggregated by frame-wise superpoint-IoU tracking) with point-cloud proposals from pretrained 3D segmenters, then iteratively merges overlapping tracklets, refines them by multi-view consensus, and deletes proposals contained inside others. On the classification side it replaces CLIP with Alpha-CLIP, which consumes object masks as an extra alpha channel, and adds a Standardized Maximum Similarity (SMS) score computed per text query to reject low-confidence proposals. The authors report that this recipe reaches 32.7 mAP on ScanNet200 and 31.3 mAP on S3DIS under the Top-K protocol, outperforming previous open-vocabulary methods across all AP and AR metrics, and state that it even surpasses an end-to-end closed-vocabulary baseline in that setting.

Load-bearing premise

The headline margin rests on the assumption that every competitor is given the same number of allowed predictions per scene and that the ablation numbers from a subset of the validation set hold on the full benchmark; if either assumption fails, the reported gains may shrink.

Editorial extensions

If this is right

  • On ScanNet200, the combined pipeline raises mAP from 23.7 to 32.7 over the previous best and lifts tail-class mAP to 33.1, so rare object categories become substantially more findable.
  • On S3DIS, mAP rises from 28.9 to 31.3 and mAR from 44.1 to 48.2, showing that fusing image- and point-cloud-based proposals mainly buys recall.
  • The classification-stage changes alone add about 5.2 mAP on ScanNet200 (Alpha-CLIP +3.0, SMS filtering +2.2), indicating that open-vocabulary performance is not determined by proposal recall alone.
  • The same recipe transfers to synthetic Replica scenes, reaching 25.7 mAP and 48.8 mAR in the 2D+3D Top-K setting, so the method does not overfit one dataset.
  • Because image-only proposals match or beat previous full pipelines on several metrics, the 2D-grounding-to-3D-lifting chain is a reusable component for future systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The SMS score is a generic normalization: any retrieval pipeline whose scores are not calibrated across queries could apply per-query standardization without retraining, a transfer the paper does not explore.
  • The modular design means the recipe should keep improving as 2D grounding models and vision-language encoders improve; the reported numbers are tied to Grounded SAM and Alpha-CLIP, not to the overall architecture.
  • The paper's own ScanNet++ result shows the iterative merge/removal step helps less on small objects, so a size-adaptive merging threshold or a minimum-proposal-size guard is the natural follow-up.
  • At roughly 597 seconds per scene on a single GPU, most of it in 2D grounding and classification, the pipeline is a strong offline benchmark rather than a real-time system; speeding up or distilling those stages is an open test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a two-stage open-vocabulary 3D instance segmentation pipeline. In the proposal stage it combines image-based proposals from Grounded SAM with point-cloud proposals from Mask3D/ISBNet, introducing overlap removal in 2D, superpoint lifting with visibility filtering, frame-wise sIOU tracking, iterative merging/removal with multi-view refinement, and inclusion-based removal. In the classification stage it replaces CLIP with Alpha-CLIP and introduces a Standardized Maximum Similarity (SMS) score for false-positive filtering. Experiments on ScanNet200, S3DIS, and Replica report large gains over existing OV-3DIS methods (for example, 32.7 vs. 23.7 mAP on ScanNet200 under Top-K 2D+3D), and the abstract claims to surpass a closed-vocabulary method. The paper includes detailed pseudocode, a computation-time analysis, and sensitivity studies.

Significance. If the results hold under a matched comparison protocol, the paper is a substantial empirical contribution: it demonstrates that carefully combining existing concepts (2D grounding, tracking-based aggregation, iterative merging/removal, Alpha-CLIP) yields large and consistent gains over prior open-vocabulary 3D instance segmentation systems across multiple benchmarks. The ablations attribute specific gains to individual components, and the algorithm pseudocode and sensitivity analysis are useful for reproducibility. The main risks are that the 3D-only state-of-the-art claim rests on a Top-K budget mismatch, and that some ablations are reported on an unspecified subset of the validation set; both are addressable without changing the core method.

major comments (3)
  1. [Table 1 / Supplement B / Sec. 4.2] The Top-K 3D-only comparison on ScanNet200 uses K=300 for the proposed method and K=600 for OpenYOLO3D, as stated in Supplement B. Table 1 reports 29.0 vs. 24.7 mAP for this row, and Sec. 4.2 attributes the 4.3-point gap entirely to the classification module while noting that the proposals are identical. Because the Top-K budget changes which low-confidence proposal-class pairs are retained, the AP difference is confounded with the budget mismatch. Please rerun the comparison under a matched budget (both K=300 or both K=600) and report both outcomes, and do not claim a classification-only improvement until the budgets are matched.
  2. [Abstract and Table 1] The abstract claims the framework 'even surpasses an end-to-end closed-vocabulary method,' but under the Top-1 protocol in Table 1 Mask3D reaches 26.9 mAP while Ours (2D+3D) reaches 25.8 mAP. The claim only holds under the Top-K protocol, which is a different evaluation setting from the one used for closed-vocabulary methods. Please qualify the claim to state the protocol, or remove the claim from the abstract.
  3. [Tables 5, 6, and 10] The ablation studies for tracklet matching, iterative merging/removal with overlap removal and refinement, and hyperparameter sensitivity are reported on 'the subset of the ScanNet200 validation set,' but the subset is not specified (size, selection criterion, or scene list). Without this information, the component contributions are not reproducible and may not generalize to the full benchmark. Please specify the subset, or better, report these ablations on the full validation set; otherwise the corresponding conclusions should be limited accordingly.
minor comments (5)
  1. [Supplement B] The exact SMS threshold values (τ_SMS) used for each experiment are never listed; 'we adopt different SMS thresholds tailored to each experiment' is not sufficient for reproducibility. Please include the exact thresholds or state the selection rule.
  2. [Sec. 3.1, Eq. (2)] The sIOU definition is undefined when the two instance mask sets have no co-visible superpoints; please specify the convention (for example, set the value to 0) for that case.
  3. [Sec. 3.1] The overlap-removal step is described in a single sentence, despite being a key component in the Table 6 ablation; a precise algorithmic description or pseudocode would improve reproducibility.
  4. [Supplement D.1, Table 10] The hyperparameter sensitivity table does not clearly indicate which column corresponds to the default configuration used in the main experiments; please add a marker or caption note.
  5. [Figures 11 and 12 captions] The word 'Exteneded' should be 'Extended' in the captions of Figures 11 and 12.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the paper is an empirical system whose benchmark claims rest on measured outputs, not on definitions or self-citation chains.

full rationale

The paper proposes an engineering recipe for open-vocabulary 3D instance segmentation: it combines existing 2D grounding, 3D lifting/tracking, merging/removal, and Alpha-CLIP classification. There is no claimed first-principles derivation chain whose conclusion equals its input by construction. The SMS score is defined as a per-query z-score of the CLIP similarity matrix (Sec. 3.2) and is used as a filtering rule with a tuned threshold; it is not a fitted constant renamed as a prediction. Hyperparameters such as τ_img, τ_inst, τ_tracking, τ_merge, τ_ref, τ_incl, and τ_SMS are tuned on validation sets, which is standard practice and does not make the reported mAP values circular. Self-citations [6] and [21] appear only to motivate a dimension-reduction trick for the Replica domain gap; they are not load-bearing for the main SOTA claim, and no uniqueness theorem is imported from the authors' prior work. The most serious issue in the paper, the Top-K budget mismatch noted in Supplement B (K=300 for Ours vs K=600 for OpenYOLO3D in the ScanNet200 3D-only row), is a comparison-protocol fairness concern, not a circularity: the paper's numbers are not derived from OpenYOLO3D's outputs or from its own assumptions in a way that forces the reported gap. Since no quoted step reduces to its own input by definition, the derivation is self-contained and the circularity score is 0.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a set of hand-set thresholds, reliance on several pre-trained models, and consistency of the evaluation protocol. No new physical or formal entities are introduced; the SMS score is a normalization of existing similarity scores.

free parameters (8)
  • tau_img = 0.1
    Visibility threshold for 3D superpoints in Eq. (1); chosen on validation.
  • tau_inst = 0.3
    Instance support threshold in Eq. (1); chosen on validation.
  • tau_tracking = 0.3
    sIOU threshold for tracklet matching in Fig. 3; chosen on validation.
  • tau_merge = 0.3 (0.7 for Replica)
    IOU threshold for merging 3D proposals; adjusted per dataset.
  • tau_ref = 0.4
    Multi-view consensus threshold for refinement; disabled on Replica.
  • tau_incl = 0.99
    Inclusion ratio threshold for removing partial proposals.
  • tau_SMS = per dataset, within [-1,1]
    Threshold for SMS-based filtering; tailored to each experiment.
  • Top-K = 300 for 2D-only and 3D-only, 600 for 2D+3D
    Number of top class predictions kept for evaluation; follows prior work but mismatched for OpenYOLO3D in the 3D-only setting.
assumptions (4)
  • domain assumption Grounded SAM reliably detects and segments objects in indoor RGB images across the target datasets.
    Used for all 2D grounding; failures propagate directly to 3D proposals.
  • domain assumption Pre-trained Mask3D and ISBNet provide class-agnostic 3D proposals that are useful for open-vocabulary classification.
    The 3D-only and 2D+3D settings rely on these models.
  • domain assumption Alpha-CLIP features are more object-centric than CLIP for projected 3D masks, and SAM-generated masks on the projections are accurate enough.
    The main classification improvement rests on this premise.
  • domain assumption The ScanNet200 and S3DIS validation annotations are treated as ground truth for tuning and evaluation.
    All metrics are computed against these labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Details Matter for Indoor Open-vocabulary 3D Instance Segmentation." pith.science (2026). https://pith.science/paper/JC2KM4PE

@misc{pith2026250723134,
  author       = {Pith},
  title        = {Pith review of: Details Matter for Indoor Open-vocabulary 3D Instance Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JC2KM4PE}},
  note         = {Machine review of arXiv:2507.23134}
}
read the original abstract

Unlike closed-vocabulary 3D instance segmentation that is often trained end-to-end, open-vocabulary 3D instance segmentation (OV-3DIS) often leverages vision-language models (VLMs) to generate 3D instance proposals and classify them. While various concepts have been proposed from existing research, we observe that these individual concepts are not mutually exclusive but complementary. In this paper, we propose a new state-of-the-art solution for OV-3DIS by carefully designing a recipe to combine the concepts together and refining them to address key challenges. Our solution follows the two-stage scheme: 3D proposal generation and instance classification. We employ robust 3D tracking-based proposal aggregation to generate 3D proposals and remove overlapped or partial proposals by iterative merging/removal. For the classification stage, we replace the standard CLIP model with Alpha-CLIP, which incorporates object masks as an alpha channel to reduce background noise and obtain object-centric representation. Additionally, we introduce the standardized maximum similarity (SMS) score to normalize text-to-proposal similarity, effectively filtering out false positives and boosting precision. Our framework achieves state-of-the-art performance on ScanNet200 and S3DIS across all AP and AR metrics, even surpassing an end-to-end closed-vocabulary method.

Figures

Figures reproduced from arXiv: 2507.23134 by the authors.

Figure 1
Figure 1. Examples of open-vocabulary predictions from our method in the ScanNet200 dataset [7]. Our method effectively retrieves instances based on functional descriptions (e.g., drink water, heat mac & cheese) and object attributes (e.g., red chair). dicted proposals into open-vocabulary queries. While some works [2, 51] directly generate 3D proposals from point clouds using pretrained 3D networks [40, 47], other ap￾proache… view at source ↗
Figure 2
Figure 2. Overview of image-based 3D proposal generation. We first remove overlaps between 2D predictions within each frame and lift them to 3D point cloud using a camera projection matrix. Afterward, we aggregate 3D-lifted predictions across frames using a frame-wise sIOU metric with tracking. These 3D proposals are further iteratively merged and refined to progressively merge similar proposals based on a predefined threshol… view at source ↗
Figure 4
Figure 4. Effectiveness of 3D proposal refinement. Red boxes indicate the object of interest, and segments of different colors de￾note 3D superpoints. Without refinement, the 3D instance proposal often extends beyond the object boundaries due to noisy 2D-to-3D projections or inaccurate mask predictions. With refinement, irrel￾evant 3D superpoints are removed, and our method successfully removes 3D superpoints that do not belo… view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Visualization of merged and removed proposals in the ScanNet200 dataset. Overlapping and noisy proposals often emerge after instance tracking. We effectively handle these issues by merging duplicate proposals and eliminating noisy ones, ensur￾ing high-quality proposals…
Figure 7
Figure 7. Figure 7: Qualitative comparisons on the ScanNet200 dataset. Black regions indicate empty predictions (no object), while red boxes highlight objects missed by other methods but successfully detected by ours. 3D instance masks are colored randomly. Methods mAP mAP50 mAP25 mAR mAR…
Figure 8
Figure 8. Figure 8: Computation time analysis of various factors on the subset of ScanNet200 validation set. of distributional gap when we apply 3D instance segmenta￾tion models trained on other datasets such as ScanNet200. As reported in [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 10
Figure 10. Figure 10: Impacts of varying SMS filtering thresholds on the AP metric. The red line denotes the AP values across different SMS filtering thresholds, and the blue line indicates AP without using SMS filtering. The green vertical lines indicate a desirable range of SMS filtering…
Figure 11
Figure 11. Figure 11: Exteneded qualitative comparisons on the ScanNet200 dataset. Black regions indicate empty predictions (no object), while red boxes highlight objects missed by other methods but successfully detected by ours. 3D instance masks are colored randomly. 16 [PITH_FULL_IMAGE…
Figure 12
Figure 12. Figure 12: Exteneded qualitative comparisons on the ScanNet200 dataset. Black regions indicate empty predictions (no object), while red boxes highlight objects missed by other methods but successfully detected by ours. 3D instance masks are colored randomly. Ground Truth Ours Gr…
Figure 13
Figure 13. Figure 13: Qualitative results of our method on the Replica [49] (left) and S3DIS [1] (right) datasets. Black regions indicate empty predictions (no object). 3D instance masks are colored randomly. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14 [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SpaCeFormer: Fast Proposal-Free Open-Vocabulary 3D Instance Segmentation

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    SpaCeFormer delivers 11.1 zero-shot mAP on ScanNet200 (2.8x prior proposal-free best) and runs 2-3 orders of magnitude faster than multi-stage 2D+3D pipelines by using spatial window attention and Morton-curve seriali...

  2. SpaCeFormer: Fast Proposal-Free Open-Vocabulary 3D Instance Segmentation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    A single-antenna WiFi CSI system uses a two-path reflection model, BNR subcarrier selection, spectral clustering, and frequency-trend phase ID to reconstruct respiratory waveforms and biomarkers with location robustness.

  3. Cross-Attentive Multiview Fusion of Vision-Language Embeddings

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    CAMFusion fuses multiview 2D vision-language embeddings via cross-attention and multiview consistency self-supervision to produce better 3D semantic and instance representations, outperforming averaging and reaching S...

  4. MV3DIS: Multi-View Mask Matching via 3D Guides for Zero-Shot 3D Instance Segmentation

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    MV3DIS uses 3D-guided mask matching and depth consistency to produce more consistent multi-view 2D masks that refine into accurate zero-shot 3D instances.

Reference graph

Works this paper leans on

71 extracted references · 52 canonical work pages · cited by 3 Pith papers

  1. [1]

    3d seman- tic parsing of large-scale indoor spaces

    Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioan- nis Brilakis, Martin Fischer, and Silvio Savarese. 3d seman- tic parsing of large-scale indoor spaces. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 1534–1543, 2016. 6, 7, 14, 17

    work page 2016

  2. [2]

    Open-yolo 3d: Towards fast and accurate open-vocabulary 3d instance segmentation

    Mohamed El Amine Boudjoghra, Angela Dai, Jean Lahoud, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, and Fahad Shahbaz Khan. Open-yolo 3d: Towards fast and accurate open-vocabulary 3d instance segmentation. arXiv preprint arXiv:2406.02548, 2024. 1, 3, 6, 7, 12, 14

    arXiv 2024

  3. [3]

    Zero-shot semantic segmentation.Advances in Neural Information Processing Systems, 32, 2019

    Maxime Bucher, Tuan-Hung Vu, Matthieu Cord, and Patrick P´erez. Zero-shot semantic segmentation.Advances in Neural Information Processing Systems, 32, 2019. 2

    2019

  4. [4]

    Hierarchical aggregation for 3d instance segmentation

    Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, and Xinggang Wang. Hierarchical aggregation for 3d instance segmentation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 15467–15476,

    work page

  5. [5]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xing- gang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16901–16911, 2024. 2, 3

    work page 2024

  6. [6]

    Robustnet: Improving domain generalization in urban-scene segmentation via in- stance selective whitening

    Sungha Choi, Sanghun Jung, Huiwon Yun, Joanne T Kim, Seungryong Kim, and Jaegul Choo. Robustnet: Improving domain generalization in urban-scene segmentation via in- stance selective whitening. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11580–11590, 2021. 6

    work page 2021

  7. [7]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 1, 6, 8, 14

    work page 2017

  8. [8]

    Pla: Language-driven open- vocabulary 3d scene understanding

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Pla: Language-driven open- vocabulary 3d scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7010–7019, 2023. 1

    work page 2023

Show all 71 references
  1. [9]

    Lowis3d: Language-driven open-world instance-level 3d scene understanding

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Lowis3d: Language-driven open-world instance-level 3d scene understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  2. [10]

    Learning regional purity for instance segmentation on 3d point clouds

    Shichao Dong, Guosheng Lin, and Tzu-Yi Hung. Learning regional purity for instance segmentation on 3d point clouds. In European Conference on Computer Vision, pages 56–72. Springer, 2022. 2

    2022

  3. [11]

    3d-mpa: Multi-proposal aggre- gation for 3d semantic instance segmentation

    Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, and Matthias Nießner. 3d-mpa: Multi-proposal aggre- gation for 3d semantic instance segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9031–9040, 2020. 2

    2020

  4. [12]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    Martin Ester, Hans-Peter Kriegel, J ¨org Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, pages 226–231,

  5. [13]

    Efficient graph-based image segmentation

    Pedro F Felzenszwalb and Daniel P Huttenlocher. Efficient graph-based image segmentation. International journal of computer vision, 59:167–181, 2004. 2, 8

    2004

  6. [14]

    Scal- ing open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision, pages 540–557. Springer, 2022. 2

    2022

  7. [15]

    Open- vocabulary object detection via vision and language knowl- edge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open- vocabulary object detection via vision and language knowl- edge distillation. arXiv preprint arXiv:2104.13921, 2021. 2

    2021 arXiv

  8. [16]

    Sam-guided graph cut for 3d instance segmentation

    Haoyu Guo, He Zhu, Sida Peng, Yuang Wang, Yujun Shen, Ruizhen Hu, and Xiaowei Zhou. Sam-guided graph cut for 3d instance segmentation. In European Conference on Com- puter Vision, pages 234–251. Springer, 2024. 13

    2024

  9. [17]

    Occuseg: Occupancy-aware 3d instance segmentation

    Lei Han, Tian Zheng, Lan Xu, and Lu Fang. Occuseg: Occupancy-aware 3d instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2940–2949, 2020. 2

    2020

  10. [18]

    Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution

    Tong He, Chunhua Shen, and Anton Van Den Hengel. Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 354–363, 2021. 2

    2021

  11. [19]

    Clip goes 3d: Leveraging prompt tuning for language grounded 3d recognition

    Deepti Hegde, Jeya Maria Jose Valanarasu, and Vishal Pa- tel. Clip goes 3d: Leveraging prompt tuning for language grounded 3d recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2028– 2038, 2023. 3

    2023

  12. [20]

    3d-sis: 3d se- mantic instance segmentation of rgb-d scans

    Ji Hou, Angela Dai, and Matthias Nießner. 3d-sis: 3d se- mantic instance segmentation of rgb-d scans. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4421–4430, 2019. 2

    2019

  13. [21]

    Reclip: Refine contrastive language image pre-training with source free domain adaptation

    Xuefeng Hu, Ke Zhang, Lu Xia, Albert Chen, Jiajia Luo, Yuyin Sun, Ken Wang, Nan Qiao, Xiao Zeng, Min Sun, et al. Reclip: Refine contrastive language image pre-training with source free domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Compu...

    2024

  14. [22]

    Decorre- lated batch normalization

    Lei Huang, Dawei Yang, Bo Lang, and Jia Deng. Decorre- lated batch normalization. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 791–800, 2018. 6

    2018

  15. [23]

    Segment3d: Learning fine-grained class-agnostic 3d segmentation without manual labels

    Rui Huang, Songyou Peng, Ayca Takmaz, Federico Tombari, Marc Pollefeys, Shiji Song, Gao Huang, and Francis Engel- mann. Segment3d: Learning fine-grained class-agnostic 3d segmentation without manual labels. In European Confer- ence on Computer Vision , pages 278–295. Springer,...

    2024

  16. [24]

    Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation

    Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, and Joan Lasenby. Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation. arXiv preprint arXiv:2309.00616, 2023. 1, 3, 6

    2023 arXiv

  17. [25]

    Con- 9 ceptfusion: Open-set multimodal 3d mapping.arXiv preprint arXiv:2302.07241, 2023

    Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Alaa Maalouf, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, et al. Con- 9 ceptfusion: Open-set multimodal 3d mapping.arXiv preprint arXiv:2302.07241, 2023. 3

    2023 arXiv

  18. [26]

    Dynamic filter networks

    Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Luc V Gool. Dynamic filter networks. Advances in neural informa- tion processing systems, 29, 2016. 2

    2016

  19. [27]

    Pointgroup: Dual-set point group- ing for 3d instance segmentation

    Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point group- ing for 3d instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and Pattern recognition, pages 4867–4876, 2020. 2

    2020

  20. [28]

    Multi- modal classifiers for open-vocabulary object detection

    Prannay Kaul, Weidi Xie, and Andrew Zisserman. Multi- modal classifiers for open-vocabulary object detection. InIn- ternational Conference on Machine Learning, pages 15946– 15969. PMLR, 2023. 2

    2023

  21. [29]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 19729–19739,

  22. [30]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 202...

    2023

  23. [31]

    Oneformer3d: One transformer for unified point cloud segmentation

    Maxim Kolodiazhnyi, Anna V orontsova, Anton Konushin, and Danila Rukhovich. Oneformer3d: One transformer for unified point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20943–20953, 2024. 2

    2024

  24. [32]

    3d instance segmentation via multi-task met- ric learning

    Jean Lahoud, Bernard Ghanem, Marc Pollefeys, and Mar- tin R Oswald. 3d instance segmentation via multi-task met- ric learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9256–9266, 2019. 2

    2019

  25. [33]

    Seg- ment any 3d object with language

    Seungjun Lee, Yuyang Zhao, and Gim Hee Lee. Seg- ment any 3d object with language. arXiv preprint arXiv:2404.02157, 2024. 3

    2024 arXiv

  26. [34]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. arXiv preprint arXiv:2201.03546, 2022. 2

    2022 arXiv

  27. [35]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pag...

    2023

  28. [36]

    Instance segmentation in 3d scenes using semantic superpoint tree networks

    Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, and Kui Jia. Instance segmentation in 3d scenes using semantic superpoint tree networks. In Proceedings of the IEEE/CVF international conference on computer vision , pages 2783– 2792, 2021. 2

    2021

  29. [37]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 2, 3

    2023 arXiv

  30. [38]

    Learning gaussian instance seg- mentation in point clouds

    Shih-Hung Liu, Shang-Yi Yu, Shao-Chi Wu, Hwann-Tzong Chen, and Tyng-Luh Liu. Learning gaussian instance seg- mentation in point clouds. arXiv preprint arXiv:2007.09860,

    2007 arXiv

  31. [39]

    Ovir-3d: Open-vocabulary 3d in- stance retrieval without training on 3d data

    Shiyang Lu, Haonan Chang, Eric Pu Jing, Abdeslam Boular- ias, and Kostas Bekris. Ovir-3d: Open-vocabulary 3d in- stance retrieval without training on 3d data. In Conference on Robot Learning, pages 1610–1620. PMLR, 2023. 1, 2, 3, 4, 6, 7, 8

    2023

  32. [40]

    Isbnet: a 3d point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution

    Tuan Duc Ngo, Binh-Son Hua, and Khoi Nguyen. Isbnet: a 3d point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13550–13559, 2023. ...

    2023

  33. [41]

    Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance

    Phuc Nguyen, Tuan Duc Ngo, Evangelos Kalogerakis, Chuang Gan, Anh Tran, Cuong Pham, and Khoi Nguyen. Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 4018–402...

    2024

  34. [42]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 815–824, 2023. 3, 6

    2023

  35. [43]

    Lp-ovod: Open- vocabulary object detection by linear probing

    Chau Pham, Truong Vu, and Khoi Nguyen. Lp-ovod: Open- vocabulary object detection by linear probing. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 779–788, 2024. 2

    2024

  36. [44]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  37. [45]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 ,

    arXiv

  38. [46]

    Unsuper- vised domain adaptation using feature-whitening and con- sensus loss

    Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, and Elisa Ricci. Unsuper- vised domain adaptation using feature-whitening and con- sensus loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9471–9480,

  39. [47]

    Mask3d: Mask trans- former for 3d semantic instance segmentation

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask trans- former for 3d semantic instance segmentation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 8216–8223. IEEE, 2023. 1, 2, 3, 6

    2023

  40. [48]

    Whitening and coloring batch transform for gans

    Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening and coloring batch transform for gans. arXiv preprint arXiv:1806.00420, 2018. 6

    2018 arXiv

  41. [49]

    The replica dataset: A digital replica of indoor spaces

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl 10 Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797,

    1906 arXiv

  42. [50]

    Alpha- clip: A clip model focusing on wherever you want

    Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Alpha- clip: A clip model focusing on wherever you want. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13019–13029, 2024. 2, 5, 8, 12

    2024

  43. [51]

    Open- mask3d: Open-vocabulary 3d instance segmentation

    Ayc ¸a Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Open- mask3d: Open-vocabulary 3d instance segmentation. arXiv preprint arXiv:2306.13631, 2023. 1, 2, 3, 5, 6, 7, 12

    2023 arXiv

  44. [52]

    Conditional con- volutions for instance segmentation

    Zhi Tian, Chunhua Shen, and Hao Chen. Conditional con- volutions for instance segmentation. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part I 16 , pages 282–298. Springer, 2020. 2

    2020

  45. [53]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2

    2017

  46. [54]

    Softgroup for 3d instance segmentation on point clouds

    Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo. Softgroup for 3d instance segmentation on point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2708– 2717, 2022. 2

    2022

  47. [55]

    Object-aware dis- tillation pyramid for open-vocabulary object detection

    Luting Wang, Yi Liu, Penghui Du, Zihan Ding, Yue Liao, Qiaosong Qi, Biaolong Chen, and Si Liu. Object-aware dis- tillation pyramid for open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 11186–11196, 2023. 2

    2023

  48. [56]

    Sgpn: Similarity group proposal network for 3d point cloud instance segmentation

    Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neu- mann. Sgpn: Similarity group proposal network for 3d point cloud instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2569–2578, 2018. 2

    2018

  49. [57]

    Groupvit: Semantic segmentation emerges from text supervision

    Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 18134–18144, 2022. 2

    2022

  50. [58]

    Open-vocabulary panop- tic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiao- long Wang, and Shalini De Mello. Open-vocabulary panop- tic segmentation with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 2955–2966, 2023. 2

    2023

  51. [59]

    Maskclus- tering: View consensus based mask graph clustering for open-vocabulary 3d instance segmentation

    Mi Yan, Jiazhao Zhang, Yan Zhu, and He Wang. Maskclus- tering: View consensus based mask graph clustering for open-vocabulary 3d instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 28274–28284, 2024. 1, 2, 3, 4, 8

    2024

  52. [60]

    Learning ob- ject bounding boxes for 3d instance segmentation on point clouds

    Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning ob- ject bounding boxes for 3d instance segmentation on point clouds. Advances in neural information processing systems, 32, 2019. 2

    2019

  53. [61]

    Regionplc: Regional point-language contrastive learning for open-world 3d scene understanding

    Jihan Yang, Runyu Ding, Weipeng Deng, Zhe Wang, and Xi- aojuan Qi. Regionplc: Regional point-language contrastive learning for open-world 3d scene understanding. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19823–19832, 2024. 1

    2024

  54. [62]

    Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023. 2, 6, 13

    2023 arXiv

  55. [63]

    Detclipv2: Scalable open- vocabulary object detection pre-training via word-region alignment

    Lewei Yao, Jianhua Han, Xiaodan Liang, Dan Xu, Wei Zhang, Zhenguo Li, and Hang Xu. Detclipv2: Scalable open- vocabulary object detection pre-training via word-region alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23497– 2...

    2023

  56. [64]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. In Proceedings of the International Confer- ence on Computer Vision (ICCV), 2023. 13

    2023

  57. [65]

    Gspn: Generative shape proposal network for 3d instance segmentation in point cloud

    Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3947–3956, 2019. 2

    2019

  58. [66]

    Sai3d: Segment any in- stance in 3d scenes

    Yingda Yin, Yuzheng Liu, Yang Xiao, Daniel Cohen-Or, Jingwei Huang, and Baoquan Chen. Sai3d: Segment any in- stance in 3d scenes. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3292–3302, 2024. 1, 2, 3, 6, 8, 13

    2024

  59. [67]

    Open-vocabulary detr with conditional matching

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-vocabulary detr with conditional matching. In European Conference on Computer Vision , pages 106–122. Springer, 2022. 2

    2022

  60. [68]

    Regionclip: Region- based language-image pretraining

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chun- yuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region- based language-image pretraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2022

  61. [69]

    Detecting twenty-thousand classes using image-level supervision

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Kr¨ahenb¨uhl, and Ishan Misra. Detecting twenty-thousand classes using image-level supervision. In European Confer- ence on Computer Vision, pages 350–368. Springer, 2022. 1, 2

    2022

  62. [70]

    a blurry photo of {CLASS NAME} in a room,

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. Advances in Neural Information Processing Systems, 36, 2024. 2 11 A. Algorithm on 3D Proposal Merge and Re- finement ...

    2024

  63. [71]

    stuff” Classes. We present re- sults that include

    the number of image frames, 3) the number of instances, and 4) the number of different semantic classes present in the scene. At last, we show the stage-wise computation time of our method. As shown, we can see meaningful corre- lations between those factors and computation ti...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.