Pith. sign in

REVIEW 2 major objections 8 minor 68 references

Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation

T0 review · 2 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Relation3D claims that adding explicit relation modeling at the superpoint level and between instance queries improves point cloud instance segmentation beyond current transformer-based methods, reporting gains on ScanNetV2, ScanNet++…

desk verdict Solid incremental gains in 3D instance segmentation with an unvalidated superpoint assumption that needs addressing before full acceptance. read the letter →

arxiv 2506.17891 v1 pith:BIVVLMNE submitted 2025-06-22 cs.CV

Jiahao Lu , Jiacheng Deng This is my paper · ORCID
classification cs.CV
keywords pointcloudinstancesegmentationsuperpointaggregationcontrastivelearningrelation-awareself-attentiontransformerdecoder3DsceneunderstandingScanNetV2S3DIS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

3D instance segmentation means labeling every object in a scanned scene with a class and a per-object mask. Relation3D argues that transformer-based systems for this task neglect two internal relations: those among the precomputed point clusters called superpoints, and those among instance queries. To fix this, it adds an adaptive superpoint aggregation module, a contrastive-learning-guided refinement module that pulls superpoint features of the same instance together and pushes different instances apart, and a relation-aware self-attention that adds position and geometry biases between queries. The paper reports consistent gains over other transformer-based methods on ScanNetV2, ScanNet++, ScanNet200, and S3DIS; on ScanNetV2 validation it reports mAP 62.5, AP@50 80.2, and AP@25 87.0. The claim matters because relation modeling could let an end-to-end query-based segmenter resolve nearby, similar-looking objects without extra proposal or grouping stages.

What carries the argument

The machinery is the superpoint with its instance-relation prior. ASAM aggregates point features into superpoints by predicting per-point weights from max-pooling and mean-pooling differences, so boundaries and corners get higher weight. CLSR then refines superpoint features in a cross-attention path every few decoder layers, and a contrastive loss drives the normalized superpoint similarity matrix toward a ground-truth instance relation matrix. RSA takes each query's predicted mask, computes its bounding box, builds six log features (three center offsets scaled by query size and three size ratios), encodes them, and adds the embedding to the self-attention logits. The ablation sequence in the paper shows these components together moving the validation mAP from 59.8 to 62.5.

What would settle it

Measure the fraction of superpoints that contain points from more than one ground-truth instance on ScanNetV2, or rerun Relation3D after deliberately merging superpoints across object boundaries; if the reported gains persist when superpoints are impure, the homogeneous-superpoint assumption is not what carries the result.

Watch

Extended reading notes

Core claim

The central claim is that pair-wise relation priors belong inside the superpoint features and between instance queries, not only between scene and query through mask attention. A superpoint is a precomputed cluster of points that should belong to a single surface or object. Relation3D replaces plain pooling with learned per-point weights that emphasize boundaries and corners, then refines superpoint features in a decoder-side cross-attention path in which superpoints attend to instance queries; a contrastive loss measures the normalized superpoint similarity matrix against an instance-derived relation prior, pulling same-instance superpoints together and pushing different-instance superpoints apart. On the query side, relation-aware self-attention computes each pair of queries' relative mask-box center distances and size ratios, encodes them with sinusoidal encoding, and adds the resulting bias to the attention logits. On ScanNetV2 validation the paper reports mAP 62.5, AP@50 80.2, and AP@25 87.0, with higher numbers than the Maft baseline on the hidden test set, and gains on ScanNet++, ScanNet200, and S3DIS.

Load-bearing premise

The method depends on the preprocessing step that groups points into superpoints being accurate: if a superpoint contains points from two different objects, the contrastive signal and the aggregation weights get contradictory information.

Editorial extensions

If this is right

  • The ablation sequence shows the three modules are complementary: ASAM alone, CLSR alone, and RSA alone each add a gain, and the full combination gives the largest improvement.
  • The relation priors remain useful as the number of categories and instances grows, which is where the reported gains on ScanNet++ and ScanNet200 are largest.
  • The reported convergence curves and attention maps show the relation priors pull attention toward relevant queries earlier in training.
  • Compared with the Maft baseline, Relation3D adds about 2.0M parameters and a modest runtime increase, so the gains are not bought by a substantially larger model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same relation-aware attention bias could be applied to 2D mask transformers, because the positional and geometric features are computed from predicted masks rather than from 3D structure.
  • Beyond the paper, the contrastive relation prior could be produced from the model's own predicted masks at test time, making superpoint refinement self-supervised and removing the need for ground-truth instance labels in this module.
  • Beyond the paper, a soft relation prior encoding geometric compatibility such as coplanarity, symmetry, or containment would test whether the method's gains come specifically from the binary same-instance relation or from any informative pairwise signal.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. The paper proposes Relation3D, a transformer-based 3D instance segmentation method that introduces three components: an adaptive superpoint aggregation module (ASAM), a contrastive learning-guided superpoint refinement module (CLSR), and relation-aware self-attention (RSA). The authors claim state-of-the-art results on ScanNetV2, ScanNet++, ScanNet200, and S3DIS, with ablations showing that each module contributes to the final performance. The work is positioned as addressing the lack of explicit relation modeling among scene features and among instance queries in existing transformer-based methods.

Significance. If the reported results are reproducible, the paper makes a solid empirical contribution: it evaluates on four benchmarks, provides per-category breakdowns in the supplementary material, reports runtime/parameter analysis, and includes ablations for each proposed module. The convergence analysis and weight visualizations give additional insight into why the components help. However, the central modules rely on an unvalidated superpoint input, the code link is a placeholder, and the headline improvement numbers are computed against inconsistent baselines. These issues limit the verifiability of the claimed state of the art.

major comments (2)
  1. [3.3-3.4 (Eqs. 3-5)] The contrastive relation prior and ASAM both assume that each superpoint is internally homogeneous with respect to instance identity, but the manuscript does not describe how superpoints are generated, what geometric partition parameters are used, or how many superpoints result per scene. If a superpoint straddles two ground-truth instances or an instance and background, the binary relation in Eq. (3) is ill-defined and the BCE loss in Eq. (5) gives conflicting supervision. This is load-bearing because Table 6 shows that ASAM+CLSR contribute 1.7 mAP over the baseline (59.8 to 61.5) and the full model reaches 62.5. Please specify the superpoint generation procedure, report instance-purity statistics (e.g., the fraction of superpoints contained in a single ground-truth instance on each benchmark), or modify the formulation to handle impure superpoints.
  2. [4.2, Table 2] The headline gains in Section 4.2 are computed against inconsistent baselines: mAP +2.6 and AP@50 +3.7 are relative to Maft‡ (59.9/76.5), AP@25 +2.5 is relative to Maft (84.5), while the ablation baseline [A] (Maft with surface normals and NMS) is 59.8/77.4/85.4. Please report a single consistent baseline for all metrics or explicitly show the per-metric baseline for each reported delta; as written, the claimed improvements cannot be directly verified from Table 2.
minor comments (8)
  1. [3.4, Eq. (5)] The expression 'S+ 1 2' is ambiguous; please write (S+1)/2 and state that S is the cosine similarity after L2-normalization, so that the argument to BCE lies in [0,1].
  2. [Abstract, Section 1, Section 4.1] The code link 'this website' is a placeholder; please provide an actual URL or an explicit statement about the code release timeline.
  3. [Section 4.1 and all result tables] No error bars or multiple-seed results are reported. Please either report mean and standard deviation over at least three runs or explicitly state that results are from a single run.
  4. [Section 3.3] The text says 'we concatenate F′max and F′mean to [F′max,F′min]'; the final bracket should read [F′max,F′mean] (or the notation should be made consistent).
  5. [Supplementary Table 9 caption] The caption says 'Best performance of box supervised methods is in boldface,' which appears to be a copy-paste artifact; this table compares fully supervised methods and the caption should be corrected.
  6. [Supplementary Section 8] The sentence 'The two of them work together to achieve 14 out of 18 categories' is unclear; please rephrase to state which methods are combined and what comparison is being made.
  7. [Section 4.3] The sentence 'their contributes comparably' should read 'their contributions are comparable.'
  8. [Section 4.1] Consider reporting sensitivity to the number of queries K, since K changes from 400 to 500 across datasets and no ablation is shown for this hyperparameter.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Relation3D is a standard supervised method evaluated on external benchmarks, with no prediction that reduces to a fitted input by construction.

full rationale

The paper's claimed contributions—ASAM, CLSR, and RSA—are all supervised components whose outputs are evaluated against held-out benchmarks. ASAM computes point-level aggregation weights from MLPs and softmax normalizes them over each superpoint; this is a learned pooling operation, not a parameter fitted to the final metric. CLSR uses ground-truth instance annotations to build the binary relation prior of Eq. 3 and applies a BCE contrastive loss in Eq. 5 to pull same-instance superpoint features together; this is standard labeled supervision, not a derivation of the labels from the model. RSA injects positional and geometric biases computed from each query's predicted mask and bounding box into self-attention (Eq. 7), an iterative refinement mechanism common in detection and segmentation; the predicted masks are not forced to equal the input. The reported gains in Tables 2–5 are measured on ScanNetV2, ScanNet++, ScanNet200, and S3DIS validation/test splits, which are external to the training procedure. No parameter is fitted to the benchmark metric and then renamed as a prediction. The paper does cite prior work by the same authors (e.g., QueryFormer, BSNet, PAMBA), but these citations are not load-bearing: no central premise, uniqueness theorem, or forced choice depends on them, and the main architectural ideas are compared against and built upon external baselines such as Maft and Relation-DETR. The code URL being a placeholder and the lack of details about superpoint generation are reproducibility limitations, not circularity. The superpoint instance-homogeneity assumption is an input modeling assumption whose violation would add noise, but it is not an equation that makes the output equal to the input by construction. Therefore no circular step is present, and a score of 0 is appropriate.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method rests on several domain assumptions about superpoint quality and the usefulness of predicted masks as geometric priors, plus a few manually tuned hyperparameters. There are no new physical or mathematical entities introduced.

free parameters (3)
  • number of queries K = 400 (500 for ScanNet++ and ScanNet200)
    Tuned on validation sets; larger datasets use more queries.
  • superpoint refinement interval r = 3
    Chosen via ablation on ScanNetV2 and ScanNet200 validation (Table 14).
  • loss weights lambda_1 through lambda_6 = 0.5, 1, 1, 0.5, 0.5, 1
    Manually set and not ablated.
assumptions (3)
  • domain assumption Superpoints are homogeneous with respect to instance boundaries
    The contrastive relationship matrix R_GT in Eq. 3 labels superpoint pairs as same-instance or different-instance, which is only valid if superpoints do not mix objects. If superpoints straddle boundaries, the BCE target is inconsistent. This is invoked throughout Section 3.4.
  • domain assumption Predicted masks from earlier decoder layers provide bounding boxes that are useful for relation modeling
    The RSA uses masks computed from queries to derive bounding boxes; early in training these masks are noisy, yet the method assumes they still encode useful geometric relations. This appears in Section 3.5.
  • ad hoc to paper The log-scaled positional and geometric relationship encoding captures useful inductive biases
    The specific encoding of relative position and size differences in Section 3.5 is a design choice not motivated by theory, though it is adapted from Relation-DETR.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation." pith.science (2026). https://pith.science/paper/BIVVLMNE

@misc{pith2026250617891,
  author       = {Pith},
  title        = {Pith review of: Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BIVVLMNE}},
  note         = {Machine review of arXiv:2506.17891}
}
read the original abstract

3D instance segmentation aims to predict a set of object instances in a scene, representing them as binary foreground masks with corresponding semantic labels. Currently, transformer-based methods are gaining increasing attention due to their elegant pipelines and superior predictions. However, these methods primarily focus on modeling the external relationships between scene features and query features through mask attention. They lack effective modeling of the internal relationships among scene features as well as between query features. In light of these disadvantages, we propose \textbf{Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation}. Specifically, we introduce an adaptive superpoint aggregation module and a contrastive learning-guided superpoint refinement module to better represent superpoint features (scene features) and leverage contrastive learning to guide the updates of these features. Furthermore, our relation-aware self-attention mechanism enhances the capabilities of modeling relationships between queries by incorporating positional and geometric relationships into the self-attention mechanism. Extensive experiments on the ScanNetV2, ScanNet++, ScanNet200 and S3DIS datasets demonstrate the superior performance of Relation3D.

Figures

Figures reproduced from arXiv: 2506.17891 by the authors.

Figure 1
Figure 1. T-SNE visualization of the superpoint-level feature distributions on ScanNetV2 validation set. Different colors rep￾resent different instances. Our method highlights better inter-object diversity and intra-object similarity. Point feature variation (Maft) 1.8603 (Excessive) Setting Maft Ours (Stage 1) Ours (Stage 2) Ours (Stage 3) Lcont 1.057 0.7255 0.5841 0.5739 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) The overall framework of our method Relation3D. (b) The details of our proposed adaptive superpoint aggregation module. (c) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The superpoint refinement module. Superpoint-level [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visualization of instance segmentation results on Scan￾NetV2 and ScanNet++ validation set. The red boxes highlight the key regions. learning rate of 0.0002. Point clouds are voxelized with a size of 0.02m. For hyperparameters, we tune K, r as 400, 3 respectively. λ1, λ…
Figure 5
Figure 5. Figure 5: (a) Comparison of attention maps for traditional self-attention vs. relation-aware self-attention. We display the progression of attention maps from layer 1, 3, 5. (b) Comparison of attention weight distributions for traditional self-attention vs. relation-aware self-a…
Figure 7
Figure 7. Figure 7: , our method demonstrates a faster convergence speed compared to the baseline. This improvement can be attributed to the relation priors introduced by CLSR and RSA: contrastive learning provides relation priors for super￾points to guide feature aggregation, while RSA i…
Figure 8
Figure 8. Figure 8: More visualization of instance segmentation results on ScanNetV2 validation set. The red boxes highlight the key regions 5 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: More T-SNE visualization of the superpoint-level feature distributions on ScanNetV2 validation set. 6 [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: More visualization of weights in the adaptive superpoint aggregation module. 7 [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 53 canonical work pages

  1. [1]

    Instance seg- mentation based 6d pose estimation of industrial objects using point clouds for robotic bin-picking.Robotics and Computer- Integrated Manufacturing, 82:102541, 2023

    Chungang Zhuang, Shaofei Li, and Han Ding. Instance seg- mentation based 6d pose estimation of industrial objects using point clouds for robotic bin-picking.Robotics and Computer- Integrated Manufacturing, 82:102541, 2023. 1

    work page 2023

  2. [2]

    Deep learning-based smart task assistance in wearable augmented reality.Robotics and Computer- Integrated Manufacturing, 63:101887, 2020

    Kyeong-Beom Park, Minseok Kim, Sung Ho Choi, and Jae Yeol Lee. Deep learning-based smart task assistance in wearable augmented reality.Robotics and Computer- Integrated Manufacturing, 63:101887, 2020. 1

    work page 2020

  3. [3]

    Snap2cad: 3d indoor environ- ment reconstruction for ar/vr applications using a smartphone device.Computers & Graphics, 100:116–124, 2021

    Alessandro Manni, Damiano Oriti, Andrea Sanna, Francesco De Pace, and Federico Manuri. Snap2cad: 3d indoor environ- ment reconstruction for ar/vr applications using a smartphone device.Computers & Graphics, 100:116–124, 2021. 1

    work page 2021

  4. [4]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024. 1

    work page 2024

  5. [5]

    Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting.arXiv preprint arXiv:2410.07707, 2024

    Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Ji- ahao Lu, Wenfei Yang, Tianzhu Zhang, and Yongdong Zhang. Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting.arXiv preprint arXiv:2410.07707, 2024

    arXiv

  6. [6]

    Align3r: Aligned monocular depth estimation for dynamic videos.arXiv preprint arXiv:2412.03079, 2024

    Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic videos.arXiv preprint arXiv:2412.03079, 2024

    arXiv 2024

  7. [7]

    Dn-4dgs: Denoised deformable network with temporal-spatial aggregation for dynamic scene rendering.arXiv preprint arXiv:2410.13607, 2024

    Jiahao Lu, Jiacheng Deng, Ruijie Zhu, Yanzhe Liang, Wenfei Yang, Tianzhu Zhang, and Xu Zhou. Dn-4dgs: Denoised deformable network with temporal-spatial aggregation for dynamic scene rendering.arXiv preprint arXiv:2410.13607, 2024

    arXiv

  8. [8]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis.arXiv preprint arXiv:2308.09713,

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis.arXiv preprint arXiv:2308.09713,

    arXiv

Show all 68 references
  1. [9]

    Towards end-to-end lane detection: an instance segmentation approach

    Davy Neven, Bert De Brabandere, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. Towards end-to-end lane detection: an instance segmentation approach. In2018 IEEE intelligent vehicles symposium (IV), pages 286–291. IEEE, 2018. 1

    2018

  2. [10]

    A survey of autonomous driving: Common practices and emerging technologies.IEEE access, 8:58443– 58469, 2020

    Ekim Yurtsever, Jacob Lambert, Alexander Carballo, and Kazuya Takeda. A survey of autonomous driving: Common practices and emerging technologies.IEEE access, 8:58443– 58469, 2020. 1

    2020

  3. [11]

    Compar- ison of the selected state-of-the-art 3d indoor scanning and point cloud generation methods.Remote sensing, 9(8):796,

    Ville V Lehtola, Harri Kaartinen, Andreas N¨uchter, Risto Kai- jaluoto, Antero Kukko, Paula Litkey, Eija Honkavaara, Tomi Rosnell, Matti T Vaaja, Juho-Pekka Virtanen, et al. Compar- ison of the selected state-of-the-art 3d indoor scanning and point cloud generation methods.Rem...

  4. [12]

    Rescan: Inductive instance segmentation for indoor rgbd scans

    Maciej Halber, Yifei Shi, Kai Xu, and Thomas Funkhouser. Rescan: Inductive instance segmentation for indoor rgbd scans. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 2541–2550, 2019. 1

    2019

  5. [13]

    Pamba: Enhancing global inter- action in point clouds via state space model

    Zhuoyuan Li, Yubo Ai, Jiahao Lu, ChuXin Wang, Jiacheng Deng, Hanzhi Chang, Yanzhe Liang, Wenfei Yang, Shifeng Zhang, and Tianzhu Zhang. Pamba: Enhancing global inter- action in point clouds via state space model. InProceedings of the AAAI Conference on Artificial Intelligence,...

    2025

  6. [14]

    Diff3detr: Agent-based diffusion model for semi-supervised 3d object detection

    Jiacheng Deng, Jiahao Lu, and Tianzhu Zhang. Diff3detr: Agent-based diffusion model for semi-supervised 3d object detection. InEuropean Conference on Computer Vision, pages 57–73. Springer, 2024. 1

    2024

  7. [15]

    Gspn: Generative shape proposal network for 3d instance segmentation in point cloud

    Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3947–3956, 2019. 1, 2

    2019

  8. [16]

    3d-sis: 3d seman- tic instance segmentation of rgb-d scans

    Ji Hou, Angela Dai, and Matthias Nießner. 3d-sis: 3d seman- tic instance segmentation of rgb-d scans. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4421–4430, 2019. 6

    2019

  9. [17]

    Learning object bounding boxes for 3d instance segmentation on point clouds

    Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning object bounding boxes for 3d instance segmentation on point clouds. Advances in neural information processing systems, 32, 2019. 1, 3

    2019

  10. [18]

    3d-mpa: Multi-proposal aggre- gation for 3d semantic instance segmentation

    Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, and Matthias Nießner. 3d-mpa: Multi-proposal aggre- gation for 3d semantic instance segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9031–9040, 2020. 1, 6, 2, 3

    2020

  11. [19]

    Instance segmentation in 3d scenes using semantic superpoint tree networks

    Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, and Kui Jia. Instance segmentation in 3d scenes using semantic superpoint tree networks. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2783– 2792, 2021. 6, 2, 3, 4

    2021

  12. [20]

    Softgroup for 3d instance segmentation on point clouds

    Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo. Softgroup for 3d instance segmentation on point clouds. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2708–2717,

  13. [21]

    Mask3d for 3d semantic instance segmentation.arXiv preprint arXiv:2210.03105, 2022

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d for 3d semantic instance segmentation.arXiv preprint arXiv:2210.03105, 2022. 1, 2, 3, 5, 6, 4

    2022 arXiv

  14. [22]

    Superpoint transformer for 3d scene instance segmentation

    Jiahao Sun, Chunmei Qing, Junpeng Tan, and Xiangmin Xu. Superpoint transformer for 3d scene instance segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2393–2401, 2023. 2, 3, 6, 4

    2023

  15. [23]

    Query refinement transformer for 3d instance segmentation

    Jiahao Lu, Jiacheng Deng, Chuxin Wang, Jianfeng He, and Tianzhu Zhang. Query refinement transformer for 3d instance segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 18516–18526, 2023. 2, 3, 6, 4

    2023

  16. [24]

    Mask-attention-free transformer for 3d instance 9 segmentation

    Xin Lai, Yuhui Yuan, Ruihang Chu, Yukang Chen, Han Hu, and Jiaya Jia. Mask-attention-free transformer for 3d instance 9 segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3693–3703, 2023. 2, 3, 5, 6, 7, 4

    2023

  17. [25]

    Beyond the final layer: Hierarchical query fusion transformer with agent-interpolation initialization for 3d instance segmentation

    Jiahao Lu, Jiacheng Deng, and Tianzhu Zhang. Beyond the final layer: Hierarchical query fusion transformer with agent-interpolation initialization for 3d instance segmentation. arXiv preprint arXiv:2502.04139, 2025. 1

    2025 arXiv

  18. [26]

    Per- pixel classification is not all you need for semantic segmen- tation.Advances in Neural Information Processing Systems, 34:17864–17875, 2021

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per- pixel classification is not all you need for semantic segmen- tation.Advances in Neural Information Processing Systems, 34:17864–17875, 2021. 1, 3

    2021

  19. [27]

    Masked-attention mask trans- former for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask trans- former for universal image segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1290–1299, 2022. 2

    2022

  20. [28]

    Core-text: Improving scene text detection with contrastive relational reasoning

    Jingyang Lin, Yingwei Pan, Rongfeng Lai, Xuehang Yang, Hongyang Chao, and Ting Yao. Core-text: Improving scene text detection with contrastive relational reasoning. In2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2021. 2, 3

    2021

  21. [29]

    Relation detr: Exploring explicit position relation prior for object detection.arXiv preprint arXiv:2407.11699, 2024

    Xiuquan Hou, Meiqin Liu, Senlin Zhang, Ping Wei, Badong Chen, and Xuguang Lan. Relation detr: Exploring explicit position relation prior for object detection.arXiv preprint arXiv:2407.11699, 2024. 3, 5

    2024 arXiv

  22. [30]

    Relation-enhanced detr for component detection in graphic design reverse engineering

    Xixuan Hao, Danqing Huang, Jieru Lin, and Chin-Yew Lin. Relation-enhanced detr for component detection in graphic design reverse engineering. InProceedings of the Thirty- Second International Joint Conference on Artificial Intelli- gence, pages 4785–4793, 2023. 2, 3

    2023

  23. [31]

    Sas: Seg- ment any 3d scene with integrated 2d priors.arXiv preprint arXiv:2503.08512, 2025

    Zhuoyuan Li, Jiahao Lu, Jiacheng Deng, Hanzhi Chang, Lifan Wu, Yanzhe Liang, and Tianzhu Zhang. Sas: Seg- ment any 3d scene with integrated 2d priors.arXiv preprint arXiv:2503.08512, 2025. 2

    2025 arXiv

  24. [32]

    Supervised contrastive learning.Advances in neural information processing systems, 33:18661–18673,

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning.Advances in neural information processing systems, 33:18661–18673,

  25. [33]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. InInternational conference on machine learning, pages 1597–1607. PMLR, 2020

    2020

  26. [34]

    Bsnet: Box- supervised simulation-assisted mean teacher for 3d instance segmentation.arXiv preprint arXiv:2403.15019, 2024

    Jiahao Lu, Jiacheng Deng, and Tianzhu Zhang. Bsnet: Box- supervised simulation-assisted mean teacher for 3d instance segmentation.arXiv preprint arXiv:2403.15019, 2024. 2

    2024 arXiv

  27. [35]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 2, 5, 1

    2017

  28. [36]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023. 2, 5

    2023

  29. [37]

    Language- grounded indoor 3d semantic segmentation in the wild

    David Rozenberszki, Or Litany, and Angela Dai. Language- grounded indoor 3d semantic segmentation in the wild. In European Conference on Computer Vision, pages 125–141. Springer, 2022. 2, 5

    2022

  30. [38]

    3d semantic parsing of large-scale indoor spaces

    Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1534–1543, 2016. 2, 5

    2016

  31. [39]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. InProceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 2

    2017

  32. [40]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660,

  33. [41]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017. 3

    2017

  34. [42]

    The hungarian method for the assignment problem.Naval research logistics quarterly, 2(1-2):83–97,

    Harold W Kuhn. The hungarian method for the assignment problem.Naval research logistics quarterly, 2(1-2):83–97,

  35. [43]

    Pointgroup: Dual-set point group- ing for 3d instance segmentation

    Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point group- ing for 3d instance segmentation. InProceedings of the IEEE/CVF conference on computer vision and Pattern recog- nition, pages 4867–4876, 2020. 3, 6, 2

    2020

  36. [44]

    Isbnet: a 3d point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution

    Tuan Duc Ngo, Binh-Son Hua, and Khoi Nguyen. Isbnet: a 3d point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 13550–13559, 2023. 3, 6, 2

    2023

  37. [45]

    Spherical mask: Coarse-to-fine 3d point cloud instance segmentation with spherical repre- sentation

    Sangyun Shin, Kaichen Zhou, Madhu Vankadari, Andrew Markham, and Niki Trigoni. Spherical mask: Coarse-to-fine 3d point cloud instance segmentation with spherical repre- sentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4060–4069,

  38. [46]

    Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers

    Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xi- ang, Philip HS Torr, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. InProceedings of the IEEE/CVF conference on...

    2021

  39. [47]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M Ni, and Heung-Yeung Shum. Mask dino: Towards a unified transformer-based framework for object detection and segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2023

  40. [48]

    Spconv: Spatially sparse convolu- tion library

    Spconv Contributors. Spconv: Spatially sparse convolu- tion library. https://github.com/traveller59/ spconv, 2022. 3

    2022

  41. [49]

    Pytorch: An 10 imperative style, high-performance deep learning library.Ad- vances in neural information processing systems, 32, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An 10 imperative style, high-performance deep learning library.Ad- vances in neural information processing systems,...

    2019

  42. [50]

    Efficient non- maximum suppression

    Alexander Neubeck and Luc Van Gool. Efficient non- maximum suppression. In18th international conference on pattern recognition (ICPR’06), volume 3, pages 850–855. IEEE, 2006. 5

    2006

  43. [51]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 5

    2017 arXiv

  44. [52]

    Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution

    Tong He, Chunhua Shen, and Anton Van Den Hengel. Dyco3d: Robust instance segmentation of 3d point clouds through dynamic convolution. InProceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 354–363, 2021. 6, 2, 3

    2021

  45. [53]

    Maskgroup: Hierarchical point grouping and masking for 3d instance segmentation

    Min Zhong, Xinghao Chen, Xiaokang Chen, Gang Zeng, and Yunhe Wang. Maskgroup: Hierarchical point grouping and masking for 3d instance segmentation. In2022 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2022. 6, 2, 3

    2022

  46. [54]

    Occuseg: Occupancy-aware 3d instance segmentation

    Lei Han, Tian Zheng, Lan Xu, and Lu Fang. Occuseg: Occupancy-aware 3d instance segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2940–2949, 2020. 6, 2, 3

    2020

  47. [55]

    Hierarchical aggregation for 3d instance segmentation

    Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, and Xinggang Wang. Hierarchical aggregation for 3d instance segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15467–15476, 2021. 6, 2, 3, 4

    2021

  48. [56]

    3d instances as 1d kernels

    Yizheng Wu, Min Shi, Shuaiyuan Du, Hao Lu, Zhiguo Cao, and Weicai Zhong. 3d instances as 1d kernels. InComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXIX, pages 235–252. Springer, 2022. 6, 2, 3

    2022

  49. [57]

    Frustum pointnets for 3d object detection from rgb-d data

    Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb-d data. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 918–927, 2018. 2, 3

    2018

  50. [58]

    3d instance segmentation via multi-task metric learning

    Jean Lahoud, Bernard Ghanem, Marc Pollefeys, and Mar- tin R Oswald. 3d instance segmentation via multi-task metric learning. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 9256–9266, 2019. 2, 3

    2019

  51. [59]

    Point cloud instance segmen- tation using probabilistic embeddings

    Biao Zhang and Peter Wonka. Point cloud instance segmen- tation using probabilistic embeddings. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8883–8892, 2021. 2, 3

    2021

  52. [60]

    Embodiedsam: Online segment any 3d thing in real time.arXiv preprint arXiv:2408.11811, 2024

    Xiuwei Xu, Huangxing Chen, Linqing Zhao, Ziwei Wang, Jie Zhou, and Jiwen Lu. Embodiedsam: Online segment any 3d thing in real time.arXiv preprint arXiv:2408.11811, 2024. 4 11 Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation Supplementary Material

    2024 arXiv

  53. [61]

    We then provide additional discussion about position embed- ding (Section 9)

    Overview In this supplementary material, we begin by presenting a more detailed comparison of quantitative metrics on Scan- NetV2 [ 35] validation set and test set (Section 8). We then provide additional discussion about position embed- ding (Section 9). To further validate th...

  54. [62]

    As the table illustrates, our method achieves the best performance in 14 out of 18 categories

    Detailed results on ScanNetV2 validation and hidden test set The detailed results for each category on ScanNetV2 vali- dation set are reported in Table 9. As the table illustrates, our method achieves the best performance in 14 out of 18 categories. The two of them work togeth...

  55. [63]

    Discussion about position embedding In DETR-based methods, query typically consist of two embeddings: a content embedding and a position embed- ding. In the transformer decoder, the position embedding is added to the content embedding and then input into the self-attention/cro...

  56. [64]

    Table 13 presents model parameters and runtime per scan for various methods evaluated on ScanNetV2 validation set

    Parameter and Runtime Analysis. Table 13 presents model parameters and runtime per scan for various methods evaluated on ScanNetV2 validation set. For a fair comparison, all runtimes are measured on the same RTX 4090 GPU. Compared to Maft, our method achieves better performanc...

  57. [65]

    More Visualization Qualitative comparison (Figure 8):To vividly illustrate the differences between our method and baseline, we visualize qualitative results in Figure 8. From the regions highlighted in the last row, we observe that the baseline method tends to confuse chairs w...

  58. [66]

    By varyingr, we evaluate the balance between computational efficiency and the quality of feature refinement

    More ablution study We conduct an ablation study to analyze the effect of dif- ferent values ofr on model performance. By varyingr, we evaluate the balance between computational efficiency and the quality of feature refinement. The results of this study are detailed in Table 1...

  59. [67]

    Thus, we set r= 3

    reduces the frequency of feature refinement, the compu- tational cost is consistently reduced. Thus, we set r= 3 . Moreover, we find that the hyperparameterr demonstrates strong robustness, consistently achieving a good balance between accuracy and efficiency across different datasets

  60. [68]

    Limitation and future work Existing indoor 3D instance segmentation methods primar- ily focus on static objects and are typically performed of- 3 Method Parameter(M)Runtime(ms) HAIS [55] 30.9 525 SSTNet [19] / 663 SPFormer [22] 17.6 390 Mask3D [21] 39.6 525 SoftGroup [20] 30.9...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.