Pith. sign in

REVIEW 2 major objections 5 minor 30 references

PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a permutation-invariant point-cloud segmentation network that combines local EdgeConv features with global NetVLAD context, and reports state-of-the-art results on S3DIS and ScanNet.

desk verdict A genuinely useful architecture paper whose central permutation-invariance proof does not hold up; the fix is to reframe the claim as equivariance. read the letter →

arxiv 1908.05425 v1 pith:XYP2UZBZ submitted 2019-08-15 cs.CV

Na Zhao , Tat-Seng Chua , Gim Hee Lee This is my paper · ORCID
classification cs.CV
keywords pointcloudsemanticsegmentationpermutationinvarianceEdgeConvNetVLADglobalcontextlocalstructure3Dscenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PS2-Net is a deep network for semantic segmentation of 3D scene point clouds, built from four stacked encoder blocks. Each encoder uses EdgeConv to extract local geometric features from a static K-nearest-neighbor graph and NetVLAD to aggregate a global scene descriptor, then concatenates the two. The paper argues this design is permutation invariant, meaning the network output does not depend on the order of the input points, and supplies a proof of that property. On the S3DIS and ScanNet indoor benchmarks, the paper reports state-of-the-art mean IoU, with the largest gains on rare classes.

What carries the argument

The load-bearing component is the stackable encoder, which pairs EdgeConv with NetVLAD. EdgeConv builds a fixed K-nearest-neighbor graph in the input space, forms edge features of the form [x_i, x_j − x_i], processes them through shared MLPs, and aggregates with channel-wise max and average pooling; NetVLAD then produces a global descriptor by soft-assigning each point feature to M cluster centers and summing the residuals. The paper's Lemma 1 is the central identity used to argue invariance: it claims that permuting input rows only permutes the intermediate feature rows, and that the NetVLAD sum over all points is unaffected by that reordering.

What would settle it

Take a trained PS2-Net, run a scene point cloud through it, then run the same cloud with the point rows shuffled, and compare the per-point label predictions at identical spatial locations; any difference shows the network is not permutation invariant. Also, inspect the proof: it does not track what happens to the KNN index tensor when the input rows are permuted, so the calculation that shows the final NetVLAD vector is unchanged does not by itself establish that per-point outputs are unchanged.

Watch

Extended reading notes

Core claim

The central claim is that a point-cloud segmentation network can capture both fine-grained local structure and whole-scene context without sacrificing permutation invariance, and that this combination sets a new state of the art. Concretely, the authors propose PS2-Net, whose encoder computes edge features as concatenations of a point feature with the difference to each neighbor, pools them with max and average operations, and then feeds the result to a NetVLAD layer that softly assigns point features to 16 learned cluster centers and sums residuals into a global descriptor. The paper states a lemma that permuting the rows of the input matrix leaves the output unchanged, and offers a proof based on the symmetric aggregation inside EdgeConv and the summation inside NetVLAD. Empirically, on S3DIS the network improves mean IoU from 56.1 to 61.56, or from 65.39 to 66.60 under the PointCNN data-prep setup, and on ScanNet it reaches 44.90 mIoU, surpassing the comparisons reported.

Load-bearing premise

The proof assumes that reordering the input points leaves every point's K-nearest-neighbor list unchanged; if that is false, the network's per-point output can change under permutation even though the global descriptor does not.

Editorial extensions

If this is right

  • Point-cloud segmentation can be done in a single end-to-end network that is invariant to input ordering without projecting or voxelizing the cloud.
  • Stacking the encoder deepens the receptive field, so local and global cues can be combined at multiple scales in one architecture.
  • The NetVLAD global descriptor resolves ambiguities between classes with similar local appearance, such as walls versus white boards.
  • Static KNN graphs avoid the cost of recomputing neighborhoods in feature space, making the network practical on large scenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A corrected invariance proof would need to track how the K-nearest-neighbor index lists permute when input rows are swapped; until then, the theorem as written establishes invariance of the global descriptor and equivariance of point-wise features.
  • Testing invariance empirically is straightforward and separate from the benchmark numbers: feed the same cloud in two random row orders through a trained model and compare per-point predictions.
  • The encoder's combination of local graph convolution and soft-assignment global pooling is a general recipe that could transfer to other unordered inputs, such as point sets in higher dimensions or general graphs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes PS^2-Net, a deep network for point-based semantic segmentation of 3D indoor scenes. The architecture stacks four encoders, each combining an EdgeConv module for local structure and a NetVLAD module for global context; per-point features from all encoders are concatenated and passed to shared MLPs for classification. The authors claim that the network is permutation invariant and provide a theoretical proof in Section 3.3, and they report state-of-the-art results on the S3DIS and ScanNet benchmarks under multiple data-preprocessing protocols. The empirical study includes comparisons with PointNet, PointNet++, RSNet, DGCNN, RNNCF, and PointCNN, plus ablations of the local/global components and hyperparameters.

Significance. If the empirical results are reproducible, the paper demonstrates that fusing NetVLAD global descriptors with EdgeConv local features yields competitive semantic segmentation on large-scale indoor point clouds, improving mIoU over contemporary baselines such as DGCNN and RSNet. The architecture is simple and the ablation study in Table 3 and Figure 5 supports the contribution of both components. However, the paper's central theoretical claim — a proof of permutation invariance — is seriously flawed: the proof establishes only invariance of the NetVLAD global descriptor and, by the authors' own derivation, the EdgeConv output is permuted, not invariant. The correct property for per-point dense segmentation is permutation equivariance, not invariance. Because the proof is a headline contribution in the abstract and Section 3.3, the manuscript needs major revision to correct the statement and proof of the symmetry property. The empirical claims are separately testable and not undercut by the proof issue, but the theoretical contribution must be reframed.

major comments (2)
  1. [Section 3.3, Lemma 1 and Eqs. (2), (7)-(10)] The proof of Lemma 1 is invalid. The sentence 'As the reordering does not affect the order of nearest neighbors, the KNN indices are still given by K' is false: if the input point rows are permuted, the KNN index rows must be permuted accordingly, and for a row that now contains point p_j, the neighbor set must be the old neighbor set of p_j, not the old neighbor set of p_i. The subsequent derivation in Eqs. (7)-(10) proves only that the NetVLAD global descriptor V is invariant, not that the per-point output L is unchanged. In fact, the proof itself shows that the EdgeConv output Y is permuted (the line defining \tilde{Y}). Since the encoder output is the concatenation of Y and a tiled copy of V, the network is permutation-equivariant, not permutation-invariant. The claim in the abstract and Section 3.3 that PS2-Net is permutation invariant is therefore unsupported.
  2. [Section 3.3, Lemma 1] The symmetry property being proved is mis-specified. For dense per-point segmentation, the correct requirement is permutation equivariance: permuting the input rows must permute the output rows in exactly the same way, so that each point retains its label. The statement of Lemma 1 — 'the output of the network remains unchanged' — is not the property a segmentation network should have, and proving it would mean the network ignores the identity of each point. The authors should restate their theoretical contribution as a proof of permutation equivariance and then prove that property by tracking the permutation through the KNN indexing and the EdgeConv/NetVLAD operations. This restatement also affects the comparison with prior works: PointNet++ and DGCNN, for example, are already permutation-equivariant in the per-point sense, so the claim that prior methods 'violate or do not achieve permutation invariance' is misleading.
minor comments (5)
  1. [Abstract and Section 1] The abstract contains the typo 'start-of-the-art' instead of 'state-of-the-art', and the network name is typeset inconsistently as both 'PS2-Net' and 'PS^2-Net' throughout the paper; please unify the notation.
  2. [Section 4.2] In the implementation details paragraph, the sentence 'Skip link is is added from the output of EdgeConv to the output of NetVLAD' has a duplicated 'is'.
  3. [Section 3.3, Eqs. (7)-(8)] The notation f_m^\nu is defined in Eq. (8) as a per-point function, but it is used in Eq. (7) as if it were an aggregation over all points. Please introduce notation that distinguishes the per-point soft-assignment contribution from the summed NetVLAD descriptor to make the proof easier to follow and to avoid the appearance of a notational shortcut.
  4. [Table 1] The class-wise IoU entries for DGCNN are all em-dashes; please provide the values if available or state explicitly that DGCNN's per-class IoUs were not reported in the original paper.
  5. [Table 3 and Section 4.5] The ablation 'w/o NetVLAD' replaces the NetVLAD module with max-pooling over the K-nearest neighbors, which is a local aggregation rather than a global-context aggregation. This choice makes the comparison against the full model less directly interpretable as a test of global context; the text should clarify that this baseline removes global pooling but does not necessarily isolate the effect of global context alone.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical SOTA claim is grounded in external benchmarks, the self-citation to PointNetVLAD is supporting rather than load-bearing, and the permutation-invariance proof contains a correctness gap rather than a circular reduction.

full rationale

The paper's central empirical claims are evaluated on external benchmarks (S3DIS and ScanNet) against published baselines, and the reported gains are not produced by fitting a parameter and then predicting a closely related quantity. The architecture openly combines EdgeConv from DGCNN and NetVLAD, with an acknowledged prior adoption of NetVLAD for point clouds in a paper co-authored by one of the present authors; that citation is supporting context rather than load-bearing, since NetVLAD's aggregation is a standard sum-over-points operation and the evaluation here is external to that prior work. The ablation study removes components and measures performance on Area 5 as a held-out test, which is a controlled empirical comparison, not a derivation from the components' definitions. The only substantive concern is in Section 3.3, where the proof of Lemma 1 assumes 'the reordering does not affect the order of nearest neighbors, the KNN indices are still given by K.' That premise is not correct as stated, and the proof as written establishes only that the NetVLAD global descriptor is invariant, not that the per-point segmentation output is unchanged. This is a mathematical correctness gap, not a circularity: the claimed invariance is not equivalent by construction to the proof's inputs, and the empirical state-of-the-art claim remains independently testable on the external benchmarks. Accordingly, no circular step is identified.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The empirical SOTA claim rests on standard hyperparameter choices (encoder count, K, M) selected via validation ablations on S3DIS Area 5; these are not fitted to the test sets. The theoretical invariance claim rests on a specific premise about KNN indices under permutation that is false as written. No new physical entities are introduced.

free parameters (3)
  • Number of stacked encoders = 4
    Chosen by ablation on S3DIS Area 5 (Figure 5a); used in all experiments.
  • Number of nearest neighbors K in EdgeConv = 20
    Chosen by ablation on S3DIS Area 5 (Figure 5b); controls local region size.
  • Number of NetVLAD clusters M = 16
    Chosen by ablation on S3DIS Area 5 (Figure 5c); produces 16x128 global descriptor compressed to 128.
assumptions (2)
  • ad hoc to paper Permuting the rows of the input point cloud does not change the KNN index matrix K
    Used in the proof of Lemma 1 (Section 3.3). In reality, row indices of neighbors change under permutation; the proof should permute the index tensor. This is a load-bearing but false premise in the proof.
  • domain assumption A static KNN graph computed in the original 3D coordinate space remains informative for grouping local regions across all stacked feature spaces
    Section 3.1 states static metric-based KNN graphs are used because they may supervise local structure learning with spatial constraints. The assumption is that spatial proximity in input space is sufficient for all encoders.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation." pith.science (2026). https://pith.science/paper/XYP2UZBZ

@misc{pith2026190805425,
  author       = {Pith},
  title        = {Pith review of: PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYP2UZBZ}},
  note         = {Machine review of arXiv:1908.05425}
}
read the original abstract

In this paper, we present the PS^2-Net -- a locally and globally aware deep learning framework for semantic segmentation on 3D scene-level point clouds. In order to deeply incorporate local structures and global context to support 3D scene segmentation, our network is built on four repeatedly stacked encoders, where each encoder has two basic components: EdgeConv that captures local structures and NetVLAD that models global context. Different from existing start-of-the-art methods for point-based scene semantic segmentation that either violate or do not achieve permutation invariance, our PS^2-Net is designed to be permutation invariant which is an essential property of any deep network used to process unordered point clouds. We further provide theoretical proof to guarantee the permutation invariance property of our network. We perform extensive experiments on two large-scale 3D indoor scene datasets and demonstrate that our PS2-Net is able to achieve state-of-the-art performances as compared to existing approaches.

Figures

Figures reproduced from arXiv: 1908.05425 by the authors.

Figure 1
Figure 1. Our PS2 -Net segments raw point cloud into semantic homogeneous regions. the non-permutation invariance property of neural networks [18, 29], which work well on 2D image pixels that are ar￾ranged in a regular orderly structure, but would fail catas￾trophically when used on 3D point clouds where the points exist in an unorderly and irregular manner. To circumvent the permutation invariance problem, sev￾eral existing … view at source ↗
Figure 2
Figure 2. The network architecture of our PS2 -Net. coordinate can be permuted into a latent and potentially canonical order with the χ-transformation. The learned χ￾transformation is further packed with typical convolutions to form a new process to extract features from local regions. However, the χ-transformation is not permutation invariant. KCNet [21] presents a kernel correlation layer to exploit lo￾cal geometric structu… view at source ↗
Figure 3
Figure 3. PS2 -Net semantic segmentation results on S3DIS. significant improvement. Additionally, our PS2 -Net outper￾forms the three existing RNN-based methods [8, 10, 28] in the mIoU criteria. In the lower part of [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: PS2 -Net semantic segmentation results on ScanNet. 4.4. Segmentation on ScanNet Dataset 4.4.1 Data preparation Similar to S3DIS, there are two data preparation setups in the ScanNet dataset. The first setup is proposed by Point￾Net++ [19]. It follows the same setup in …
Figure 5
Figure 5. Figure 5: Effectiveness of hyper-parameters on S3DIS [A5]. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visualized examples for ablation test of PS [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 27 canonical work pages

  1. [1]

    Arandjelovic, P

    R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic. Netvlad: Cnn architecture for weakly supervised place recognition. In CVPR, pages 5297–5307, 2016. 2, 3, 4

    work page 2016

  2. [2]

    Armeni, O

    I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. 3d semantic parsing of large- scale indoor spaces. In CVPR, pages 1534–1543, 2016. 5

    work page 2016

  3. [3]

    Badrinarayanan, A

    V . Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis & Ma- chine Intelligence, (12):2481–2495, 2017. 1

    work page 2017

  4. [4]

    Boulch, J

    A. Boulch, J. Guerry, B. Le Saux, and N. Audebert. Snapnet: 3d point cloud semantic labeling with 2d deep segmentation networks. Computers & Graphics, 71:189–198, 2018. 1

    work page 2018

  5. [5]

    Brock, T

    A. Brock, T. Lim, J. M. Ritchie, and N. Weston. Generative and discriminative voxel modeling with convolutional neural networks. arXiv preprint arXiv:1608.04236, 2016. 1

    arXiv 2016

  6. [6]

    L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully con- nected crfs. IEEE Transactions on Pattern Analysis & Ma- chine Intelligence, 40(4):834–848, 2018. 1

    work page 2018

  7. [7]

    A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Niessner. Scannet: Richly-annotated 3d reconstruc- tions of indoor scenes. In CVPR, pages 5828–5839, 2017. 1, 5, 7

    work page 2017

  8. [8]

    Engelmann, T

    F. Engelmann, T. Kontogianni, A. Hermans, and B. Leibe. Exploring spatial context for 3d semantic segmentation of point clouds. In CVPR, pages 716–724, 2017. 1, 2, 6

    work page 2017

Show all 30 references
  1. [9]

    K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick. Mask r-cnn. In ICCV, pages 2980–2988. IEEE, 2017. 1

    2017

  2. [10]

    Huang, W

    Q. Huang, W. Wang, and U. Neumann. Recurrent slice net- works for 3d segmentation of point clouds. In CVPR, pages 2626–2635, 2018. 1, 2, 6, 7, 8

    2018

  3. [11]

    Kalogerakis, M

    E. Kalogerakis, M. Averkiou, S. Maji, and S. Chaudhuri. 3d shape segmentation with projective convolutional networks. In CVPR, pages 6630–6639, 2017. 1

    2017

  4. [12]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 5

    2014 arXiv

  5. [13]

    Landrieu and M

    L. Landrieu and M. Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. In CVPR, pages 4558–4567. 1, 2

  6. [14]

    F. J. Lawin, M. Danelljan, P. Tosteberg, G. Bhat, F. S. Khan, and M. Felsberg. Deep projective 3d semantic segmentation. In International Conference on Computer Analysis of Images and Patterns, pages 95–107. Springer, 2017. 1

    2017

  7. [15]

    Y . Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen. Pointcnn: Convolution on x-transformed points. In Advances in Neural Information Processing Systems, pages 820–830, 2018. 1, 2, 6, 7, 8

    2018

  8. [16]

    J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431– 3440, 2015. 1

    2015

  9. [17]

    Maturana and S

    D. Maturana and S. Scherer. V oxnet: A 3d convolutional neural network for real-time object recognition. InIntelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pages 922–928. IEEE, 2015. 1

    2015

  10. [18]

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652–660, 2017. 1, 2, 5, 6, 7, 8

    2017

  11. [19]

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hi- erarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems , pages 5099–5108, 2017. 1, 2, 5, 7, 8

    2017

  12. [20]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox. U-net: Convo- lutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention , pages 234–241. Springer,

  13. [21]

    Y . Shen, C. Feng, Y . Yang, and D. Tian. Mining point cloud local structures by kernel correlation and graph pooling. In CVPR, volume 4, 2018. 2, 3

    2018

  14. [22]

    Sinha, J

    A. Sinha, J. Bai, and K. Ramani. Deep learning 3d shape surfaces using geometry images. In ECCV, pages 223–240. Springer, 2016. 1

    2016

  15. [23]

    H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller. Multi- view convolutional neural networks for 3d shape recognition. In CVPR, pages 945–953, 2015. 1

    2015

  16. [24]

    Tchapmi, C

    L. Tchapmi, C. Choy, I. Armeni, J. Gwak, and S. Savarese. Segcloud: Semantic segmentation of 3d point clouds. In 3D Vision (3DV), 2017 International Conference on, pages 537–

    2017

  17. [25]

    M. A. Uy and G. H. Lee. Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition. CVPR,

  18. [26]

    Y . Wang, Y . Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics, 2019. 2, 3, 4, 6, 8

    2019

  19. [27]

    Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912–1920, 2015. 1

    1912

  20. [28]

    X. Ye, J. Li, H. Huang, L. Du, and X. Zhang. 3d recurrent neural networks with context fusion for point cloud semantic segmentation. In ECCV, pages 415–430. Springer, 2018. 1, 2, 6

    2018

  21. [29]

    Zaheer, S

    M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola. Deep sets. In Advances in Neural Information Processing Systems , pages 3391–3401,

  22. [30]

    w/o Local

    W. Zeng and T. Gevers. 3dcontextnet: Kd tree guided hier- archical learning of point clouds using local and global con- textual cues. In ECCV, pages 314–330, 2018. 1 Appendix A. More Visualizations and Discus- sions on PS2-Net Variants Fig. 6 shows the visualizations of some q...

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.