REVIEW 2 major objections 7 minor 4 cited by
PCRNet: Point Cloud Registration Network using PointNet Encoding
T0 review · 2 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that the rigid transform aligning two point clouds can be obtained by comparing PointNet feature vectors, and that an iterative version matches a global-optimal baseline at orders-of-magnitude lower compute.
desk verdict A solid, clearly-written engineering paper whose honest niche is category-specific fast registration; the abstract and Section 4.3 overstate generalization and accuracy. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is PointNet's global feature vector, produced by a symmetric max-pooling operation over per-point multilayer perceptron outputs; the paper treats this vector as a pose-sensitive shape encoding, so two point clouds of the same shape in different poses should have measurably different features. In PCRNet the transform is regressed directly: a Siamese pair of shared-weight MLPs (sizes 64, 64, 64, 128, 1024) encodes source and template, the two global feature vectors are concatenated, and five fully connected layers (1024, 1024, 512, 512, 256) plus a seven-unit output layer predict translation and a unit quaternion. In the iterative version, the predicted transform is applied to the source before the next encoding, the final transform is the composition of the per-iteration transforms, and the training loss is the Earth Mover Distance between the transformed source and the template.
What would settle it
Take a rotationally symmetric object such as a uniformly sampled sphere, rotate the source by a known angle inside the training range, and see whether PCRNet recovers it; because max-pooled features of a sphere are nearly invariant to rotation, the network should fail, exposing the dependence on pose information actually surviving in the feature vector. More quantitatively, plot $\|\varphi(P_S) - \varphi(P_T)\|$ against rotation angle for a fixed shape; any flat region in that curve marks an interval of poses the regressor cannot distinguish.
Extended reading notes
Core claim
The paper's central claim is that the rigid-body transformation $T \in SE(3)$ aligning a source point cloud $P_S$ to a template $P_T$ can be recovered from the PointNet global feature vectors $\varphi(P_S)$ and $\varphi(P_T)$: the goal is to find the transform that minimizes the difference between the two feature vectors. Rather than solving that minimization with a classical optimizer, PCRNet concatenates the two feature vectors and trains fully connected layers to output the translation vector and the normalized rotation quaternion directly. The iterative PCRNet repeats the process: it applies the predicted transform to the source, re-encodes, and composes the per-iteration transforms as $T = T^{(n)} \times \dots \times T^{(1)}$. On the paper's experiments, iterative PCRNet reaches an AUC (area under the success-ratio curve) of 0.9943 on noisy data with a mean rotation error around one degree, compared with Go-ICP's AUC of 1.0 and mean error of 0.45 degrees, while taking 146 ms on average versus Go-ICP's $2.7\times 10^5$ ms.
Load-bearing premise
The load-bearing assumption is that PointNet's max-pooled global feature vector retains enough information about both shape and orientation that a fixed fully connected network can regress the rigid transform from the two feature vectors alone; if max-pooling discards pose-critical details, the regression problem is ill-posed.
Editorial extensions
If this is right
- Iterative PCRNet, trained on a specific object category, registers noisy point clouds with accuracy close to the globally optimal Go-ICP while running about three orders of magnitude faster in the reported experiments.
- Because the whole pipeline is differentiable and avoids closest-point correspondence searches, it can execute on a GPU and be integrated as a trainable module inside larger deep networks.
- The single-shot PCRNet trades accuracy for speed, providing a fast pre-alignment that the paper suggests may be sufficient as a preliminary step for detection and segmentation.
- The method's noise robustness is tied to the noise distribution seen in training: performance is best near the training noise level and degrades toward ICP-level beyond it.
- PointNetLK generalizes better to object categories unseen in training, whereas iterative PCRNet excels when some representation of the test category is present, indicating a specificity-generalizability trade-off in PointNet-based registration.
Reading between the lines
- Beyond the paper's claims, the same feature-comparison design should extend to partial and occluded point clouds by replacing max-pooling with a pooling scheme that preserves local spatial cues; the paper lists partial clouds only as future work.
- Beyond the paper's claims, the network's success on different instances of the same category suggests it learns a category-level pose prior rather than a purely geometric one; a direct test is measuring how accuracy degrades as intra-category shape variation increases.
- Beyond the paper's claims, the fixed MLP regressor trained on rotations up to ±45° should be sensitive to out-of-range poses; a natural stress test is whether extrapolation beyond that range fails gracefully or collapses.
- Beyond the paper's claims, a minimal well-posedness check is whether the feature distance $\|\varphi(P_S) - \varphi(P_T)\|$ grows monotonically with rotation angle; if the curve is flat over any interval, the regression is ill-posed on that interval regardless of training.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces PCRNet, a learning-based point cloud registration framework that encodes source and template point clouds with a shared PointNet and regresses a rigid transformation from the concatenated global feature vectors. Two variants are presented: a single-shot network and an iterative network that composes incremental predictions per Eq. (1). Training uses an Earth Mover's Distance loss between the transformed source and template. Experiments on ModelNet40 investigate generalization across categories, category-specific training, single-model training, robustness to Gaussian noise, and runtime against ICP, PointNetLK, and Go-ICP; a real-world chair replacement demo on S3DIS is also shown.
Significance. The paper's main value is empirical: it demonstrates that a simple fully-connected regressor on PointNet features can align point clouds accurately when trained on the same object category, with strong noise robustness and fast inference, and it positions PCRNet within the PointNet-based registration family alongside PointNetLK. The authors release code, and the experimental protocol is reasonably thorough for the time, including AUC curves, noise sweeps, and qualitative real-world results. However, the headline claims of generalization to unseen shapes and accuracy comparable to Go-ICP are not supported by the paper's own numbers, which weakens the significance as stated.
major comments (2)
- [Section 4.3, Table 1] The statement 'iterative PCRNet is similar to Go-ICP in terms of accuracy' is not supported by the reported numbers. In Table 1, Go-ICP achieves mean rotation error 0.45 deg and translation error 0.0016, while iterative PCRNet achieves 1.03 deg and 0.0085, i.e., more than 2x and 5x larger, respectively, with AUC 0.994 versus 1.000. Moreover, the comparison uses a single car model with 100 random transforms and category-specific training, which is the most favorable regime for the learned method and the least favorable for Go-ICP. I recommend either removing this claim or making it precise, e.g., 'comparable to Go-ICP among learning-based and classical baselines on category-specific benchmarks.'
- [Abstract and Section 4.1] The abstract's claim that the framework can produce approaches 'general to unseen shapes' is contradicted by the unseen-category result in Section 4.1: iterative PCRNet's AUC is 0.682, below ICP's 0.802 and far below PointNetLK's 0.998. The paper later attributes this to the 'inherent limitation of the learning capacity of PCRNet,' which is reasonable, but the abstract and Section 6 should be revised to acknowledge that generalization to unseen shapes is weak and that the main strength is category-specific accuracy.
minor comments (7)
- [Section 3.1] The choice of five fully connected layers is justified only as 'seemed to be sufficient'; an ablation or a sentence on the tuning procedure would help reproducibility.
- [Figure 5] In subfigures (a) and (b), the reported rotation and translation errors are identical (2.14 deg, 0.0056 units) even though the training data differ; please check whether this is a typo.
- [Section 4.1] When reporting 'AUC of PCRNet was 0.998' for the same-category experiment, it is unclear whether this refers to single-shot PCRNet or iterative PCRNet; please label the model variant explicitly.
- [Section 4.2] The training noise is described as having maximum std dev 0.04, but Fig. 6 tests up to 0.20; clarify whether the test noise range exceeds the training range and discuss implications.
- [Section 5] The sentence 'we perform model replacement in use the semantic segmentation network' is missing a word; also, the claim that Go-ICP 'did not improve upon ICP' in this example is anecdotal and could be expanded.
- [Section 3.2] The iterative composition in Eq. (1) is stated without a convergence criterion or analysis; since the paper reports fixed numbers of iterations, a sentence on why 8 training iterations and 20 test iterations were chosen would be helpful.
- [Section 3] The premise that 'global feature vectors contain information about the geometry as well as the orientation' is central to the method but is asserted rather than verified; an analysis or ablation of the pose-sensitivity of PointNet features would strengthen the paper.
Circularity Check
No circularity: PCRNet's transform regression is an empirical supervised learning claim; the EMD loss ties the output to the registration objective without presupposing the result.
full rationale
The derivation chain is not circular. The method defines a Siamese PointNet encoder phi, concatenates global features, and regresses T in SE(3) through fully connected layers; the statement 'we calculate the rigid-body transformation T ... that minimizes the difference between phi(PS) and phi(PT)' (Sec. 3) is a design motivation, not a derivation that assumes the answer. The regression target is learned from supervised pairs generated by ground-truth rigid transforms, and the EMD loss (Eq. 2) measures alignment of the transformed source to the template; minimizing this loss during training is the learning objective, not a circular constraint. Iterative composition (Eq. 1) is an empirical refinement with no convergence theorem, which is a correctness concern rather than circularity. PointNetLK [2] is cited as a baseline and shares authors with this paper, but the paper's claims about it are tested in the paper's own experiments (e.g., Fig. 4, Table 1) rather than imported as load-bearing evidence. No fitted parameter is relabeled as a prediction, no uniqueness theorem is invoked, and no known result is renamed. The central claims are self-contained empirical claims; Table 1's 'similar to Go-ICP' wording is disputable on numerical evidence, but that is a correctness/evidence issue, not a circular reduction. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (5)
- Number of fully connected layers =
5 for PCRNet, 3 for iterative PCRNet
- Training iterations =
8
- Max test iterations =
20
- Training noise range =
0.04 max std dev
- Learning rate and decay =
1e-3, decay 0.7 every 3e6 steps
assumptions (4)
- domain assumption PointNet global feature vectors encode both geometry and orientation of the point cloud.
- domain assumption The Earth Mover Distance is a suitable training objective for point cloud registration.
- domain assumption The training distribution of random rotations in [-45,45] degrees and translations in [-1,1] is representative of test conditions.
- domain assumption The composition of per-iteration transforms converges to the true transformation.
Cite this review
Pith. "Pith review of PCRNet: Point Cloud Registration Network using PointNet Encoding." pith.science (2026). https://pith.science/paper/VN25QEVL
@misc{pith2026190807906,
author = {Pith},
title = {Pith review of: PCRNet: Point Cloud Registration Network using PointNet Encoding},
year = {2026},
howpublished = {\url{https://pith.science/paper/VN25QEVL}},
note = {Machine review of arXiv:1908.07906}
}
read the original abstract
PointNet has recently emerged as a popular representation for unstructured point cloud data, allowing application of deep learning to tasks such as object detection, segmentation and shape completion. However, recent works in literature have shown the sensitivity of the PointNet representation to pose misalignment. This paper presents a novel framework that uses the PointNet representation to align point clouds and perform registration for applications such as tracking, 3D reconstruction and pose estimation. We develop a framework that compares PointNet features of template and source point clouds to find the transformation that aligns them accurately. Depending on the prior information about the shape of the object formed by the point clouds, our framework can produce approaches that are shape specific or general to unseen shapes. The shape specific approach uses a Siamese architecture with fully connected (FC) layers and is robust to noise and initial misalignment in data. We perform extensive simulation and real-world experiments to validate the efficacy of our approach and compare the performance with state-of-art approaches.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 4 Pith papers
-
APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT--IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction
APCReg combines ordered projection alignment, overlap-gated learned matching, arch-structured hypothesis selection, and a test-time retention rule to improve CBCT-to-IOS dental registration, reporting 0.87 mm mean Cha...
-
Doctoral Thesis: Geometric Deep Learning For Camera Pose Prediction, Registration, Depth Estimation, and 3D Reconstruction
A PhD thesis showing that adding geometric priors (skyline, normals, focus cues, wavelet depth) to deep networks improves pose estimation, registration, depth prediction, and reconstruction.
-
Hierarchical Error Assessment of CAD Models for Aircraft Manufacturing-and-Measurement
A hierarchical error-assessment framework for aircraft CAD parts performs global, part-level, and circular-feature comparisons between scanned point clouds and the reference CAD model.
-
Deep Loss Convexification for Learning Iterative Models
Adding star-convexity hinge losses during training makes a model's loss landscape bowl-shaped around the ground truth and improves iterative predictions on RNN, point cloud registration, and image alignment tasks.
Reference graph
Works this paper leans on
-
[1]
M. Angelina Uy and G. Hee Lee. Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4470–4479, 2018. 2
work page 2018
-
[2]
Y . Aoki, H. Goforth, R. A. Srivatsan, and S. Lucey. Point- NetLK: Robust & Efficient Point Cloud Registration using PointNet. arXiv preprint arXiv:1903.05711, 2019. 1, 2, 3, 4, 5, 7
work page Pith review arXiv 1903
- [3]
-
[4]
R. Arun Srivatsan, M. Xu, N. Zevallos, and H. Choset. Prob- abilistic pose estimation using a Bingham distribution-based linear filter. The International Journal of Robotics Research, 37(13-14):1610–1631, 2018. 2
work page 2018
-
[5]
S. Baker and I. Matthews. Lucas-Kanade 20 years on: A uni- fying framework. International journal of computer vision , 56(3):221–255, 2004. 2, 3
work page 2004
-
[6]
P. Besl and N. D. McKay. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239–256, Feb 1992. 7
work page 1992
-
[7]
P. J. Besl and N. D. McKay. Method for registration of 3-d shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, volume 1611, pages 586–607. International Soci- ety for Optics and Photonics, 1992. 1, 2, 8
work page 1992
-
[8]
H. Bristow, J. Valmadre, and S. Lucey. Dense semantic cor- respondence where every pixel is a classifier. InProceedings of the IEEE International Conference on Computer Vision , pages 4024–4031, 2015. 2
work page 2015
Show all 41 references
-
[9]
Eckart, K
B. Eckart, K. Kim, and J. Kautz. Fast and accurate point cloud registration using trees of gaussian mixtures. arXiv preprint arXiv:1807.02587, 2018. 2
2018 arXiv
-
[10]
Elbaz, T
G. Elbaz, T. Avraham, and A. Fischer. 3d point cloud reg- istration for localization using a deep neural network auto- encoder. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition , pages 4631–4640,
-
[11]
Eldar, M
Y . Eldar, M. Lindenbaum, M. Porat, and Y . Y . Zeevi. The farthest point strategy for progressive image sampling.IEEE Transactions on Image Processing, 6(9):1305–1315, 1997. 4
1997
-
[12]
H. Fan, H. Su, and L. J. Guibas. A point set generation net- work for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 605–613, 2017. 2
2017
-
[13]
Gelfand, N
N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann. Ro- bust global registration. In Symposium on geometry process- ing, volume 2, page 5, 2005. 1, 2
2005
-
[14]
Georgakis, S
G. Georgakis, S. Karanam, Z. Wu, and J. Kosecka. Matching RGB Images to CAD Models for Object Pose Estimation. arXiv preprint arXiv:1811.07249, 2018. 2
2018 arXiv
-
[15]
Glover, G
J. Glover, G. Bradski, and R. B. Rusu. Monte carlo pose estimation with quaternion kernels and the distribution. In Robotics: Science and Systems, volume 7, page 97, 2012. 2
2012
-
[16]
Y . Guo, M. Bennamoun, F. Sohel, M. Lu, and J. Wan. 3D object recognition in cluttered scenes with local surface fea- tures: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(11):2270–2287, 2014. 2
2014
-
[17]
D. Held, S. Thrun, and S. Savarese. Learning to track at 100 fps with deep regression networks. In European Conference on Computer Vision, pages 749–765. Springer, 2016. 3
2016
-
[18]
M. B. Horowitz, N. Matni, and J. W. Burdick. Convex re- laxations of SE(2) and SE(3) for visual pose estimation. In IEEE International Conference on Robotics and Automation (ICRA), pages 1148–1154. IEEE, 2014. 2
2014
-
[19]
Izatt, H
G. Izatt, H. Dai, and R. Tedrake. Globally Optimal Object Pose Estimation in Point Clouds with Mixed-Integer Pro- gramming. In International Symposium on Robotics Re- search, 12 2017. 2, 8
2017
-
[20]
Jost and H
T. Jost and H. Hugli. A multi-resolution scheme ICP algo- rithm for fast shape registration. In Proceedings. First In- ternational Symposium on 3D Data Processing Visualization and Transmission, pages 540–543. IEEE, 2002. 2
2002
-
[21]
B. D. Lucas, T. Kanade, et al. An iterative image registration technique with an application to stereo vision. Proceedings of 7th IJCAI, 1981. 2, 3
1981
-
[22]
Z. Ma, B. Liu, F. Zhou, and J. Chen. Point reg net: Invariant features for point cloud registration using in image-guided radiation therapy. Journal of Computer and Communica- tions, 06:116–125, 01 2018. 1
2018
-
[23]
Makadia, A
A. Makadia, A. Patterson, and K. Daniilidis. Fully automatic registration of 3D point clouds. In Computer Vision and Pat- tern Recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 1297–1304. IEEE, 2006. 2
2006
-
[24]
Maron, N
H. Maron, N. Dym, I. Kezurer, S. Kovalsky, and Y . Lip- man. Point registration via efficient convex relaxation. ACM Transactions on Graphics (TOG), 35(4):73, 2016. 2
2016
-
[25]
Maturana and S
D. Maturana and S. Scherer. V oxnet: A 3d convolutional neural network for real-time object recognition. InIntelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pages 922–928. IEEE, 2015. 1, 2
2015
-
[26]
Ovsjanikov, Q
M. Ovsjanikov, Q. M ´erigot, F. M´emoli, and L. Guibas. One point isometric matching with the heat kernel. In Computer Graphics Forum, volume 29, pages 1555–1564. Wiley On- line Library, 2010. 2
2010
-
[27]
C. R. Qi, W. Liu, C. Wu, H. Su, and L. J. Guibas. Frustum pointnets for 3d object detection from RGB-D data. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 918–927, 2018. 1, 2
2018
-
[28]
C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 1(2):4, 2017. 1, 2, 8
2017
-
[29]
C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hi- erarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems , pages 5099–5108, 2017. 1, 2
2017
-
[30]
Rusinkiewicz and M
S. Rusinkiewicz and M. Levoy. Efficient variants of the ICP algorithm. In 3dim, volume 1, pages 145–152, 2001. 2
2001
-
[31]
R. B. Rusu, N. Blodow, and M. Beetz. Fast point feature his- tograms (FPFH) for 3D registration. In IEEE International Conference on Robotics and Automation, pages 3212–3217. IEEE, 2009. 1, 2
2009
-
[32]
V ongkulbhisal, F
J. V ongkulbhisal, F. De la Torre, and J. P. Costeira. Dis- criminative optimization: Theory and applications to point cloud registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4104– 4112, 2017. 2
2017
-
[33]
V ongkulbhisal, B
J. V ongkulbhisal, B. Irastorza Ugalde, F. De la Torre, and J. P. Costeira. Inverse composition discriminative optimization for point cloud registration. InProceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 2993–3001, 2018. 2
2018
-
[34]
Y . Wang, Y . Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph CNN for learning on point clouds. arXiv preprint arXiv:1801.07829, 2018. 1, 2
2018 arXiv
-
[35]
Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1912–1920, 2015. 1, 2, 4, 8
1912
-
[36]
Xiang, T
Y . Xiang, T. Schmidt, V . Narayanan, and D. Fox. PoseCNN: A convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199, 2017. 2, 4
2017 arXiv
-
[37]
J. Yang, H. Li, D. Campbell, and Y . Jia. Go-ICP: A glob- ally optimal solution to 3D ICP point-set registration. IEEE transactions on pattern analysis and machine intelligence , 38(11):2241–2254, 2016. 1, 2, 7
2016
-
[38]
Z. J. Yew and G. H. Lee. 3dfeat-net: Weakly supervised local 3d features for point cloud registration. In European Conference on Computer Vision , pages 630–646. Springer,
-
[39]
W. Yuan, D. Held, C. Mertz, and M. Hebert. Iterative transformer network for 3d point cloud. arXiv preprint arXiv:1811.11209, 2018. 1, 2, 8
2018 arXiv
-
[40]
W. Yuan, T. Khot, D. Held, C. Mertz, and M. Hebert. PCN: Point Completion Network. In 3D Vision (3DV), 2018 Inter- national Conference on, 2018. 2
2018
-
[41]
Zhou and O
Y . Zhou and O. Tuzel. V oxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 4490–4499, 2018. 2
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.