REVIEW 4 major objections 5 minor 43 references
Cross3DReg: Towards a Large-scale Real-world Cross-source Point Cloud Registration Benchmark
T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A 13,231-pair real-world benchmark for registering point clouds from different LiDAR sensors, with an overlap-based method that cuts rotation error by 63 percent.
desk verdict Genuinely useful real-world cross-source dataset, but the paper never says how ground-truth transforms were obtained, which is load-bearing for a benchmark. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the combination of an overlapping mask predictor (OMP) and a visual-geometric attention-guided matching module (VGAM). OMP linearly projects point superpoint features and image features into a shared space, fuses them with multi-head attention, and outputs a per-superpoint overlap probability, turning it into a binary mask that removes non-overlapping points before matching. VGAM then takes the masked superpoints, injects image context through cross-attention with positional encodings, and refines the result with geometric self-attention, producing features whose similarity matrix is dual-normalized and thresholded to select coarse correspondences. Dense corresp
What would settle it
Independently measure the mutual calibration of the 64-beam rotating lidar and the hybrid lidar in the Cross3DReg platform, e.g., by scanning a calibration target or using a surveyed scene; if recomputed ground-truth transforms differ from the published labels by more than the reported success threshold (2 degrees or 0.5 m) on a nontrivial fraction of pairs, the benchmark and the reported 87.1 percent recall no longer stand.
Extended reading notes
Core claim
The central claim is that Cross3DReg is the currently largest and first real-world multimodal cross-source point cloud registration dataset—13,231 pairs from a 64-beam rotating mechanical lidar and a hybrid semi-solid-state lidar, with accompanying RGB images that share the view but have no known calibration to the lidars. On top of it, the paper claims an overlap-based method that uses the unaligned images to predict which superpoints in each cloud lie in the common region, discards the rest, and then matches with a visual-geometric attention-guided module. In evaluation, the method reports the lowest relative rotation error (6.68 degrees) and relative translation error (1.01 meters) and th
Load-bearing premise
The ground-truth rigid transformations between the two LiDAR sensors are never described or validated; if the mutual calibration between them is inaccurate, every training label, evaluation number, and comparative conclusion in the paper is unfounded.
Editorial extensions
If this is right
- With 13,231 real-world pairs, deep registration models can be trained and tested on genuinely cross-sensor data instead of image-derived reconstructions.
- Same-source registration methods (Predator, CoFiNet, FCGF) collapse to near-zero recall on this data, showing the domain gap is real and measurable.
- The proposed overlap mask prediction plus visual-geometric attention generalizes across pose estimators: it outperforms GeoTrans with LGR, RANSAC, and weighted SVD.
- The dataset and code are to be released, giving the community a common benchmark with fixed thresholds (RRE < 2 degrees, RTE < 0.5 m).
- Using unaligned images to prune non-overlapping points reduces interference from noise and outliers during feature matching.
Reading between the lines
- If the ground-truth transformations are confirmed by an independent calibration procedure, Cross3DReg could become the default stress test for cross-source registration, since it is the first to combine real rotating and hybrid lidar scans at scale.
- The overlap-mask mechanism does not require camera-lidar extrinsics, so it might transfer to other sensor pairs (e.g., radar-lidar or multi-lidar setups) where calibration is unavailable.
- The reported gains mix two contributions—overlap filtering and visual-geometric attention; an ablation isolating each suggests both matter on this dataset, but the interaction with camera coverage is untested, so performance could drop when the common view covers less of the scan.
- Because the images are unaligned, the method learns a soft geometric correspondence between 2D image features and 3D superpoints; a natural stress test would vary the camera's field of view and distance to see how much overlap prediction degrades.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Cross3DReg, a large-scale real-world cross-source point cloud registration dataset containing 13,231 pairs of scans from a rotating mechanical LiDAR and a hybrid semi-solid-state LiDAR, together with unaligned RGB images. It also proposes a registration framework that predicts overlap masks with the help of the unaligned images and then uses a visual-geometric attention module to establish correspondences. On the new dataset, the method is reported to outperform several baselines, including GeoTrans, reducing RRE by 63.2%, RTE by 40.2%, and improving RR by 5.4 percentage points. The dataset and code are promised for public release.
Significance. The claimed contribution is potentially significant: a large real-world cross-source benchmark would address a clear gap in the field, and the proposed method is a reasonable attempt to use unaligned images to guide overlap prediction. The dataset scale and the inclusion of real sensor noise, density differences, and structural pattern variation are strengths. However, the benchmark's validity rests entirely on the accuracy of the ground-truth rigid transformations between the two LiDARs, and this is never described or validated. The evaluation also omits several cross-source-specific state-of-the-art methods cited in the paper, and no external validation on existing benchmarks is provided. If the ground-truth issue is resolved and the comparisons are completed, the paper could make a solid contribution; in its current form, the central empirical claims are not yet fully supported.
major comments (4)
- [Eq. (12) and Tables 2-3] The paper never states how the ground-truth rigid transformations between the hybrid LiDAR scans and the mechanical LiDAR scans were obtained. The text only mentions that camera-LiDAR calibration is unavailable and defers details to the supplementary material. Since every training label (e.g., the overlap mask in Eq. 12) and every evaluation metric (RRE/RTE/RR in Tables 2-3) depends on these transforms, their accuracy is load-bearing. Without a description of the acquisition procedure (e.g., target-based calibration, odometry, SLAM, or manual alignment) and some validation (e.g., residual errors, visual inspection, or cycle consistency), the dataset cannot serve as a reliable benchmark. This issue must be addressed in the main text, not only in the supplementary.
- [Experiments, Tables 2-3] The paper claims state-of-the-art performance but does not compare against several cross-source-specific methods that it cites in Related Work, notably Cross-PCR (Zhao et al. 2025), SPEAL (Xiong et al. 2024), and FF-LOGO (Ma et al. 2024). The only cross-source method included is VRHCF, and it obtains 0% RR. Since the stated contribution is to cross-source registration, omitting the most relevant recent methods makes the 'state-of-the-art' claim unsupported. The experiments should include these methods or explain why they cannot be adapted to the new dataset.
- [Experiments, especially Tables 2-4] The evaluation is conducted solely on the authors' own, unreleased dataset. There is no external validation on existing cross-source benchmarks such as 3DCGS or KITTI-CrossSource, which the paper itself discusses. This is especially important because the literature cited in the paper suggests that methods designed for synthetic cross-source settings may not transfer to real-world data; without a cross-dataset experiment, the generalization of the proposed method is unknown. Additionally, no error bars or multiple-run statistics are reported, so the observed improvements over GeoTrans (e.g., RR 87.1 vs. 81.7) cannot be assessed for statistical significance.
- [Eq. (12) and Method] The ground-truth overlap mask is defined as whether a source superpoint has a correspondence to a target superpoint under the unknown ground-truth transformation (with an apparent typo: 'correspondent to P_j' should presumably be 'Q_j'). This means the mask supervision is derived from the same correspondences that the matching module is trained to produce and that are later used to evaluate registration. While this is not circular in a logical sense, it creates a strong dependency: any bias or error in the ground-truth transform propagates into both the mask predictor and the matching module. Independent validation of the transforms, or an alternative source of overlap labels, is necessary to break this loop.
minor comments (5)
- [Eq. (12)] Typo: '\hat{P}_i correspondent to \hat{P}_j' should read '\hat{P}_i correspondent to \hat{Q}_j' (or similar), as the correspondence is between source and target point clouds.
- [References] The paper cites Huang et al. 2021b as the source of the 3DCGS dataset, but the corresponding reference in the bibliography is 'A comprehensive survey on point cloud registration'. The correct dataset reference should be provided.
- [Tables 2-4, Abstract] The full method is reported as RR=87.1 in Table 2 but RR=87.2 in Table 4; both should be consistent. Also, the abstract's '5.4%' improvement is a percentage-point increase, not a relative improvement; the phrasing is misleading.
- [Method, loss definition] The phrase 'Euclidean paradigm' in Eq. (1) should be 'Euclidean norm'. Minor.
- [Table 1] The KITTI-CrossSource row lists the number of pairs as '− ∗' and open-source as '×', which is consistent with the text, but 'Img' is listed as '✓' although the dataset is unreleased; the table could clarify that the image modality is present in the described dataset even if not publicly available.
Circularity Check
No significant circularity; the derivation is an empirical benchmark plus a learned registration pipeline, not a tautological reduction.
full rationale
The paper contains no derivation step in which an output quantity is defined as its own input. The dataset size (13,231 pairs) is a measured collection statistic, not an implication of the method. The overlap-mask predictor is supervised by Focal Loss (Eq. 14) against ground-truth masks (Eq. 12); even though those masks are generated from point-cloud correspondences that also supervise matching losses, a learned predictor can fail to match those labels, so the supervision is not a tautology. The attention fusion and matching equations (Eqs. 2-11) are standard learnable transformations; no equation reduces the final pose to the input features by construction. The reported RRE/RTE/RR improvements are empirical comparisons on the authors' own dataset, with no fitted parameter renamed as a prediction. The paper does cite the authors' earlier works (Xu et al. 2024, 2025), but only in related-work positioning; they are not load-bearing for the dataset claim or the method. The main unresolved issue is not circularity but benchmark validity: the paper never states how the ground-truth rigid transforms between the mechanical and hybrid LiDAR were obtained or validated, and it does not report a train/test split. Those omissions affect trustworthiness and generalization but do not make the derivation circular.
Assumptions & free parameters
free parameters (3)
- Voxel size for preprocessing =
0.25 m
- Overlap confidence threshold lambda (Eq. 4) =
0.5
- Focal loss hyperparameters alpha and gamma =
alpha=0.25, gamma=2.0
assumptions (4)
- domain assumption The two LiDAR sensors were mounted on the same UGV so that every source-target pair is related by a fixed rigid transformation that can be recovered with usable accuracy.
- domain assumption Unaligned RGB images share sufficient field-of-view and spatial layout with the point clouds that fusing image features helps predict the point-cloud overlap region.
- domain assumption The superpoint-level overlap ground truth defined in Eq. (12), based on whether a source superpoint has a correspondent in the target, is a faithful label for geometric overlap.
- domain assumption KPConv features trained on this dataset transfer across scenes within the dataset and the split is representative.
Cite this review
Pith. "Pith review of Cross3DReg: Towards a Large-scale Real-world Cross-source Point Cloud Registration Benchmark." pith.science (2026). https://pith.science/paper/4ONBHYYI
@misc{pith2026250906456,
author = {Pith},
title = {Pith review of: Cross3DReg: Towards a Large-scale Real-world Cross-source Point Cloud Registration Benchmark},
year = {2026},
howpublished = {\url{https://pith.science/paper/4ONBHYYI}},
note = {Machine review of arXiv:2509.06456}
}
abstract
Cross-source point cloud registration, which aims to align point cloud data from different sensors, is a fundamental task in 3D vision. However, compared to the same-source point cloud registration, cross-source registration faces two core challenges: the lack of publicly available large-scale real-world datasets for training the deep registration models, and the inherent differences in point clouds captured by multiple sensors. The diverse patterns induced by the sensors pose great challenges in robust and accurate point cloud feature extraction and matching, which negatively influence the registration accuracy. To advance research in this field, we construct Cross3DReg, the currently largest and real-world multi-modal cross-source point cloud registration dataset, which is collected by a rotating mechanical lidar and a hybrid semi-solid-state lidar, respectively. Moreover, we design an overlap-based cross-source registration framework, which utilizes unaligned images to predict the overlapping region between source and target point clouds, effectively filtering out redundant points in the irrelevant regions and significantly mitigating the interference caused by noise in non-overlapping areas. Then, a visual-geometric attention guided matching module is proposed to enhance the consistency of cross-source point cloud features by fusing image and geometric information to establish reliable correspondences and ultimately achieve accurate and robust registration. Extensive experiments show that our method achieves state-of-the-art registration performance. Our framework reduces the relative rotation error (RRE) and relative translation error (RTE) by $63.2\%$ and $40.2\%$, respectively, and improves the registration recall (RR) by $5.4\%$, which validates its effectiveness in achieving accurate cross-source registration.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Ao, S.; Hu, Q.; Yang, B.; Markham, A.; and Guo, Y. 2021. Spinnet: Learning a general surface descriptor for 3d point cloud registration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11753--11762
work page 2021
-
[4]
Bai, X.; Luo, Z.; Zhou, L.; Fu, H.; Quan, L.; and Tai, C.-L. 2020. D3feat: Joint learning of dense detection and description of 3d local features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6359--6367
work page 2020
-
[5]
Besl, P. J.; and McKay, N. D. 1992. Method for registration of 3-D shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, 586--606. Spie
work page 1992
-
[6]
Chen, H.; Wei, Z.; Xu, Y.; Wei, M.; and Wang, J. 2022. Imlovenet: Misaligned image-supported registration network for low-overlap point cloud pairs. In ACM SIGGRAPH 2022 conference proceedings, 1--9
work page 2022
-
[7]
Choy, C.; Park, J.; and Koltun, V. 2019. Fully convolutional geometric features. In Proceedings of the IEEE/CVF international conference on computer vision, 8958--8966
work page 2019
-
[8]
Huang, S.; Gojcic, Z.; Usvyatsov, M.; Wieser, A.; and Schindler, K. 2021 a . Predator: Registration of 3d point clouds with low overlap. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 4267--4276
work page 2021
Show all 43 references
-
[9]
Huang, X.; Fan, L.; Wu, Q.; Zhang, J.; and Yuan, C. 2019. Fast registration for cross-source point clouds by using weak regional affinity and pixel-wise refinement. In 2019 IEEE International Conference on Multimedia and Expo (ICME), 1552--1557. IEEE
2019
-
[10]
Huang, X.; Mei, G.; and Zhang, J. 2023 a . Cross-source point cloud registration: Challenges, progress and prospects. Neurocomputing, 548: 126383
2023
-
[11]
Huang, X.; Mei, G.; and Zhang, J. 2023 b . Cross-source point cloud registration: Challenges, progress and prospects. Neurocomputing, 548: 126383
2023
-
[12]
Huang, X.; Mei, G.; Zhang, J.; and Abbas, R. 2021 b . A comprehensive survey on point cloud registration. arXiv preprint arXiv:2103.02690
2021 arXiv
-
[13]
Huang, X.; Zhang, J.; Fan, L.; Wu, Q.; and Yuan, C. 2017 a . A systematic approach for cross-source point cloud registration by preserving macro and micro structures. IEEE Transactions on Image Processing, 26(7): 3261--3276
2017
-
[14]
Huang, X.; Zhang, J.; Wu, Q.; Fan, L.; and Yuan, C. 2017 b . A coarse-to-fine algorithm for matching and registration in 3D cross-source point clouds. IEEE Transactions on Circuits and Systems for Video Technology, 28(10): 2965--2977
2017
-
[15]
Jiang, H.; Xie, J.; Yang, J.; Yu, L.; and Zheng, J. 2025. Zero-shot RGB-D Point Cloud Registration with Pre-trained Large Vision Model. In Proceedings of the Computer Vision and Pattern Recognition Conference, 16943--16952
2025
-
[16]
Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Doll \'a r, P. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980--2988
2017
-
[17]
Lu, W.; Wan, G.; Zhou, Y.; Fu, X.; Yuan, P.; and Song, S. 2019. Deepvcp: An end-to-end deep neural network for point cloud registration. In Proceedings of the IEEE/CVF international conference on computer vision, 12--21
2019
-
[18]
Ma, N.; Wang, M.; Han, Y.; and Liu, Y.-J. 2024. Ff-logo: Cross-modality point cloud registration with feature filtering and local to global optimization. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 744--750. IEEE
2024
-
[19]
Mu, J.; Bie, L.; Du, S.; and Gao, Y. 2024. ColorPCR: Color point cloud registration with multi-stage geometric-color fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21061--21070
2024
-
[20]
Qin, Z.; Yu, H.; Wang, C.; Guo, Y.; Peng, Y.; Ilic, S.; Hu, D.; and Xu, K. 2023. Geotransformer: Fast and robust point cloud registration with geometric transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8): 9806--9821
2023
-
[21]
S.; and Gao, Y
Ren, C.; Feng, Y.; Zhang, W.; Zhang, X.-P. S.; and Gao, Y. 2024. Multi-scale consistency for robust 3D registration via hierarchical Sinkhorn tree. Advances in Neural Information Processing Systems, 37: 91798--91826
2024
-
[22]
Rocco, I.; Cimpoi, M.; Arandjelovi \'c , R.; Torii, A.; Pajdla, T.; and Sivic, J. 2018. Neighbourhood consensus networks. Advances in neural information processing systems, 31
2018
-
[23]
Rusinkiewicz, S.; and Levoy, M. 2001. Efficient variants of the ICP algorithm. In Proceedings third international conference on 3-D digital imaging and modeling, 145--152. IEEE
2001
-
[24]
B.; Blodow, N.; and Beetz, M
Rusu, R. B.; Blodow, N.; and Beetz, M. 2009. Fast point feature histograms (FPFH) for 3D registration. In 2009 IEEE international conference on robotics and automation, 3212--3217. IEEE
2009
-
[25]
B.; Blodow, N.; Marton, Z
Rusu, R. B.; Blodow, N.; Marton, Z. C.; and Beetz, M. 2008. Aligning point cloud views using persistent feature histograms. In 2008 IEEE/RSJ international conference on intelligent robots and systems, 3384--3391. IEEE
2008
-
[26]
Sarlin, P.-E.; DeTone, D.; Malisiewicz, T.; and Rabinovich, A. 2020. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4938--4947
2020
-
[27]
Segal, A.; Haehnel, D.; and Thrun, S. 2009. Generalized-icp. In Robotics: science and systems, volume 2, 435. Seattle, WA
2009
-
[28]
Sun, J.; Shen, Z.; Wang, Y.; Bao, H.; and Zhou, X. 2021. LoFTR: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8922--8931
2021
-
[29]
R.; Deschaud, J.-E.; Marcotegui, B.; Goulette, F.; and Guibas, L
Thomas, H.; Qi, C. R.; Deschaud, J.-E.; Marcotegui, B.; Goulette, F.; and Guibas, L. J. 2019. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, 6411--6420
2019
-
[30]
N.; Kaiser, .; and Polosukhin, I
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30
2017
-
[31]
Wang, G.; Zheng, Y.; Wu, Y.; Guo, Y.; Liu, Z.; Zhu, Y.; Burgard, W.; and Wang, H. 2025. End-to-end 2d-3d registration between image and lidar point cloud for vehicle localization. IEEE Transactions on Robotics
2025
-
[32]
Wang, H.; Liu, Y.; Dong, Z.; and Wang, W. 2022. You only hypothesize once: Point cloud registration with rotation-equivariant descriptors. In Proceedings of the 30th ACM International Conference on Multimedia, 1630--1641
2022
-
[33]
Wimbauer, F.; Yang, N.; Von Stumberg, L.; Zeller, N.; and Cremers, D. 2021. MonoRec: Semi-supervised dense reconstruction in dynamic environments from a single moving camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6112--6122
2021
-
[34]
Xiong, K.; Zheng, M.; Xu, Q.; Wen, C.; Shen, S.; and Wang, C. 2024. Speal: Skeletal prior embedded attention learning for cross-source point cloud registration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 6279--6287
2024
-
[35]
Xu, H.; Liu, S.; Wang, G.; Liu, G.; and Zeng, B. 2021. Omnet: Learning overlapping mask for partial-to-partial point cloud registration. In Proceedings of the IEEE/CVF international conference on computer vision, 3132--3141
2021
-
[36]
Xu, Z.; Gao, X.; Jiang, X.; Cheng, S.; Zhang, Q.; Li, W.; and Gao, X. 2025. S2Reg: Structure-semantics collaborative point cloud registration. Pattern Recognition, 161: 111290
2025
-
[37]
Xu, Z.; Jiang, X.; Gao, X.; Gao, R.; Gu, C.; Zhang, Q.; Li, W.; and Gao, X. 2024. IGReg: Image-Geometry-Assisted Point Cloud Registration via Selective Correlation Fusion. IEEE Transactions on Multimedia
2024
-
[38]
Yu, H.; Li, F.; Saleh, M.; Busam, B.; and Ilic, S. 2021. Cofinet: Reliable coarse-to-fine correspondences for robust pointcloud registration. Advances in Neural Information Processing Systems, 34: 23872--23884
2021
-
[39]
Yu, H.; Qin, Z.; Hou, J.; Saleh, M.; Li, D.; Busam, B.; and Ilic, S. 2023. Rotation-invariant transformer for point cloud matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5384--5393
2023
-
[40]
Zhang, Y.; Yu, J.; Huang, X.; Zhou, W.; and Hou, J. 2022 a . Pcr-cg: Point cloud registration via deep explicit color and geometry. In European Conference on Computer Vision, 443--459. Springer
2022
-
[41]
Zhang, Z.; Sun, J.; Dai, Y.; Zhou, D.; Song, X.; and He, M. 2022 b . End-to-end learning the partial permutation matrix for robust 3D point cloud registration. In Proceedings of the AAAI conference on Artificial Intelligence, volume 36, 3399--3407
2022
-
[42]
Zhao, G.; Du, Z.; Guo, Z.; and Ma, H. 2024. Vrhcf: Cross-source point cloud registration via voxel representation and hierarchical correspondence filtering. In 2024 IEEE International Conference on Multimedia and Expo (ICME), 1--6. IEEE
2024
-
[43]
Zhao, G.; Guo, Z.; Du, Z.; and Ma, H. 2025. Cross-PCR: A Robust Cross-Source Point Cloud Registration Framework. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 10403--10411
2025
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.