REVIEW 4 major objections 5 minor 27 references
Cross-Modal Visual Relocalization in Prior LiDAR Maps Utilizing Intensity Textures
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read LiDAR intensity textures let a camera relocalize in prior maps
desk verdict A sensible intensity-texture pipeline for camera-to-LiDAR-map relocalization, but the evidence is narrow and the ground-truth provenance is under-reported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the intensity-channel panoramic map image produced by HEC projection from the prior LiDAR map. HEC adjusts cube-face sampling with an arctan-based formula to densify and homogenize the projected pixels, discarding the top and ground surfaces, and the two-stage equalization then aligns the intensity histogram with grayscale image appearance. This projected image acts as the proxy that brings camera pixels and LiDAR points into one modality, allowing standard visual place recognition and matching networks to carry cross-modal association without special cross-modal training.
What would settle it
Run the exact pipeline with a sparser or differently structured LiDAR map (for example, a 16-beam Velodyne scan of a highway or tunnel) and measure RR(1m/3°) on a comparably sized query set; if the recall drops below roughly 0.5, the intensity-texture transfer claim collapses. Alternatively, replace HEC with equirectangular projection while keeping every other module identical and check whether the recall drop matches the paper's ablation, where RR(1m/3°) falls from 0.97 to 0.61 on the College sequence.
Extended reading notes
Core claim
The central claim is that intensity-texture consistency between LiDAR intensity images and grayscale camera images is sufficient for cross-modal visual relocalization, provided the map is projected with HEC (Hybrid Equiangular Cube Projection) and equalized in two stages: a 3D map histogram equalization for initial normalization followed by CLAHE for contrast enhancement. This makes pretrained visual models directly applicable: NetVLAD for coarse retrieval, SuperPoint for local feature extraction, and LightGlue for matching, with the resulting 2D-2D correspondences lifted to 2D-3D using the stored map points. The paper reports Recall@1 of 0.73 on the College sequence and relocalization recall RR(1m/3°) of 0.97 and 0.79 on the two campus sequences, clearly exceeding a depth-channel variant of the same pipeline.
Load-bearing premise
The entire pipeline assumes that after HEC projection and two-stage equalization, LiDAR intensity images are texture-consistent enough with grayscale camera images that off-the-shelf visual networks can match them reliably, and this assumption is tested only on two self-collected campus sequences using a Livox Avia LiDAR.
Editorial extensions
If this is right
- If the intensity-texture transfer holds, existing pretrained visual localization stacks can be reused for LiDAR-map relocalization with zero fine-tuning.
- The intensity channel can replace depth or geometry as the primary cross-modal bridge, simplifying the 2D-3D association step.
- The complete hierarchical pipeline offers single-frame global localization, not just place recognition or registration alone, which is directly useful for loop closure and global pose initialization in LiDAR SLAM.
- The method's success on campus scenes suggests that similar intensity-based texture proxies could be adopted in other map-based localization settings where camera images must be matched against non-visual sensors.
Reading between the lines
- The reliance on pretrained visual networks implies the strongest test is LiDAR sparsity: on a 16-beam or otherwise sparser sensor, the intensity images would have larger holes and the texture consistency would degrade, likely dropping both retrieval and pose recall.
- The paper's own stated future work of retraining retrieval and matching networks to learn intensity-grayscale consistency suggests the authors see the current off-the-shelf transfer as effective but not optimal; a specialized network could push recall higher on harder scenes.
- The covisibility inlier selection idea, which prunes matches by how many map images observe a point, is not tied to intensity specifically and could generalize to any cross-modal matching task where a texture proxy image is built from a 3D map.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a hierarchical cross-modal visual relocalization system that estimates a camera's 6DoF pose inside a pre-built LiDAR map. The map is first rendered as panoramic intensity images using HEC projection, then coarse candidate retrieval is performed with NetVLAD and a covisibility-clustering reranking step, and fine pose estimation is obtained from SuperPoint+LightGlue two-stage 2D-3D association followed by covisibility inlier selection and PnP with RANSAC. Experiments are conducted on two self-collected SJTU campus sequences, reporting R@1 of 0.73 on the College sequence and relocalization recall RR(1m/3°) of 0.97 on College and 0.79 on Dormitory, along with an ablation on the College sequence.
Significance. If the reported results hold, the paper would show that pretrained visual feature extractors and matchers can be applied to LiDAR intensity images without any fine-tuning, which is practically valuable for cross-modal relocalization. The HEC projection and two-stage equalization are reasonable engineering contributions aimed at reducing the modality gap, and the intensity-versus-depth internal comparison in Table III is a useful control that isolates the role of intensity texture. The main significance is limited by the evaluation being restricted to two private sequences with no public benchmark, no error bars, and no independent ground-truth verification, so the generality of the claims is not yet established.
major comments (4)
- [IV-A] Every quantitative result in Tables I and III depends on the ground-truth camera poses, which Section IV-A describes only as 'obtained by map-based LiDAR localization within the same prior maps.' The paper does not report whether a LiDAR scan was captured together with each query camera image, the extrinsic calibration between the camera and the LiDAR, or the expected accuracy of the LiDAR localization itself. If the quoted poses are LiDAR poses rather than camera poses, then a constant camera-to-LiDAR offset and rotation enter all error computations; with RR thresholds as tight as 0.5 m and 1 degree, even a small unmodeled offset could materially change the reported RR values. The authors should specify the sensor setup, report the extrinsic calibration and its residual error, and validate the ground-truth accuracy, for example by showing reprojection errors against known targets or by providing the LiDAR localization residuals.
- [IV-B] The quantitative evaluation covers only two self-collected sequences on a single campus, and no error bars, query counts, or per-query error distributions are reported. The ablation in Table II is performed on only the College sequence. As a result, the paper's broader claim that intensity-texture consistency enables reliable relocalization is not supported beyond these two environments. The authors should report the number of queries, give confidence intervals or repeated-run statistics, and ideally evaluate on at least one independent or public dataset, or on additional sequences with different point-cloud density and appearance conditions.
- [IV-B, Table III] The fine-relocalization comparison is limited to a depth-channel variant of the authors' own pipeline, which is an internal control rather than a state-of-the-art cross-modal localization baseline. The introduction and conclusion claim superiority over existing methods, but Tables I and III do not establish this for pose estimation because no existing cross-modal relocalization or 2D-3D registration method is compared at the pose level. The authors should either add comparisons to published cross-modal localization/registration baselines or explicitly restrict the claim to 'the intensity channel improves over the depth channel within this hierarchical pipeline.'
- [Table II] The ablation study shows small differences for several components: removing covisibility clustering and removing two-stage equalization both lower RR(1m/3°) from 0.97 to 0.92 on a single sequence. Without error bars, repeated trials, or additional sequences, these differences are not clearly distinguishable from noise. At the same time, the 0.5m/1° RR values show large drops for some ablations (e.g., from 0.25 to 0.06 without two-stage 2D-3D association), which suggests the component has a strong effect only at very tight thresholds. The authors should provide statistical support and discuss which conclusions are robust to the choice of threshold.
minor comments (5)
- [I] The phrase 'Struct-from-Motion' should be 'Structure-from-Motion,' and the sentence 'we propose a hierarchical ... system' would benefit from grammatical cleanup.
- [III-B] 'Habrid Equiangular Cube Projection' appears to be a typo for 'Hybrid Equiangular Cube Projection'; also, the variables in Equations (2) and (3) and the meaning of the adjusting parameter t should be defined precisely.
- [III-C1] The 'sliding window strategy' used to maximize similarity scores for four cube patches is described only in one sentence; the window size, stride, and how the four scores are combined should be stated.
- [III-D1] The criteria for discarding a query-map pair when 'the largest cluster does not contain enough matches' are not specified; the threshold should be reported.
- [III-D2] Covisibility of a local feature point is defined as the number of map images that contain the point, but the paper does not explain how this count is computed for points on continuous surfaces or how the covisibility threshold is chosen.
Circularity Check
No significant circularity: the method uses external pretrained models and standard geometric solvers, and no test-set parameter is fitted and renamed as a prediction.
full rationale
The paper's derivation chain is self-contained with respect to its claimed contribution. The map projection uses the fixed HEC geometric formula from an external reference [23] plus standard histogram equalization and CLAHE, the coarse retrieval uses pretrained NetVLAD, the local matching uses pretrained SuperPoint/LightGlue, and pose estimation is solved by standard PnP with RANSAC (Eq. 4). No parameter in these stages is fitted to the evaluation data and then reported as a prediction; the ablations in Table II remove components and measure the resulting change rather than re-fitting them. The self-citations [2,4] are contextual literature references and are not load-bearing for the central claim. The one legitimate concern is in Sec. IV-A, where ground-truth camera poses are stated to be 'obtained by map-based LiDAR localization within the same prior maps'; the paper does not report the camera-LiDAR extrinsic or the LiDAR localization accuracy, which is a missing-support/verifiability issue for the absolute metrics. However, this is an evaluation-design concern, not a circular derivation: the predicted poses are not equal by construction to the LiDAR-derived ground truth, and the intensity-versus-depth comparison in Table III provides an internal control that does not depend on the absolute ground-truth accuracy. Therefore the paper receives a circularity score of 0.
Assumptions & free parameters
free parameters (6)
- Map trajectory sample interval =
1 m
- Local map filter max distance =
50 m
- Projection image resolution =
1920x480 (four 480x480 faces)
- Top-K retrieval candidates for fine relocalization =
50
- Covisibility clustering and inlier selection thresholds
- RANSAC and PnP thresholds
assumptions (5)
- domain assumption Intensity channel values of LiDAR points, after HEC projection and two-stage equalization, are texture-consistent enough with camera grayscale images for pretrained visual feature extractors to match them.
- domain assumption Pretrained NetVLAD, SuperPoint, and LightGlue models transfer to intensity images without any fine-tuning.
- domain assumption Covisibility, defined as the number of map images containing a local feature point, indicates whether a correspondence is an inlier.
- domain assumption Ground-truth camera poses obtained by map-based LiDAR localization within the same prior map are accurate enough for the 0.5 m and 1 degree evaluation thresholds.
- standard math The Hybrid Equiangular Cube Projection formulas (Eq. 2 and 3) from [23] produce a denser, more uniform projection.
Cite this review
Pith. "Pith review of Cross-Modal Visual Relocalization in Prior LiDAR Maps Utilizing Intensity Textures." pith.science (2026). https://pith.science/paper/4QFJQRET
@misc{pith2026241201299,
author = {Pith},
title = {Pith review of: Cross-Modal Visual Relocalization in Prior LiDAR Maps Utilizing Intensity Textures},
year = {2026},
howpublished = {\url{https://pith.science/paper/4QFJQRET}},
note = {Machine review of arXiv:2412.01299}
}
read the original abstract
Cross-modal localization has drawn increasing attention in recent years, while the visual relocalization in prior LiDAR maps is less studied. Related methods usually suffer from inconsistency between the 2D texture and 3D geometry, neglecting the intensity features in the LiDAR point cloud. In this paper, we propose a cross-modal visual relocalization system in prior LiDAR maps utilizing intensity textures, which consists of three main modules: map projection, coarse retrieval, and fine relocalization. In the map projection module, we construct the database of intensity channel map images leveraging the dense characteristic of panoramic projection. The coarse retrieval module retrieves the top-K most similar map images to the query image from the database, and retains the top-K' results by covisibility clustering. The fine relocalization module applies a two-stage 2D-3D association and a covisibility inlier selection method to obtain robust correspondences for 6DoF pose estimation. The experimental results on our self-collected datasets demonstrate the effectiveness in both place recognition and pose estimation tasks.
Figures
Reference graph
Works this paper leans on
-
[1]
Monocular camera localization in 3d lidar maps,
T. Caselitz, B. Steder, M. Ruhnke, and W. Burgard, “Monocular camera localization in 3d lidar maps,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2016, pp. 1926–1931
work page 2016
-
[2]
Cross- modal monocular localization in prior lidar maps utilizing semantic consistency,
C. Zhang, H. Zhao, C. Wang, X. Tang, and M. Yang, “Cross- modal monocular localization in prior lidar maps utilizing semantic consistency,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023, pp. 4004–4010
work page 2023
-
[3]
I2d-loc: Camera localization via image to lidar depth flow,
K. Chen, H. Yu, W. Yang, L. Yu, S. Scherer, and G.-S. Xia, “I2d-loc: Camera localization via image to lidar depth flow,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 194, pp. 209–221, 2022
work page 2022
-
[4]
Global pose initialization based on gridded gaussian distribution with wasserstein distance,
C. Yang, Z. Zhou, H. Zhuang, C. Wang, and M. Yang, “Global pose initialization based on gridded gaussian distribution with wasserstein distance,” IEEE Transactions on Intelligent Transportation Systems , 2023
work page 2023
-
[5]
Bow3d: Bag of words for real-time loop closing in 3d lidar slam,
Y . Cui, X. Chen, Y . Zhang, J. Dong, Q. Wu, and F. Zhu, “Bow3d: Bag of words for real-time loop closing in 3d lidar slam,” IEEE Robotics and Automation Letters , vol. 8, no. 5, pp. 2828–2835, 2022
work page 2022
-
[6]
Robust image retrieval-based visual localization using kapture,
M. Humenberger, Y . Cabon, N. Guerin, J. Morat, V . Leroy, J. Re- vaud, P. Rerole, N. Pion, C. de Souza, and G. Csurka, “Robust image retrieval-based visual localization using kapture,” arXiv preprint arXiv:2007.13867, 2020
arXiv 2007
-
[7]
From coarse to fine: Robust hierarchical localization at large scale,
P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk, “From coarse to fine: Robust hierarchical localization at large scale,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 12 716–12 725
work page 2019
-
[8]
Lip-loc: Lidar image pretraining for cross-modal localization,
S. Shubodh, M. Omama, H. Zaidi, U. S. Parihar, and M. Krishna, “Lip-loc: Lidar image pretraining for cross-modal localization,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 948–957
2024
Show all 27 references
-
[9]
Global visual localization in lidar-maps through shared 2d- 3d embedding space,
D. Cattaneo, M. Vaghi, S. Fontana, A. L. Ballardini, and D. G. Sorrenti, “Global visual localization in lidar-maps through shared 2d- 3d embedding space,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2020, pp. 4365–4371
2020
-
[10]
2d3d-matr: 2d-3d matching transformer for detection-free registration between im- ages and point clouds,
M. Li, Z. Qin, Z. Gao, R. Yi, C. Zhu, Y . Guo, and K. Xu, “2d3d-matr: 2d-3d matching transformer for detection-free registration between im- ages and point clouds,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 14 128–14 138
2023
-
[11]
2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud,
M. Feng, S. Hu, M. H. Ang, and G. H. Lee, “2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud,” in 2019 International Conference on Robotics and Automation (ICRA) . IEEE, 2019, pp. 4790–4796
2019
-
[12]
Robust place recognition using an imaging lidar,
T. Shan, B. Englot, F. Duarte, C. Ratti, and D. Rus, “Robust place recognition using an imaging lidar,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 5469–5475
2021
-
[13]
Real-time simultaneous localization and mapping with lidar intensity,
W. Du and G. Beltrame, “Real-time simultaneous localization and mapping with lidar intensity,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 4164–4170
2023
-
[14]
General, single-shot, target-less, and automatic lidar-camera extrinsic calibration toolbox,
K. Koide, S. Oishi, M. Yokozuka, and A. Banno, “General, single-shot, target-less, and automatic lidar-camera extrinsic calibration toolbox,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 11 301–11 307
2023
-
[15]
Superpoint: Self- supervised interest point detection and description,
D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self- supervised interest point detection and description,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 224–236
2018
-
[16]
Lightglue: Local feature matching at light speed,
P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “Lightglue: Local feature matching at light speed,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 627– 17 638
2023
-
[17]
Recalling direct 2d- 3d matches for large-scale visual localization,
Z. Song, C. Wang, Y . Liu, and S. Shen, “Recalling direct 2d- 3d matches for large-scale visual localization,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 1191–1197
2021
-
[18]
Hscnet++: Hierarchical scene coordinate classification and regression for visual localization with transformer,
S. Wang, Z. Laskar, I. Melekhov, X. Li, Y . Zhao, G. Tolias, and J. Kannala, “Hscnet++: Hierarchical scene coordinate classification and regression for visual localization with transformer,” International Journal of Computer Vision , pp. 1–21, 2024
2024
-
[19]
Learning to detect scene landmarks for camera localization,
T. Do, O. Miksik, J. DeGol, H. S. Park, and S. N. Sinha, “Learning to detect scene landmarks for camera localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 11 132–11 142
2022
-
[20]
Delight: An efficient descriptor for global localisation using lidar intensities,
K. P. Cop, P. V . Borges, and R. Dub´e, “Delight: An efficient descriptor for global localisation using lidar intensities,” in 2018 IEEE Interna- tional Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 3653–3660
2018
-
[21]
Intensity-slam: Intensity assisted localization and mapping for large scale environment,
H. Wang, C. Wang, and L. Xie, “Intensity-slam: Intensity assisted localization and mapping for large scale environment,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 1715–1721, 2021
2021
-
[22]
A comprehen- sive comparison of projections in omnidirectional super-resolution,
H. Pi, S. Tian, M. Lu, J. Liu, Y . Guo, and S. Zhang, “A comprehen- sive comparison of projections in omnidirectional super-resolution,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5
2023
-
[23]
Efficient projection and coding tools for 360° video,
J.-L. Lin, Y .-H. Lee, C.-H. Shih, S.-Y . Lin, H.-C. Lin, S.-K. Chang, P. Wang, L. Liu, and C.-C. Ju, “Efficient projection and coding tools for 360° video,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems , vol. 9, no. 1, pp. 84–97, 2019
2019
-
[24]
Netvlad: Cnn architecture for weakly supervised place recognition,
R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “Netvlad: Cnn architecture for weakly supervised place recognition,” in Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5297–5307
2016
-
[25]
Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition,
S. Hausler, S. Garg, M. Xu, M. Milford, and T. Fischer, “Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 14 141–14 152
2021
-
[26]
Rethinking visual geo- localization for large-scale applications,
G. Berton, C. Masone, and B. Caputo, “Rethinking visual geo- localization for large-scale applications,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 4878–4888
2022
-
[27]
Eigenplaces: Training viewpoint robust models for visual place recognition,
G. Berton, G. Trivigno, B. Caputo, and C. Masone, “Eigenplaces: Training viewpoint robust models for visual place recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 11 080–11 090. TABLE III: The Fine Relocalization Results of Di...
2023
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.