REVIEW 4 major objections 6 minor 27 references
LIR-LIVO: A Lightweight,Robust LiDAR/Vision/Inertial Odometry with Illumination-Resilient Deep Features
T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper claims that a lightweight LiDAR-inertial-visual odometry system can keep pose estimation accurate in low light by giving deep-learned visual features metric depths directly from LiDAR point clouds.
desk verdict A credible LIVO system with a genuinely new front-end integration and strong Hilti low-light results, but the SOTA claim is overstated and the LiDAR depth-association step is under-validated at depth discontinuities. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery that carries the argument is LiDAR depth association feeding a lightweight sliding-window visual update inside an error-state iterated Kalman filter. A sweep-recombination step first realigns LiDAR scans so their end timestamp matches the camera image timestamp. Each keypoint from the learned detector (SuperPoint) is then located on a unit sphere around the camera, and the five nearest LiDAR points define a candidate local plane; a 0.05 m point-to-plane residual check validates the association, and the feature's depth is solved on that plane. The matched features, established by the learned matcher LightGlue, enter a reprojection residual that treats those depths and all past keyframe poses as constants, so the filter only has to estimate the new keyframe's pose. A separate depth-distribution step bins features across ten depth intervals so both near and far points contribute complementary constraints on translation and rotation.
What would settle it
Take a motion-captured scene containing thin foreground structures in front of a background wall, and compare the depth LIR-LIVO assigns to each image feature against the true geometry. If features whose five nearest LiDAR points straddle the foreground/background edge receive visibly wrong depths and the pose error grows on exactly those frames, the depth-association assumption is falsified; if the 0.05 m residual check rejects those associations and pose error stays low, the assumption holds.
Extended reading notes
Core claim
The paper's central claim is that associating every deep visual feature with an accurate LiDAR-derived depth makes the vision part of a fused odometry system both more accurate and cheaper than the standard triangulation-and-landmark pipeline. Depth comes from projecting LiDAR points onto a unit sphere around the camera, taking the five nearest points to each feature, fitting a plane, accepting the association only if the point-to-plane residual is below 0.05 m, and solving for the feature's depth on that plane. Because the depth is treated as fixed, the reprojection residual in the filter optimizes only the pose of the newest keyframe against a small sliding window of past keyframes, with older poses and depths held constant. The reported results are lower absolute trajectory errors than prior open-source systems on the majority of the evaluated aerial, handheld, and low-light sequences, including successful estimation in sequences where comparison methods diverge or fail.
Load-bearing premise
The load-bearing assumption is that for every visual feature, the five closest LiDAR points in the camera's direction lie on one flat surface, so the depth solved from their fitted plane is the true depth of the image feature.
Editorial extensions
If this is right
- A robot or drone using this system can expect pose estimates to remain stable when ambient light drops, because the visual front end does not depend on optical flow or intensity corners.
- The visual subsystem's memory footprint is smaller than in landmark-based LIVO systems, since no 3D visual map is maintained; only a sliding window of keyframes with feature depths is kept.
- Accuracy no longer depends on having a wide camera baseline for triangulation, so the system can stay accurate during slow or small-motion segments as long as LiDAR depth association succeeds.
- Near and far features are explicitly balanced, so the estimator does not let distant points dominate rotation or nearby points dominate translation in any single frame.
Reading between the lines
- An unstated consequence is that the same depth-association recipe could work with any depth source, such as a stereo camera or RGB-D sensor, preserving the illumination robustness of the learned features without requiring LiDAR.
- The 0.05 m plane residual threshold means the system's reliability in cluttered scenes hinges on the five nearest LiDAR points lying on one surface; scenes with thin structures or moving objects would expose this and are not separately stress-tested in the reported benchmarks.
- The paper's ablation isolates the uniform-depth-distribution component, but not the depth association or the learned front end; a reader who wants to attribute the gains should test variants that replace only one component at a time.
- If the depth-association failures are rare, the system's accuracy should degrade gracefully rather than catastrophically, because the iterated Kalman filter can still use the LiDAR-inertial update as a backup; this could be tested by injecting synthetic wrong depths into the visual update and measuring pose error growth.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents LIR-LIVO, a LiDAR-inertial-visual odometry system that couples a FAST-LIO2-style direct LiDAR-inertial estimator with a visual frontend based on SuperPoint feature extraction and LightGlue matching. Visual feature depths are obtained by associating each feature ray with nearby LiDAR points on the unit sphere, fitting a local plane, and solving for the feature depth; a uniform depth distribution heuristic is used to select features across ten depth intervals. The visual subsystem keeps a small sliding window of keyframes and optimizes only the newest pose by minimizing reprojection errors with previously associated depths held fixed. Experiments on NTU-VIRAL, Hilti'22, and R3LIVE benchmark sequences report lower RMSE than several open-source baselines on most sequences, and timing measurements indicate real-time operation on a desktop GPU.
Significance. If the claims hold, the paper would make a useful contribution by showing that a lightweight LIVO system can benefit from deep visual features and LiDAR-derived depths without maintaining a visual landmark map, and it would provide evidence of robustness in low-light conditions where classical visual frontends fail. The manuscript has concrete strengths: it is validated on external datasets with independent ground truth, it compares against multiple open-source baselines, it includes a timing breakdown, and the code is publicly released. The central claim of state-of-the-art accuracy is, however, currently stated too broadly relative to the paper's own tables, and the depth-association mechanism that underpins the visual measurement model is not validated at depth discontinuities. These issues are addressable in revision but need substantive work.
major comments (4)
- [Section IV.B, Eq. (1)] The plane-fitting formulation as printed is vacuous: with b = [0,0,0,0,0]^T, the least-squares solution is x = (A^T A)^{-1} A^T b = 0, which cannot serve as the plane normal/offset vector. This makes the depth-association equations in Eq. (1) and the subsequent depth computation in Eq. (2) non-reproducible. Please provide the correct plane estimation formulation (e.g., fit n^T p = l with an appropriate non-zero b vector, or use SVD of mean-centered points) and ensure Eq. (2) is consistent with it.
- [Section V.A, Tables II-III and Abstract] The abstract and Section V.B claim state-of-the-art accuracy, but the tables contain multiple counterexamples: in Table II, LIR-LIVO is worse than SR-LIVO on eee03 (0.261 vs. 0.216), nya02 (0.253 vs. 0.190), and sbs01 (0.152 vs. 0.120); in Table III, it is worse than SR-LIVO on hku campus seq 00 (0.029 vs. 0.020) and worse than both R3LIVE and SR-LIVO on LiDAR Degenerate (0.076 vs. 0.064 and 0.053). The claim should be qualified to 'most sequences,' and since many differences are at the centimeter level, statistical significance or at least repeated-run variability should be reported.
- [Section IV.B and IV.E.2] The depth association validates only that the five nearest LiDAR points are approximately coplanar (0.05 m point-to-plane threshold); it does not verify that the feature's true scene point lies on that plane. At depth discontinuities, thin structures, or object boundaries, the five angularly nearest points can all lie on a foreground surface while the feature lies on a background surface, yielding a biased depth that Eq. (6) treats as fixed truth. No robust kernel or chi-square gate for depth outliers is described. I recommend adding a quantitative evaluation of depth-association accuracy (e.g., fraction of associated depths within a tolerance of ground-truth depth) and an outlier-rejection mechanism in the visual update, otherwise the reported low-light gains cannot be attributed to correct depth association rather than the LiDAR-inertial subsystem alone.
- [Section V.A and V.B] The text states that evaluations were conducted on '20 sequences from NTU-VIRAL, Hilti'22, R3LIVE-Dataset, and FAST-LIVO-Dataset,' but Tables II and III report only NTU-VIRAL, Hilti'22, and R3LIVE sequences; no FAST-LIVO-Dataset results appear. In addition, Section V.B reports an eee01 error of 0.139 m while Table II lists 0.164 m for LIR-LIVO on that sequence. These inconsistencies must be reconciled, as they affect the reproducibility of the reported results.
minor comments (6)
- [Section III, Fig. 2 caption] The caption refers to 'SuperPoint and SuperGlue,' but the text and Section IV.C use LightGlue; please align the terminology.
- [Section IV.B, Eq. (3)] The notation is unclear: δj and δk are called inverse depths, but the formula writes δk outside the bracket and δj inside; please clarify the relationship between the inverse-depth parametrization and the camera projection function π_c and its derivative π'_c.
- [Section IV.B] There are several typos, including 'A silding window' and 'pc_i ... in the u'; the latter should likely read 'in the unit sphere.' Please proofread the mathematical and algorithmic descriptions.
- [Section V.C, Table IV] The comparison between SuperPoint/LightGlue and Shi-Tomasi/BFMatcher should state the feature count, image resolution, and whether all components ran on the same hardware with the same sliding-window size; otherwise the timing comparison is difficult to interpret.
- [Section V.D, Table V] The text references 'Table V-C' but the table is labeled 'TABLE V'; also clarify how the 50 uniformly depth-sampled points interact with the ten depth intervals described in Section IV.B.
- [Section IV.E.2] The phrase 'estimatoin strategy' is a typo for 'estimation strategy.' More substantively, the criterion for keyframe selection is described only vaguely as 'variation in its prior pose relative to the K1 keyframe'; please specify the threshold or procedure.
Circularity Check
No circularity: LIR-LIVO is evaluated on external benchmarks, and its depth-association and uniform-depth-distribution steps are design choices rather than predictions fitted to the target result.
full rationale
The paper's central claim is that LIR-LIVO achieves state-of-the-art accuracy and robustness, supported by comparisons on the independent NTU-VIRAL, Hilti'22, and R3LIVE-Dataset benchmarks with external ground truth or trajectory consistency. No parameter of the visual front end or depth-association module is fitted to these benchmark outcomes; SuperPoint and LightGlue are pretrained external models, and the 0.05 m coplanarity threshold in Section IV.B is a stated design parameter borrowed and made stricter from [7], not a fitted constant. The depth-association procedure (Eqs. (1)-(2)) computes a feature depth as the intersection of the feature ray with a local plane fit to nearby LiDAR points; this is a geometric measurement model, and the visual reprojection residual (Eq. (6)) uses that depth as a fixed constant. While this model may be inaccurate at depth discontinuities, that is a correctness/robustness concern, not a circularity: the measurement is not defined in terms of the pose output being predicted. The uniform depth-distribution strategy is motivated by the analytical Jacobians in Eq. (3) and is ablated on NTU-VIRAL, but the ablation is a mild self-evaluation, not a case of fitting a parameter to the claimed benchmark result. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no renaming of a known result as a new derivation. The derivation chain is therefore self-contained and externally falsifiable; no circular step reduces the conclusions to the inputs.
Assumptions & free parameters
free parameters (5)
- Depth association plane residual threshold =
0.05 m
- Sliding window size =
5 keyframes
- Depth distribution intervals =
10 intervals over 1 to 200 m
- Parallax threshold for keyframe inclusion =
15 pixels
- Uniform depth sampling count =
50 points per frame
assumptions (4)
- domain assumption SuperPoint and LightGlue pretrained models transfer to the benchmark camera and illumination conditions.
- standard math The scene is locally planar for both LiDAR point-to-plane residuals and depth association.
- domain assumption Camera-LiDAR-IMU extrinsics and time offsets are pre-calibrated and fixed.
- standard math IMU state propagation follows the standard error-state iterated Kalman filter model with biases as random walks.
Cite this review
Pith. "Pith review of LIR-LIVO: A Lightweight,Robust LiDAR/Vision/Inertial Odometry with Illumination-Resilient Deep Features." pith.science (2026). https://pith.science/paper/7PVVEX55
@misc{pith2026250208676,
author = {Pith},
title = {Pith review of: LIR-LIVO: A Lightweight,Robust LiDAR/Vision/Inertial Odometry with Illumination-Resilient Deep Features},
year = {2026},
howpublished = {\url{https://pith.science/paper/7PVVEX55}},
note = {Machine review of arXiv:2502.08676}
}
read the original abstract
In this paper, we propose LIR-LIVO, a lightweight and robust LiDAR-inertial-visual odometry system designed for challenging illumination and degraded environments. The proposed method leverages deep learning-based illumination-resilient features and LiDAR-Inertial-Visual Odometry (LIVO). By incorporating advanced techniques such as uniform depth distribution of features enabled by depth association with LiDAR point clouds and adaptive feature matching utilizing Superpoint and LightGlue, LIR-LIVO achieves state-of-the-art (SOTA) accuracy and robustness with low computational cost. Experiments are conducted on benchmark datasets, including NTU-VIRAL, Hilti'22, and R3LIVE-Dataset. The corresponding results demonstrate that our proposed method outperforms other SOTA methods on both standard and challenging datasets. Particularly, the proposed method demonstrates robust pose estimation under poor ambient lighting conditions in the Hilti'22 dataset. The code of this work is publicly accessible on GitHub to facilitate advancements in the robotics community.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Wisth, David, et al. ”Unified multi-modal landmark tracking for tightly coupled LiDAR-inertial-visual odometry.” IEEE Robotics and Automa- tion Letters 6.2 (2021): 1004-1011
work page 2021
-
[2]
”LOAM: LiDAR odometry and mapping in real-time.” Robotics: Science and systems
Zhang, Ji, and Sanjiv Singh. ”LOAM: LiDAR odometry and mapping in real-time.” Robotics: Science and systems. V ol. 2. No. 9. 2014
work page 2014
-
[3]
Xu, Wei, et al. ”Fast-lio2: Fast direct LiDAR-inertial odometry.” IEEE Transactions on Robotics 38.4 (2022): 2053-2073
work page 2022
-
[4]
Shan, Tixiao, et al. ”Lio-sam: Tightly-coupled LiDAR inertial odometry via smoothing and mapping.” 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2020
work page 2020
-
[5]
Qin, Tong, Peiliang Li, and Shaojie Shen. ”Vins-mono: A robust and versatile monocular visual-inertial state estimator.” IEEE transactions on robotics 34.4 (2018): 1004-1020
work page 2018
-
[6]
Lin, Jiarong, and Fu Zhang. ”R3LIVE: A Robust, Real-time, RGB- colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and map- ping package.” 2022 International Conference on Robotics and Automa- tion (ICRA). IEEE, 2022
work page 2022
-
[7]
Tang, Hailiang, et al. ”LE-VINS: A robust solid-state-LiDAR-enhanced visual-inertial navigation system for low-speed robots.” IEEE Transac- tions on Instrumentation and Measurement 72 (2023): 1-13
work page 2023
-
[8]
Yuan, Zikang, et al. ”SR-LIVO: LiDAR-Inertial-Visual Odometry and Mapping With Sweep Reconstruction.” IEEE Robotics and Automation Letters (2024)
work page 2024
Show all 27 references
-
[9]
”Camvox: A low-cost and accurate lidar-assisted visual slam system.” 2021 IEEE International Conference on Robotics and Automation (ICRA)
Zhu, Yuewen, et al. ”Camvox: A low-cost and accurate lidar-assisted visual slam system.” 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021
2021
-
[10]
”Lvi-sam: Tightly-coupled LiDAR-visual-inertial odometry via smoothing and mapping.” 2021 IEEE international con- ference on robotics and automation (ICRA)
Shan, Tixiao, et al. ”Lvi-sam: Tightly-coupled LiDAR-visual-inertial odometry via smoothing and mapping.” 2021 IEEE international con- ference on robotics and automation (ICRA). IEEE, 2021
2021
-
[11]
”Fast-livo2: Fast, direct LiDAR-inertial-visual odometry.” IEEE Transactions on Robotics (2024)
Zheng, Chunran, et al. ”Fast-livo2: Fast, direct LiDAR-inertial-visual odometry.” IEEE Transactions on Robotics (2024)
2024
-
[12]
”Fast-livo: Fast and tightly-coupled sparse- direct LiDAR-inertial-visual odometry.” 2022 IEEE/RSJ international conference on intelligent robots and systems (IROS)
Zheng, Chunran, et al. ”Fast-livo: Fast and tightly-coupled sparse- direct LiDAR-inertial-visual odometry.” 2022 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2022
2022
-
[13]
”Lego-loam: Lightweight and ground-optimized LiDAR odometry and mapping on variable terrain.” 2018 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS)
Shan, Tixiao, and Brendan Englot. ”Lego-loam: Lightweight and ground-optimized LiDAR odometry and mapping on variable terrain.” 2018 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS). IEEE, 2018
2018
-
[14]
”Stereo visual inertial LiDAR simultaneous localization and mapping.” 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS)
Shao, Weizhao, et al. ”Stereo visual inertial LiDAR simultaneous localization and mapping.” 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2019
2019
-
[15]
”Lic-fusion 2.0: LiDAR-inertial-camera odometry with sliding-window plane-feature tracking.” 2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS)
Zuo, Xingxing, et al. ”Lic-fusion 2.0: LiDAR-inertial-camera odometry with sliding-window plane-feature tracking.” 2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). IEEE, 2020
2020
-
[16]
”Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras.” Advances in neural information processing systems 34 (2021): 16558-16569
Teed, Zachary, and Jia Deng. ”Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras.” Advances in neural information processing systems 34 (2021): 16558-16569
2021
-
[17]
”SupSLAM: A robust visual inertial SLAM system using SuperPoint for unmanned aerial vehicles.” 2021 8th NAFOSTED Conference on Information and Computer Science (NICS)
Quach, Cong Hoang, et al. ”SupSLAM: A robust visual inertial SLAM system using SuperPoint for unmanned aerial vehicles.” 2021 8th NAFOSTED Conference on Information and Computer Science (NICS). IEEE, 2021
2021
-
[18]
”Super- point: Self-supervised interest point detection and description.” Proceed- ings of the IEEE conference on computer vision and pattern recognition workshops
DeTone, Daniel, Tomasz Malisiewicz, and Andrew Rabinovich. ”Super- point: Self-supervised interest point detection and description.” Proceed- ings of the IEEE conference on computer vision and pattern recognition workshops. 2018
2018
-
[19]
”Airslam: An efficient and illumination-robust point-line visual slam system.” arXiv preprint arXiv:2408.03520 (2024)
Xu, Kuan, et al. ”Airslam: An efficient and illumination-robust point-line visual slam system.” arXiv preprint arXiv:2408.03520 (2024)
2024 arXiv
-
[20]
”Superglue: Learning feature matching with graph neural networks.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Sarlin, Paul-Edouard, et al. ”Superglue: Learning feature matching with graph neural networks.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020
2020
-
[21]
”Light- glue: Local feature matching at light speed.” Proceedings of the IEEE/CVF International Conference on Computer Vision
Lindenberger, Philipp, Paul-Edouard Sarlin, and Marc Pollefeys. ”Light- glue: Local feature matching at light speed.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023
2023
-
[22]
”XFeat: Accelerated Features for Lightweight Image Matching.” Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition
Potje, Guilherme, et al. ”XFeat: Accelerated Features for Lightweight Image Matching.” Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. 2024
2024
-
[23]
”A real-time method for depth enhanced visual odometry.” Autonomous Robots 41 (2017): 31-43
Zhang, Ji, Michael Kaess, and Sanjiv Singh. ”A real-time method for depth enhanced visual odometry.” Autonomous Robots 41 (2017): 31-43
2017
-
[24]
”Laser–visual–inertial odometry and map- ping with high robustness and low drift.” Journal of field robotics 35.8 (2018): 1242-1264
Zhang, Ji, and Sanjiv Singh. ”Laser–visual–inertial odometry and map- ping with high robustness and low drift.” Journal of field robotics 35.8 (2018): 1242-1264
2018
-
[25]
”R2LIVE: A Robust, Real-Time, LiDAR-Inertial- Visual Tightly-Coupled State Estimator and Mapping.” IEEE Robotics and Automation Letters 6.4 (2021): 7469-7476
Lin, Jiarong, et al. ”R2LIVE: A Robust, Real-Time, LiDAR-Inertial- Visual Tightly-Coupled State Estimator and Mapping.” IEEE Robotics and Automation Letters 6.4 (2021): 7469-7476
2021
-
[26]
”Hilti-oxford dataset: A millimeter-accurate benchmark for simultaneous localization and mapping.” IEEE Robotics and Automation Letters 8.1 (2022): 408-415
Zhang, Lintong, et al. ”Hilti-oxford dataset: A millimeter-accurate benchmark for simultaneous localization and mapping.” IEEE Robotics and Automation Letters 8.1 (2022): 408-415
2022
-
[27]
”Ntu viral: A visual-inertial-ranging-LiDAR dataset, from an aerial vehicle viewpoint.” The International Journal of Robotics Research 41.3 (2022): 270-280
Nguyen, Thien-Minh, et al. ”Ntu viral: A visual-inertial-ranging-LiDAR dataset, from an aerial vehicle viewpoint.” The International Journal of Robotics Research 41.3 (2022): 270-280
2022
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.