Ground Plane-Aided Extrinsic Calibration of Inertial and RGB-D Sensors for Uncrewed Aerial Vehicles
Pith reviewed 2026-07-01 05:54 UTC · model grok-4.3
The pith
A targetless method calibrates IMU and RGB-D sensors on UAVs by aligning the floor normal with the gravity vector.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By segmenting the ground plane from RGB-D depth data using deep learning and estimating its normal vector, the method combines this with the gravity direction from the IMU accelerometer in a robust estimator to recover the extrinsic rotation between the two sensors without any planar targets or prior estimates.
What carries the argument
The robust estimation step that aligns the estimated floor normal vector from segmented depth points with the gravity vector measured in the accelerometer frame.
If this is right
- Calibration becomes possible in ordinary environments without carrying checkerboard targets to the field.
- The method supports accurate trajectory estimation for UAVs by providing reliable extrinsic parameters.
- Performance exceeds that of MATLAB calibration toolboxes while matching specialized target-based software.
- Setup time decreases since no initial guesses or specialized equipment are needed.
Where Pith is reading between the lines
- If floor segmentation remains reliable across different surfaces and lighting, the approach could apply to ground robots as well.
- Combining this with visual features might handle cases where the floor is partially obscured during flight.
- Real-time implementations could support on-the-fly recalibration during UAV missions.
Load-bearing premise
Deep-learning-based floor-segmentation reliably extracts ground points from the depth channel of RGB-D images under typical UAV operating conditions.
What would settle it
Collect flight data from a UAV with both this method and Kalibr using a checkerboard, then compare the resulting rotation parameters; a difference exceeding 2 degrees on any axis would show the targetless method fails to match established accuracy.
Figures
read the original abstract
Accurate extrinsic calibration of inertial sensors, such as Inertial Measurement Units (IMUs) and cameras is crucial for trajectory estimation of Uncrewed Aerial Vehicles (UAVs). While numerous calibration methods have been proposed, these techniques often rely on specialized equipment, planar targets, and an initial estimate of the calibration parameters. In this research, we propose a targetless calibration method designed for UAVs equipped with IMUs and RGB-Depth (RGB-D) cameras. Our approach leverages deep-learning-based floor-segmentation to extract ground points from the depth channel of RGB-D images. Subsequently, the normal vector to these points is estimated. The known orientation of the normal to the floor segment and the gravity vector sensed in the accelerometer's frame are utilized in a robust estimation approach to estimate the extrinsic calibration parameters. We illustrate that the developed method outperforms MATLAB's Toolboxes and exhibits similar performance to Kalibr without the use of specialized checkerboard targets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a targetless extrinsic calibration method for IMU and RGB-D sensors on UAVs. It uses deep-learning-based segmentation of the floor in RGB-D depth images to extract ground points and estimate the plane normal vector, which is then combined with the gravity vector measured by the accelerometer in a robust estimation procedure to recover the extrinsic parameters. The central claim is that the resulting calibration outperforms MATLAB toolboxes and achieves performance comparable to Kalibr without requiring checkerboard targets.
Significance. If the performance claims are substantiated with quantitative evidence, the approach would provide a practical targetless calibration pipeline that exploits the ground plane commonly available during UAV operation, reducing reliance on specialized hardware and thereby simplifying deployment in field robotics settings.
major comments (2)
- [Abstract] Abstract: the assertion that the method 'outperforms MATLAB's Toolboxes and exhibits similar performance to Kalibr' is presented without any quantitative metrics, error bars, test conditions, or dataset details, rendering the central performance claim impossible to evaluate.
- [Method] Method (floor-segmentation step): the normal-vector estimate used for calibration is obtained solely from deep-learning segmentation of ground points in the RGB-D depth channel. No quantitative assessment of segmentation accuracy, normal angular error, or robustness under UAV-typical conditions (lighting variation, texture, motion blur) is supplied, yet this step is load-bearing for all reported accuracy results.
Simulated Author's Rebuttal
We thank the referee for the constructive comments. We address each major comment point by point below and will revise the manuscript accordingly where the concerns are valid.
read point-by-point responses
-
Referee: [Abstract] Abstract: the assertion that the method 'outperforms MATLAB's Toolboxes and exhibits similar performance to Kalibr' is presented without any quantitative metrics, error bars, test conditions, or dataset details, rendering the central performance claim impossible to evaluate.
Authors: We agree that the abstract summarizes the performance claims without supporting quantitative details. In the revised manuscript we will expand the abstract to include key metrics (mean rotation/translation errors and standard deviations versus the baselines), error bars, test conditions, and dataset information so that the claims can be directly evaluated. revision: yes
-
Referee: [Method] Method (floor-segmentation step): the normal-vector estimate used for calibration is obtained solely from deep-learning segmentation of ground points in the RGB-D depth channel. No quantitative assessment of segmentation accuracy, normal angular error, or robustness under UAV-typical conditions (lighting variation, texture, motion blur) is supplied, yet this step is load-bearing for all reported accuracy results.
Authors: The manuscript validates the pipeline via end-to-end calibration accuracy, which implicitly depends on the segmentation quality. We acknowledge the absence of standalone segmentation metrics. In revision we will add quantitative results on segmentation accuracy (e.g., IoU) and normal-vector angular error computed on our datasets. Robustness discussion will be expanded using the lighting and texture conditions present in the collected UAV sequences; exhaustive additional tests for motion blur would require new experiments beyond the current scope. revision: partial
Circularity Check
No circularity in derivation; method is self-contained empirical proposal
full rationale
The paper describes a targetless extrinsic calibration pipeline that extracts ground points via deep-learning segmentation, estimates the floor normal, and combines it with the accelerometer gravity vector in a robust estimator. No equations, parameter-fitting steps, or derivations appear in the abstract or description. Performance claims are direct empirical comparisons to MATLAB and Kalibr rather than any 'prediction' that reduces to fitted inputs. No self-citations, uniqueness theorems, or ansatzes are invoked. The central claim therefore rests on external validation data rather than internal construction, satisfying the self-contained criterion.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The floor is a flat horizontal plane whose normal vector aligns with the gravity direction.
Reference graph
Works this paper leans on
-
[1]
Past, present, and future of aerial robotic manipulators,
Ollero, A., Tognon, M., Suarez, A., Lee, D., and Franchi, A., “Past, present, and future of aerial robotic manipulators,”IEEE TransactionsonRobotics, Vol. 38, No. 1, 2021, pp. 626–645
2021
-
[2]
Extending Kalibr: Calibrating the Extrinsics of Multiple IMUs and of Individual Axes,
Rehder, J., Nikolic, J., Schneider, T., Hinzmann, T., and Siegwart, R., “Extending Kalibr: Calibrating the Extrinsics of Multiple IMUs and of Individual Axes,”Proceedings of the 2016 IEEE International Conferenceon Roboticsand Automation(ICRA), IEEE, 2016, pp. 4304–4311. https://doi.org/10.1109/ICRA.2016.7487580
-
[3]
Monocular Visual-Inertial and Robotic-Arm Calibration in a Unifying Framework,
Zhang, Y., Liang, W., Yuan, M., He, H., Tan, J., and Pang, Z., “Monocular Visual-Inertial and Robotic-Arm Calibration in a Unifying Framework,”IEEE/CAAJournal ofAutomaticaSinica, Vol. 9, No. 1, 2021, pp. 146–159
2021
-
[4]
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator,
Qin, T., Li, P., and Shen, S., “VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator,”IEEE TransactionsonRobotics, Vol. 34, No. 4, 2018, pp. 1004–1020
2018
-
[5]
An Online Initialization and Self-Calibration Method for Stereo Visual-Inertial Odometry,
Huang, W., Liu, H., and Wan, W., “An Online Initialization and Self-Calibration Method for Stereo Visual-Inertial Odometry,” IEEE TransactionsonRobotics, Vol. 36, No. 4, 2020, pp. 1153–1170. https://doi.org/10.1109/TRO.2019.2959161
-
[6]
Observability Analysis and Keyframe-Based Filtering for Visual Inertial Odometry with Full Self-Calibration,
Huai, J., Lin, Y., Zhuang, Y., Toth, C. K., and Chen, D., “Observability Analysis and Keyframe-Based Filtering for Visual Inertial Odometry with Full Self-Calibration,”IEEE TransactionsonRobotics, Vol. 38, No. 5, 2022, pp. 3219–3237
2022
-
[7]
AdaptiveOptimizationOnlineIMUSelf-CalibrationMethodforVisual-InertialNavigationSystems,
Wang,Z.,andCheng,X.,“AdaptiveOptimizationOnlineIMUSelf-CalibrationMethodforVisual-InertialNavigationSystems,” Measurement, Vol. 180, 2021
2021
-
[8]
Extrinsic calibration of multiple inertial sensors from arbitrary trajectories,
Lee, J., Hanley, D., and Bretl, T., “Extrinsic calibration of multiple inertial sensors from arbitrary trajectories,”IEEE Robotics andAutomationLetters, Vol. 7, No. 2, 2022, pp. 2055–2062
2022
-
[9]
A dynamic and static combined camera-IMU extrinsic calibration method based on continuous-time trajectory estimation,
Ouyang, W., Lin, W., and Sun, L., “A dynamic and static combined camera-IMU extrinsic calibration method based on continuous-time trajectory estimation,”RoboticsandAutonomousSystems, Vol. 186, 2025, p. 104916
2025
-
[10]
Monocular visual–inertial state estimation with online initialization and camera–IMU extrinsic calibration,
Yang, Z., and Shen, S., “Monocular visual–inertial state estimation with online initialization and camera–IMU extrinsic calibration,”IEEETransactionsonAutomationScienceandEngineering, Vol. 14, No. 1, 2016, pp. 39–51
2016
-
[11]
An efficient solution to the five-point relative pose problem,
Nistér, D., “An efficient solution to the five-point relative pose problem,”IEEE transactions on pattern analysisand machine intelligence, Vol. 26, No. 6, 2004, pp. 756–770
2004
-
[12]
A Slope-Based Targetless Extrinsic Calibration Method for LiDAR-IMU Systems on Ground Vehicles,
Xiao, J., Guo, J., Ouyang, C., Shi, J., Xu, Y., and Wu, M., “A Slope-Based Targetless Extrinsic Calibration Method for LiDAR-IMU Systems on Ground Vehicles,”IEEE TransactionsonInstrumentation andMeasurement, 2025
2025
-
[13]
Monocular Visual Odometry using a Planar Road Model to Solve Scale Ambiguity,
Kitt, B. M., Rehder, J., Chambers, A. D., Schonbein, M., Lategahn, H., and Singh, S., “Monocular Visual Odometry using a Planar Road Model to Solve Scale Ambiguity,”Proceedings of 5th European Conferenceon Mobile Robots(ECMR ’11), 2011, pp. 43 – 48
2011
-
[14]
Ground-plane-basedabsolutescaleestimationformonocularvisualodometry,
Zhou,D.,Dai,Y.,andLi,H.,“Ground-plane-basedabsolutescaleestimationformonocularvisualodometry,” IEEETransactions onIntelligentTransportationSystems, Vol. 21, No. 2, 2019, pp. 791–802
2019
-
[15]
Enhanced low drift LiDAR SLAM with dynamic noise using ground principal plane constraint,
Wei, D., Ran, H., Li, Z., and Gao, T., “Enhanced low drift LiDAR SLAM with dynamic noise using ground principal plane constraint,”Measurement, Vol. 245, 2025, p. 116549
2025
-
[16]
GRIL-Calib: targetless ground robot IMU-LiDAR extrinsic calibration method using ground plane motion constraints,
Kim, T., Pak, G., and Kim, E., “GRIL-Calib: targetless ground robot IMU-LiDAR extrinsic calibration method using ground plane motion constraints,”IEEE RoboticsandAutomationLetters, Vol. 9, No. 6, 2024, pp. 5409–5416. 15
2024
-
[17]
Lidar-InertialLocalization withGroundConstraintinaPointCloudMap,
Ai,M.,Hokmabadi,I.A.S.,Elhabiby,M.,Moussa,M.,Zekry,A.,Mohamed,A.,andEl-Sheimy,N.,“Lidar-InertialLocalization withGroundConstraintinaPointCloudMap,” ISPRSAnnalsofthePhotogrammetry,RemoteSensingandSpatialInformation Sciences, Vol. X, 2023, pp. 613–619
2023
-
[18]
LG-VIWO: Visual-Inertial-Wheel Odometry Leveraging the Depth-aided Local Ground Constraints for Mobile Robots,
Li, W., Wang, G., Zhang, Q., and Liu, J., “LG-VIWO: Visual-Inertial-Wheel Odometry Leveraging the Depth-aided Local Ground Constraints for Mobile Robots,”IEEE TransactionsonInstrumentation andMeasurement, 2025
2025
-
[19]
Pritzl, V., Vrba, M., Tortorici, C., Ashour, R., and Saska, M., “Adaptive Estimation of UAV Altitude in Complex Indoor EnvironmentsUsingDegradedandTime-DelayedMeasurementswithTime-VaryingUncertainties,” RoboticsandAutonomous Systems, Vol. 160, 2023. https://doi.org/10.1016/j.robot.2022.104281
-
[20]
Zhao, L., Wang, W., He, Q., Yan, L., and Li, X., “Visual–Inertial Autonomous UAV Navigation in Complex Illumination and Highly Cluttered Under-Canopy Environments,”Drones, Vol. 9, No. 1, 2025. https://doi.org/10.3390/drones9010001
-
[21]
3D Global Localization of a UAV Using 2D Monte Carlo Localization and Ground Plane Extraction,
Whitney, E., Gopu, B. P. R., and Tiwari, M., “3D Global Localization of a UAV Using 2D Monte Carlo Localization and Ground Plane Extraction,”AIAA SCITECH2025Forum, 2025
2025
-
[22]
Integrated air-ground vehicles for UAV emergencylandingbasedongraphconvolutionnetwork,
Chen, J., Zhang, Y., Li, J., Du, W., Chen, Z., Liu, Z., Wang, H., and Leung, V. C., “Integrated air-ground vehicles for UAV emergencylandingbasedongraphconvolutionnetwork,” IEEEInternetofThingsJournal,Vol.9,No.12,2021,pp.9106–9116
2021
-
[23]
Floor extraction and door detection for visually impaired guidance,
Berenguel-Baeta, B., Guerrero-Viu, M., Nova, A., Bermudez-Cameo, J., Pérez-Yus, A., and Guerrero, J. J., “Floor extraction and door detection for visually impaired guidance,”202016thInternational ConferenceonControl,Automation,Roboticsand Vision(ICARCV), IEEE, 2020, pp. 1222–1229
2020
-
[24]
Accurate and robust scale recovery for monocular visual odometry based on plane geometry,
Tian, R., Zhang, Y., Zhu, D., Liang, S., Coleman, S., and Kerr, D., “Accurate and robust scale recovery for monocular visual odometry based on plane geometry,”2021 IEEE International Conferenceon Roboticsand Automation(ICRA), IEEE, 2021, pp. 5296–5302
2021
-
[25]
A survey on ground segmentation methods for automotive LiDAR sensors,
Gomes, T., Matias, D., Campos, A., Cunha, L., and Roriz, R., “A survey on ground segmentation methods for automotive LiDAR sensors,”Sensors, Vol. 23, No. 2, 2023, p. 601
2023
-
[26]
Flexible ground constrained LiDAR SLAM with a novel plane detection,
Liu, D., Ni, H., Zhou, X., Yang, N., and Yan, W., “Flexible ground constrained LiDAR SLAM with a novel plane detection,” ComputersandElectrical Engineering, Vol. 117, 2024, p. 109287
2024
-
[27]
Distinctive image features from scale-invariant keypoints,
Lowe, D. G., “Distinctive image features from scale-invariant keypoints,”International journal of computer vision, Vol. 60, No. 2, 2004, pp. 91–110
2004
-
[28]
GroundNet: Monoculargroundplanenormalestimationwithgeometricconsistency,
Man,Y.,Weng,X.,Li,X.,andKitani,K.,“GroundNet: Monoculargroundplanenormalestimationwithgeometricconsistency,” Proceedings ofthe 27thACMInternational Conferenceon Multimedia, 2019, pp. 2170–2178
2019
-
[29]
Ground segmentation and free space estimation in off-road terrain,
Hamandi, M., Asmar, D., and Shammas, E., “Ground segmentation and free space estimation in off-road terrain,”Pattern RecognitionLetters, Vol. 108, 2018, pp. 1–7
2018
-
[30]
GndNet: Fast ground plane estimation and point cloud segmentation for autonomous vehicles,
Paigwar, A., Erkent, Ö., Sierra-Gonzalez, D., and Laugier, C., “GndNet: Fast ground plane estimation and point cloud segmentation for autonomous vehicles,”2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), IEEE, 2020, pp. 2150–2156
2020
-
[31]
Use of the Hough transformation to detect lines and curves in pictures,
Duda, R. O., and Hart, P. E., “Use of the Hough transformation to detect lines and curves in pictures,”Communications of the ACM, Vol. 15, No. 1, 1972, pp. 11–15
1972
-
[32]
An integrated fast Hough transform for multidimensional data,
Li, Y., and Gan, X., “An integrated fast Hough transform for multidimensional data,”IEEEtransactionsonpattern analysisand machineintelligence, Vol. 45, No. 9, 2023, pp. 11365–11373
2023
-
[33]
DBSCAN revisited, revisited: why and how you should (still) use DBSCAN,
Schubert, E., Sander, J., Ester, M., Kriegel, H. P., and Xu, X., “DBSCAN revisited, revisited: why and how you should (still) use DBSCAN,”ACMTransactionson Database Systems(TODS), Vol. 42, No. 3, 2017, pp. 1–21
2017
-
[34]
Ransac for Robotic Applications: A Survey,
Martínez-Otzeta, J. M., Rodríguez-Moreno, I., Mendialdua, I., and Sierra, B., “Ransac for Robotic Applications: A Survey,” Sensors, Vol. 23, 2022
2022
-
[35]
An improved RANSAC for 3D point cloud plane segmentation based on normal distribution transformation cells,
Li, L., Yang, F., Zhu, H., Li, D., Li, Y., and Tang, L., “An improved RANSAC for 3D point cloud plane segmentation based on normal distribution transformation cells,”RemoteSensing, Vol. 9, No. 5, 2017, p. 433
2017
-
[36]
Flexible camera calibration by viewing a plane from unknown orientations,
Zhang, Z., “Flexible camera calibration by viewing a plane from unknown orientations,”Proceedings of the seventh ieee international conferenceoncomputer vision, Vol. 1, Ieee, 1999, pp. 666–673. 16
1999
-
[37]
Camera Calibration,
The MathWorks, I., “Camera Calibration,” , 2025. URL https://www.mathworks.com/help/vision/camera-calibration.html, accessed: August 25, 2025
2025
-
[38]
A robust and easy to implement method for IMU calibration without external equipments,
Tedaldi, D., Pretto, A., and Menegatti, E., “A robust and easy to implement method for IMU calibration without external equipments,”2014IEEE international conferenceonroboticsandautomation (ICRA), IEEE, 2014, pp. 3042–3049
2014
-
[39]
Khankalantary, S., Ranjbaran, S., and Ebadollahi, S., “Simplification of Calibration of Low-Cost MEMS Accelerometer and Its Temperature Compensation Without Accurate Laboratory Equipment,”Measurement Science and Technology, Vol. 32, 2021. https://doi.org/10.1088/1361-6501/abd0bf
-
[40]
A Calibration Method of g-sensitivity Error of MEMS Gyroscope,
Wang, Z., Chaoyang, X., and Liheng, C., “A Calibration Method of g-sensitivity Error of MEMS Gyroscope,”International ConferenceonAutonomousUnmannedSystems, Springer Singapore, 2021, pp. 1957–1965
2021
-
[41]
An Algorithm for the In-Field Calibration of a MEMS IMU,
Qureshi, U., and Golnaraghi, F., “An Algorithm for the In-Field Calibration of a MEMS IMU,”IEEE Sensors Journal, Vol. 17,
-
[42]
https://doi.org/10.1109/JSEN.2017.2751572
-
[43]
A least squares estimate of satellite attitude,
Wahba, G., “A least squares estimate of satellite attitude,”SIAM review, Vol. 7, No. 3, 1965, pp. 409–409
1965
-
[44]
Lasso Wahba’s Problem and Its Analytical Solution for Spacecraft Attitude Determination,
Wu, J., Zhou, Z., Fourati, H., and Liu, M., “Lasso Wahba’s Problem and Its Analytical Solution for Spacecraft Attitude Determination,”2021 IEEE 17th International Conferenceon AutomationScience and Engineering (CASE), IEEE, 2021, pp. 1040–1046. https://doi.org/10.1109/CASE49439.2021.9551389
-
[45]
FracGM: A Fast Fractional Programming Technique for Geman-McClure Robust Estimator,
Chen, B., Lin, Y., Chen, J., Huang, C., Chern, J., and Sun, C., “FracGM: A Fast Fractional Programming Technique for Geman-McClure Robust Estimator,”IEEE Robotics and Automation Letters, Vol. 9, No. 12, 2024, pp. 11666–11673. https://doi.org/10.1109/LRA.2024.3495372
-
[46]
Gemini 2 3D Camera (Active Stereo IR Depth Sensor),
Orbbec Technology (Shenzhen) Co., Ltd., “Gemini 2 3D Camera (Active Stereo IR Depth Sensor),” https://www.orbbec.com/ products/stereo-vision-camera/gemini-2/, 2023. Accessed: 2025-08-19
2023
-
[47]
ISM330DHCX 6-DOF IMU Breakout,
Adafruit, “ISM330DHCX 6-DOF IMU Breakout,” https://www.adafruit.com/product/4502, n.d.. Accessed: 2025-08-19
2025
-
[48]
LSM6DSOX 6-DOF IMU Breakout,
Adafruit, “LSM6DSOX 6-DOF IMU Breakout,” https://www.adafruit.com/product/4438, n.d.. Accessed: 2025-08-19
2025
-
[49]
3.4 ed., August 2013
TDK InvenSense,MPU-6050 6-DOF IMU Breakout, TDK InvenSense, rev. 3.4 ed., August 2013. URL https://invensense.tdk. com/wp-content/uploads/2015/02/MPU-6000-Datasheet1.pdf, accessed: 2025-08-19
2013
-
[50]
Rother, C., Kolmogorov, V., and Blake, A., “"GrabCut": interactive foreground extraction using iterated graph cuts,”ACM Trans.Graph., Vol. 23, No. 3, 2004, p. 309–314. https://doi.org/10.1145/1015706.1015720, URL https://doi.org/10.1145/ 1015706.1015720
-
[51]
Estimate Camera-to-IMU Transformation Using Extrinsic Calibration,
MathWorks, “Estimate Camera-to-IMU Transformation Using Extrinsic Calibration,” https://www.mathworks.com/help/nav/ ug/estimate-camera-to-imu-transformation-using-extrinsic-calibration.html, 2025. Accessed: 2025-08-19
2025
-
[52]
A least squares estimate of satellite attitude (Grace Wahba),
Farrell, J., Stuelpnagel, J., Wessner, R., Velman, J., and Brook, J., “A least squares estimate of satellite attitude (Grace Wahba),” SIAM review, Vol. 8, No. 3, 1966, pp. 384–386
1966
-
[53]
Improvement to the Implementation of the QUEST Algorithm,
Cheng, Y., and Shuster, M. D., “Improvement to the Implementation of the QUEST Algorithm,”Journal of Guidance, Control, andDynamics, Vol. 37, No. 1, 2014
2014
-
[54]
Fast Linear Quaternion Attitude Estimator Using Vector Observations,
Wu, J., Zhou, Z., Gao, B., Li, R., Cheng, Y., and Fourati, H., “Fast Linear Quaternion Attitude Estimator Using Vector Observations,”IEEE TransactionsonAutomationScienceandEngineering, Vol. 15, 2017. 17
2017
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.