REVIEW 4 major objections 5 minor 25 references
L2Calib: $SE(3)$-Manifold Reinforcement Learning for Robust Extrinsic Calibration with Degenerate Motion Resilience
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper claims that sensor extrinsics can be learned by reinforcement learning on the SE(3) manifold, with a Bingham rotation distribution, from routine weakly-excited data and no initial guess.
desk verdict RL for extrinsic calibration is a fresh idea, but the paper's central claim is unsupported because the strongest optimization baseline is absent from the exact experiments where the claim is made. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Bingham distribution on the unit-quaternion sphere $S^3$, density $p(x; M, Z) = \exp(x^\top M Z M^\top x)/N(Z)$, dispersion $Z = \mathrm{diag}(z_1, z_2, z_3, 0)$ with $z_1 \le z_2 \le z_3 \le 0$, and antipodal symmetry $p(x) = p(-x)$ — exactly the double-cover symmetry of quaternion rotations. It is the actor's rotation policy, sampled by acceptance-rejection from an Angular Central Gaussian proposal coupled to the Bingham parameters by $\Lambda = I + 2 M Z M^\top_b$ with $\sum_i 1/(b + 2 z_i) = 1$. Three mechanisms support it: a single-step MDP (initial and terminal states only), the trajectory-alignment reward $R = \exp(-(\sum_i e_t(i)/(4N) + \sum_i e_r(i)/(\
What would settle it
Take a well-excited sequence for which ground-truth extrinsics are known, and bias the reference trajectory by a constant SE(3) offset equal to the true extrinsic error. Run L2Calib unchanged: if it converges to the biased transform instead of rejecting it, the reward cannot separate extrinsic error from reference error, and the no-accurate-reference claim collapses. A complementary test: on a degenerate single-axis segment (constant angular velocity and constant acceleration), sweep the extrinsic along the degenerate axis and verify the reward is flat; if the data-selection gate admits such a
Extended reading notes
Core claim
The paper claims that LiDAR–IMU extrinsic calibration can be solved by reinforcement learning, without structured targets, fully-excited motion, or a good initial guess. The problem is recast as a Markov decision process whose action space is the $SE(3)$ manifold: the actor samples a candidate extrinsic (Bingham on $S^3$ for rotation, Gaussian in $\mathbb{R}^3$ for translation), the candidate is used to run a tightly-coupled odometry (FAST-LIO2) over the data, and the reward is the negative absolute pose error between the estimated trajectory and a reference trajectory (LiDAR odometry, RTK, or mocap). The paper argues that degeneracy awareness emerges from the learning dynamics themselves, a
Load-bearing premise
The reward is the trajectory-alignment error, assumed to be a faithful proxy for extrinsic error; this presumes the reference trajectory is accurate (the paper's LO-versus-GT ablation in Section IV-D shows GT references calibrate better) and that the odometry actually senses the extrinsic — a link the paper admits breaks on low-motion segments, where badly wrong extrinsics can still produce a falsely high trajectory-alignment reward (Section IV-D2).
Editorial extensions
If this is right
- Calibration becomes possible from routine operational data: no structured targets, no fully-excited motion sequences, and — per the paper's explicit claim — no high-quality initial extrinsic, since the agent is initialized randomly.
- The same framework should carry over to any odometry-capable sensor pair: the paper states it is applicable to visual-inertial and LiDAR-camera-IMU configurations, though only LiDAR-IMU is validated in the experiments.
- Weak-excitation robustness plus joint multi-sequence training implies recalibration can be run in the background on fleets to track drift from vibration, impacts, or thermal shifts, without dedicated calibration sessions.
- Because uninformative data is filtered rather than force-fit, the method avoids the classic symptom of degenerate calibration — large errors on axes the motion never excited — but only where the selection gate's thresholds correctly identify those axes.
- Per-sequence accuracy still varies with excitation (rotation errors above 1 degree appear on several single UAV sequences), and the reported gains concentrate in joint multi-sequence calibration, so the practical claim is strongest when multiple routine runs are pooled.
Reading between the lines
- The reported 'degenerate motion resilience' may owe more to the data-selection gate than to the RL search: by construction the agent only ever sees segments the gate deems informative, so the robustness claim is conditional on the gate's eigenvalue thresholds — the paper does not ablate how much of the gain comes from selection versus the Bingham policy.
- The single-step MDP framing suggests a natural extension the paper leaves implicit: replace the external reference trajectory with a self-supervised prior (e.g., a second odometry source or a learned trajectory model), which would remove the RTK/mocap/odometry dependency that currently limits deployment to platforms with an accurate reference.
- If multi-sequence pooling is what rescues per-axis degeneracy, one can test that hypothesis directly: build sequences that are each degenerate along a different axis and check that joint training recovers the full 6-DoF extrinsic — a cheap experiment that would isolate the paper's scalability claim.
- The reward's exponential form, with translation and rotation normalized by $4N$ and $\pi^2 N$, penalizes the two error terms asymmetrically; a rotationally biased reference could therefore be silently traded off against translation — a failure mode worth probing since the paper reports rotation errors that stay above 1 degree on several single UAV sequences.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes L2Calib, a reinforcement-learning framework for targetless LiDAR-IMU extrinsic calibration. Calibration is formulated as a single-step MDP in which an actor-critic agent samples candidate SE(3) extrinsics from a Bingham distribution for rotation and a Gaussian for translation; the reward is the trajectory alignment error between a tightly coupled odometry estimate (Fast-LIO2) and a reference trajectory (Traj-LO, RTK, or motion capture). A data-selection module filters segments with weak excitation by thresholding the minimum eigenvalues of IMU Jacobian matrices. Experiments are reported on a handheld dataset, nine NTU VIRAL sequences, and three MCD UGV sequences, with comparisons to LI-Init and, on the handheld data only, iKalibr. The paper also reports ablations of rotation parameterization, data selection, and reference-trajectory source. The stated central claim is that L2Calib outperforms traditional optimization-based methods and achieves high-precision calibration under weak excitation without requiring accurate initial extrinsics.
Significance. If the claims are substantiated, the paper would make a useful contribution: a targetless, initial-guess-free calibration method that can exploit routine operating data is practically valuable, and the SE(3)-manifold RL formulation with a Bingham distribution is a reasonable and well-motivated design. The authors are also to be credited for open-sourcing the code, for reporting the failure on sbs_01 (rather than hiding it), and for including ablations of the reference-trajectory source. However, the evidence as presented does not yet establish the headline claim of superiority over optimization-based methods under weak excitation, because the strongest optimization baseline is absent from the weak-excitation experiments and the RL results lack repeated-run statistics.
major comments (4)
- [§4-B, Tables III and IV] The abstract and conclusion claim that L2Calib 'outperforms traditional optimization-based approaches' under weak excitation. In the UAV and UGV experiments, however, the only comparison method is LI-Init, which is an initialization approach, not a batch optimization-based calibration method such as iKalibr. iKalibr appears only in the fully excited handheld experiments (Table II), where L2Calib is not consistently better (e.g., csc_01: L2Calib rotation error 1.003° vs iKalibr 0.701°). The absence of iKalibr from Tables III and IV is not explained. This is a load-bearing gap: the central claim of outperforming optimization-based methods in the weak-excitation regime is not supported by the provided comparison. Please either run iKalibr (or another continuous-time batch optimization baseline) on the NTU VIRAL and MCD sequences, or substantially temper the claim.
- [§4-A, Tables II–IV] No repeated runs or error bars are reported. PPO is stochastic, and the paper reports a single run per method/sequence. Consequently, statements such as 'L2Calib achieves comparable calibration accuracy' and the ablation comparisons (L2Calib vs L2Calib-g) cannot be assessed for statistical significance. The differences in Tables II–IV are often small (e.g., csc_03 rotation: 0.514° vs 0.734°), and without variance across seeds one cannot tell whether the reported improvements are reproducible or noise. Please report mean and standard deviation over at least 5–10 random seeds, including random extrinsic initialization and PPO seed.
- [§3-D and Eq. (5)] The method is not fully reproducible from the paper. In Eq. (5), the symbol 's' is used in the reward exponent but is never defined (reward normalization constant). In §3-D, the data-selection criterion is described only as 'computing the minimum eigenvalues of J_r^T J_r and J_t^T J_t', but no threshold, window size, or selection rule is given. These parameters are essential to the claimed efficiency and robustness. Please specify them explicitly in the text, not only in code.
- [§4-B, Table III, sbs_01] The paper states that sbs_01 'fails in all methods due to overly simple motion' and the table shows divergent rotation estimates for L2Calib (rotation errors marked ×). This is a direct counterexample to the title's 'Degenerate Motion Resilience' and to the broad claim of robust calibration under weak excitation. The authors acknowledge the failure, which is good, but the manuscript should discuss the boundary of the method's applicability: what excitation level is needed for reliable calibration, and can the data-selection module detect insufficient excitation before calibration? As written, the central claim is overstated relative to the reported results.
minor comments (5)
- [§3-A, Eq. (1)] The notation is inconsistent: the optimization variable is denoted X but also A T_B; the expression T_ref(t)^{-1} · T_B(t) · A T_B^{-1} mixes symbols. Please unify notation and check the matrix multiplication order.
- [§3-C, Eq. (5)] The subscripts in the error terms are confusing (e.g., e_R^B(i) B^A R_A^R ...). Please rewrite Eq. (5) with clear frame indices. Also, 's' is undefined (see major comment).
- [§3-D, Eq. (6)] The text refers to ω_Sk and then writes ω_Ik; the frame subscript is inconsistent. Also, 'Jacobi matrices' should be 'Jacobian matrices'.
- [§4-C, Table IV] The table reports only L2Calib, not L2Calib-g or LI-Init's full details (e.g., LI-Init rotation errors). Adding L2Calib-g on UGV would strengthen the ablation, since the Gaussian-vs-Bingham comparison is otherwise restricted to handheld and UAV data.
- [§4-D, Table VI] The reference-trajectory ablation is only on three sequences. The conclusion that 'both trajectory sources yield practically equivalent odometry reliability' is reasonable but limited; a brief discussion of reference-trajectory error and its effect on the reward signal would be useful.
Circularity Check
No circularity: the reward is the stated calibration objective evaluated against an external reference; Bingham and data-selection components are external prior work.
full rationale
The paper's core optimization loop is not circular. Eq. (1) defines the extrinsic as the minimizer of a trajectory-alignment error against a reference trajectory, and Eq. (5) is a monotone exponential re-weighting of that same error. The RL agent maximizes Eq. (5), so it directly optimizes the stated objective; there is no separate quantity being 'predicted' from a fitted input. The reference trajectory is an external input: 'reference trajectories, essential for reward computation, can be obtained from diverse sources including exteroceptive sensor-based odometry (e.g., Visual Odometry or LiDAR Odometry), Real-Time Kinematic (RTK) positioning systems, or motion capture systems.' In the experiments, Traj-LO is a LiDAR-only odometry that does not depend on the target extrinsic. The Bingham rotation parameterization is adopted from external work ('Inspired by [15], we model the probability distribution of the rotational component directly on the S3 manifold using the Bingham distribution [16]'), and the acceptance-rejection sampler is from Kent et al. [18]; these are not author-unique ansatze. The data-selection Jacobian criterion is attributed to Yang et al. [20]. The only self-citations ([1], [6] by Lv et al.) appear in the related-work survey to describe prior LiDAR-IMU calibration methods and their limitations; they are not used as a load-bearing justification for L2Calib's design or results. No uniqueness theorem is invoked, and no equation reduces by construction to a fitted parameter. The missing iKalibr baseline in weak-excitation tables is a benchmark-completeness concern, not a circularity. The sbs_01 divergence and GT-vs-LO reference ablation are reported honestly and do not indicate that the method's output is equivalent to its input.
Assumptions & free parameters
free parameters (3)
- Data selection threshold
- Reward normalization constants =
4 and pi^2
- PPO hyperparameters
assumptions (5)
- domain assumption The reference trajectory T_ref(t) is accurate enough that its alignment error reflects extrinsic error.
- domain assumption The tightly-coupled odometry's trajectory error is a reliable function of the extrinsic.
- domain assumption The data selection criteria (non-constant angular velocity and linear acceleration) ensure sufficient observability.
- domain assumption PPO can optimize the SE(3) policy to a good solution.
- standard math The Bingham distribution is a proper probability model for quaternions on S3 with antipodal symmetry.
Cite this review
Pith. "Pith review of L2Calib: $SE(3)$-Manifold Reinforcement Learning for Robust Extrinsic Calibration with Degenerate Motion Resilience." pith.science (2026). https://pith.science/paper/VYXKTK47
@misc{pith2026250806330,
author = {Pith},
title = {Pith review of: L2Calib: $SE(3)$-Manifold Reinforcement Learning for Robust Extrinsic Calibration with Degenerate Motion Resilience},
year = {2026},
howpublished = {\url{https://pith.science/paper/VYXKTK47}},
note = {Machine review of arXiv:2508.06330}
}
abstract
Extrinsic calibration is essential for multi-sensor fusion, existing methods rely on structured targets or fully-excited data, limiting real-world applicability. Online calibration further suffers from weak excitation, leading to unreliable estimates. To address these limitations, we propose a reinforcement learning (RL)-based extrinsic calibration framework that formulates extrinsic calibration as a decision-making problem, directly optimizes $SE(3)$ extrinsics to enhance odometry accuracy. Our approach leverages a probabilistic Bingham distribution to model 3D rotations, ensuring stable optimization while inherently retaining quaternion symmetry. A trajectory alignment reward mechanism enables robust calibration without structured targets by quantitatively evaluating estimated tightly-coupled trajectory against a reference trajectory. Additionally, an automated data selection module filters uninformative samples, significantly improving efficiency and scalability for large-scale datasets. Extensive experiments on UAVs, UGVs, and handheld platforms demonstrate that our method outperforms traditional optimization-based approaches, achieving high-precision calibration even under weak excitation conditions. Our framework simplifies deployment on diverse robotic platforms by eliminating the need for high-quality initial extrinsics and enabling calibration from routine operating data. The code is available at https://github.com/APRIL-ZJU/learn-to-calibrate.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Observability- aware intrinsic and extrinsic calibration of LiDAR-IMU systems
J. Lv, X. Zuo, K. Hu, J. Xu, G. Huang, and Y . Liu. “Observability- aware intrinsic and extrinsic calibration of LiDAR-IMU systems”. In: IEEE Transactions on Robotics 38.6 (2022), pp. 3734–3753
work page 2022
-
[2]
Automatic extrinsic calibration of a camera and a 3d lidar using line and plane correspondences
L. Zhou, Z. Li, and M. Kaess. “Automatic extrinsic calibration of a camera and a 3d lidar using line and plane correspondences”. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE. 2018, pp. 5562–5569
work page 2018
-
[3]
A general approach to spatiotemporal calibration in multisensor systems
J. Rehder, R. Siegwart, and P. Furgale. “A general approach to spatiotemporal calibration in multisensor systems”. In: IEEE Transactions on Robotics 32.2 (2016), pp. 383–398
work page 2016
-
[4]
Continuous-time batch estimation using temporal basis functions
P. Furgale, T. D. Barfoot, and G. Sibley. “Continuous-time batch estimation using temporal basis functions”. In: 2012 IEEE International Conference on Robotics and Automation . IEEE. 2012, pp. 2088–2095
work page 2012
-
[5]
LiDAR- camera calibration under arbitrary configurations: Observability and meth- ods
B. Fu, Y . Wang, X. Ding, Y . Jiao, L. Tang, and R. Xiong. “LiDAR- camera calibration under arbitrary configurations: Observability and meth- ods”. In: IEEE Transactions on Instrumentation and Measurement 69.6 (2019), pp. 3089–3102
work page 2019
-
[6]
Targetless calibration of lidar- imu system based on continuous-time batch estimation
J. Lv, J. Xu, K. Hu, Y . Liu, and X. Zuo. “Targetless calibration of lidar- imu system based on continuous-time batch estimation”. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE. 2020, pp. 9968–9975
work page 2020
-
[7]
Reinforcement learning meets visual odometry
N. Messikommer, G. Cioffi, M. Gehrig, and D. Scaramuzza. “Reinforcement learning meets visual odometry”. In: European Conference on Computer Vision. Springer. 2024, pp. 76–92
work page 2024
-
[8]
C. Le Gentil, T. Vidal-Calleja, and S. Huang. “3d lidar-imu calibration based on upsampled preintegrated measurements for motion distortion correction”. In: 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE. 2018, pp. 2149–2155
work page 2018
Show all 25 references
-
[9]
Unified temporal and spatial calibra- tion for multi-sensor systems
P. Furgale, J. Rehder, and R. Siegwart. “Unified temporal and spatial calibra- tion for multi-sensor systems”. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE. 2013, pp. 1280–1286
2013
-
[10]
Spatio-temporal laser to visual/inertial calibration with applications to hand-held, large scale scanning
J. Rehder, P. Beardsley, R. Siegwart, and P. Furgale. “Spatio-temporal laser to visual/inertial calibration with applications to hand-held, large scale scanning”. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE. 2014, pp. 459–465
2014
-
[11]
Accurate and Consistent Spatiotemporal Calibration for Heterogenous-Camera/ IMU/LiDAR System Based on Continuous-Time Batch Estimation
X. Li, S. Chen, S. Li, Y . Zhou, and S. Wang. “Accurate and Consistent Spatiotemporal Calibration for Heterogenous-Camera/ IMU/LiDAR System Based on Continuous-Time Batch Estimation”. In: IEEE/ASME Transactions on Mechatronics 29.3 (2024), pp. 2009–2020
2024
-
[12]
iKalibr: Unified Targetless Spatiotemporal Calibration for Resilient Integrated Inertial Systems
S. Chen, X. Li, S. Li, Y . Zhou, and X. Yang. “iKalibr: Unified Targetless Spatiotemporal Calibration for Resilient Integrated Inertial Systems”. In: IEEE Transactions on Robotics (2025), pp. 1–20
2025
-
[13]
Robust real-time lidar-inertial initialization
F. Zhu, Y . Ren, and F. Zhang. “Robust real-time lidar-inertial initialization”. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE. 2022, pp. 3948–3955
2022
-
[14]
GRIL-Calib: Targetless Ground Robot IMU- LiDAR Extrinsic Calibration Method Using Ground Plane Motion Con- straints
T. Kim, G. Pak, and E. Kim. “GRIL-Calib: Targetless Ground Robot IMU- LiDAR Extrinsic Calibration Method Using Ground Plane Motion Con- straints”. In: IEEE Robotics and Automation Letters 9.6 (2024), pp. 5409– 5416
2024
-
[15]
Bingham policy parameterization for 3d rotations in reinforcement learning
S. James and P. Abbeel. “Bingham policy parameterization for 3d rotations in reinforcement learning”. In: arXiv preprint arXiv:2202.03957 (2022)
2022 arXiv
-
[16]
An Antipodally Symmetric Distribution on the Sphere
C. Bingham. “An Antipodally Symmetric Distribution on the Sphere”. In: The Annals of Statistics 2.6 (1974), pp. 1201 –1225
1974
-
[17]
The quaternion Bingham distribution, 3D object detection, and dynamic manipulation
J. M. Glover. “The quaternion Bingham distribution, 3D object detection, and dynamic manipulation”. PhD thesis. Massachusetts Institute of Technology, 2014
2014
-
[18]
A new method to simulate the Bingham and related distributions in directional data analysis with applications
J. T. Kent, A. M. Ganeiber, and K. V . Mardia. “A new method to simulate the Bingham and related distributions in directional data analysis with applications”. In: arXiv preprint arXiv:1310.8110 (2013)
2013 arXiv
-
[19]
Least-squares estimation of transformation parameters be- tween two point patterns
S. Umeyama. “Least-squares estimation of transformation parameters be- tween two point patterns”. In: IEEE Transactions on Pattern Analysis & Machine Intelligence 13.04 (1991), pp. 376–380
1991
-
[20]
Online imu intrinsic calibration: Is it necessary?
Y . Yang, P. Geneva, X. Zuo, and G. Huang. “Online imu intrinsic calibration: Is it necessary?” In: 2020 Robotics: Science and Systems (2020)
2020
-
[21]
NTU VIRAL: A Visual-Inertial-Ranging-Lidar Dataset, From an Aerial Vehicle Viewpoint
T.-M. Nguyen, S. Yuan, M. Cao, Y . Lyu, T. H. Nguyen, and L. Xie. “NTU VIRAL: A Visual-Inertial-Ranging-Lidar Dataset, From an Aerial Vehicle Viewpoint”. In: The International Journal of Robotics Research 41.3 (2022), pp. 270–280
2022
-
[22]
MCD: Diverse Large- Scale Multi-Campus Dataset for Robot Perception
T.-M. Nguyen, S. Yuan, T. H. Nguyen, P. Yin, H. Cao, L. Xie, M. Wozniak, P. Jensfelt, M. Thiel, J. Ziegenbein, and N. Blunder. “MCD: Diverse Large- Scale Multi-Campus Dataset for Robot Perception”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...
2024
-
[23]
Traj-LO: In Defense of LiDAR-Only Odometry Using an Effective Continuous-Time Trajectory
X. Zheng and J. Zhu. “Traj-LO: In Defense of LiDAR-Only Odometry Using an Effective Continuous-Time Trajectory”. In: IEEE Robotics and Automation Letters 9.2 (2024), pp. 1961–1968
2024
-
[24]
Fast-lio2: Fast direct lidar-inertial odometry
W. Xu, Y . Cai, D. He, J. Lin, and F. Zhang. “Fast-lio2: Fast direct lidar-inertial odometry”. In: IEEE Transactions on Robotics 38.4 (2022), pp. 2053–2073
2022
-
[25]
Proximal policy optimization algorithms
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. “Proximal policy optimization algorithms”. In: arXiv preprint arXiv:1707.06347 (2017)
2017 arXiv
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.