REVIEW 3 major objections 4 minor 24 references
Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing
T0 review · 3 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read A tightly-coupled filter that feeds raw gate-corner pixel errors directly into an error-state Kalman filter can keep a monocular racing drone localized with as few as two visible corners, and an offline factor-graph smoother supplies the re
desk verdict Gate-corner ESKF is a real ADR contribution, but the headline accuracy number contradicts the table and the A2RL reference is partly self-derived — fix those before trusting the numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the measurement residual r = u_tilde - h(x_bar, p_W_G): a direct pixel-space comparison between a detected gate corner and the projection of the known 3D gate corner through the current state. In the filter, this residual's Jacobian with respect to the error state replaces the pose-estimation layer, and its covariance is inflated by a robust weight w = min(1, tau/e) based on Mahalanobis distance. In the offline smoother, the same corner reprojection factors plus IMU preintegration and soft priors from the online filter are optimized in one batch to refine the trajectory and produce a reference for evaluation.
What would settle it
Install a motion-capture system on a competition-style course and compare the offline smoother's trajectory to the motion-capture ground truth. If the smoother's deviation from motion capture is comparable to or larger than its reported gap from the online filter, the race reference is not independent and the 0.152 m RMSE is not an accuracy figure. A cheaper check: run the filter on a public dataset with motion-capture ground truth while varying the minimum visible corners; if two-corner updates do not beat four-corner updates there, the central claim fails.
Extended reading notes
Core claim
ADR-VINS is an error-state Kalman filter that treats each 2D gate-corner detection as an innovation by projecting the known 3D corner into the image and computing a pixel residual. Because it bypasses the Perspective-n-Points solver and its RANSAC filtering, it needs only two visible corners for a valid update, and robust reweighting inflates the measurement covariance for outliers. ADR-FGO is a batch factor-graph optimization that combines IMU preintegration, gate-corner reprojection factors, soft priors from ADR-VINS, and a camera-IMU extrinsics factor to smooth the full trajectory and serve as a reference. On an open high-speed flight dataset with motion-capture ground truth, the filter r
Load-bearing premise
The evaluation reference used for the uninstrumented race is an offline smoother that is anchored by soft priors taken from the same online estimator and consumes the same gate-corner detections, so if the smoother is dragged toward the estimator's drift, the race error numbers measure self-consistency rather than independent accuracy.
Editorial extensions
If this is right
- Real-time systems no longer need four visible corners; partial gate views become usable updates, reducing drift between gates.
- Eliminating the pose-solver and rejection stages cuts per-frame latency (reported under 9 ms, with detection dominating), making the pipeline fit tight camera deadlines.
- The offline smoother gives a repeatable evaluation protocol for GNSS- and motion-capture-denied arenas, letting estimation, control, and total error be separated after a flight.
- The same filter structure could generalize to any known-landmark task where distinctive corners can be detected in a monocular camera.
- The smoother's refined trajectory could also be used to retune the online filter or to supervise learned motion models for visual outages.
Reading between the lines
- The race-error numbers are consistency-with-a-smoothed-version rather than fully independent accuracy: because the smoother is anchored by soft priors from the online filter and consumes the same gate-corner detections, the reported 0.152 m may understate true error if the reference is pulled toward the estimator's drift.
- With a two-corner minimum, the filter's accuracy becomes highly sensitive to systematic detection bias; biased corner labels could yield a self-consistent but offset trajectory, a failure mode worth testing by shifting detections in a motion-capture experiment.
- The evaluation protocol could transfer to other structured environments—warehouse or agricultural flight, for instance—where a known map and distinctive corners replace racing gates.
- The smoother's refined extrinsics factor could be fed back into the online estimator after a crash or hard impact, potentially removing a mechanical-shift failure mode.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ADR-VINS, an Error-State Kalman Filter (ESKF) based monocular visual-inertial state estimator for autonomous drone racing. Instead of the conventional cascade of a general VIO with a PnP-based gate pose correction, ADR-VINS directly feeds gate-corner pixel reprojection errors into the filter, enabling updates with as few as two visible corners and using Huber reweighting in place of RANSAC. The paper also introduces ADR-FGO, an offline factor-graph optimizer that fuses IMU preintegration, gate-corner factors, ADR-VINS pose priors, and extrinsic refinement to produce a reference trajectory for post-flight evaluation. The system is evaluated on the TII-RATM dataset (MoCap ground truth) and deployed in the A2RL Drone Championship Season 2, where it is reported to run at 20.9 m/s with low reprojection error and sub-9 ms per-frame latency.
Significance. If the reported accuracy and robustness hold, the paper makes a useful practical contribution: a tightly coupled, PnP-free, low-latency estimator that degrades gracefully under partial gate visibility, plus an offline smoother for uninstrumented environments. The two-corner ablation (Fig. 5) and the runtime breakdown (Table V) support the central mechanism. However, the current empirical validation contains a serious internal inconsistency in the headline accuracy number, and the A2RL evaluation reference is partly self-referential. These issues must be resolved before the quantitative claims can be accepted.
major comments (3)
- [Abstract / Table II] The headline quantitative claim is not reproducible from the paper's own data. The per-sequence translation errors in Table II (05P .130, 06P .134, 11P .132, 12P .169, 05A .132, 06A .132, 11A .107, 12A .099, 17A .191, 18A .208) sum to 1.434 m, which averages to 0.1434 m, not the reported 0.134 m. The body text reports piloted and autonomous averages of 0.141 m and 0.145 m, which also average to 0.143 m. The abstract and the 'Avg' row state 0.134 m. Thus the most prominent accuracy figure is unsupported by the presented numbers; either the table or the headline must be corrected, and the entire set of derived claims (e.g., percentage improvements over baselines) should be recomputed accordingly.
- [Sec. IV-A3, Eq. (20); Sec. V-D] The A2RL evaluation uses ADR-FGO as the 'GT (ADR-FGO)' reference, but ADR-FGO is anchored by soft priors taken from ADR-VINS states (Eq. 20) and consumes the same gate-corner detections as ADR-VINS. The reported A2RL translation RMSE of 0.152 m is therefore a consistency measure with a smoothed version of the same system, not an independent accuracy measurement. This does not invalidate the TII-RATM MoCap results, but it weakens the competition-based accuracy claims in Sec. V-D and Table IV.
- [Sec. V-C, Initialization] All online methods are initialized with ground-truth states, as stated in Sec. V-C. This removes the initialization and convergence challenge that is often a major differentiator in practice, and it makes the reported improvement percentages (67–74% in translation) less generalizable to real deployment. In addition, no error bars or repeated trials are reported for any of the three online methods; Table II contains one run per sequence. Without this information, the statistical significance of the claimed improvements cannot be assessed.
minor comments (4)
- [Eq. (16)] The Huber weight formula appears to have a typo: w = min(1.0, τe/e) should presumably be w = min(1.0, τ/e). Please clarify.
- [Sec. V-D] The text refers to 'Sec.V.B' when comparing A2RL reprojection errors; the intended cross-reference is probably Sec. V-C (TII-RATM dataset experiments), not Sec. V-B (Metrics).
- [Abstract vs. full text] The abstract in the front matter reports 0.143 m average RMS translation error, while the abstract in the full text body reports 0.134 m, and the body text reports 0.141/0.145 m for the two subsets. These numbers should be made consistent throughout after the Table II issue is resolved.
- [Introduction] Minor typographical issues such as 'MA Vs' in the introduction (should be 'MAVs') and the spacing in 'Kore an Goverment' (should be 'Korean Government') should be corrected.
Circularity Check
A2RL evaluation is self-referential (ADR-FGO is softly anchored to ADR-VINS and uses the same detections); TII-RATM results remain independent, so partial circularity only.
-
self definitional
[Sec. IV-A3 (Eq. 20), Sec. IV-C (Eq. 23), Sec. V-D (Fig. 7 / Table IV)]
"We anchor the global optimization to the real-time ADR-VINS results using soft priors to improve convergence: r_prior,k = x_k ⊖ x̄_VINS_k ... As MoCap systsem is not available in the competition site, we utilize ADR-FGO to generate a reference trajectory for evaluation and analysis."
On A2RL, the 'GT (ADR-FGO)' reference is produced by a factor graph whose joint cost (Eq. 23) includes the soft prior r_prior,k = x_k ⊖ x̄_VINS_k from the very estimator being evaluated, and whose visual factors (Eq. 19) use the same gate-corner detections as ADR-VINS. The reported 0.152 m RMSE and Table IV's relative improvements therefore measure consistency between ADR-VINS and a smoothed trajectory pulled toward ADR-VINS, not accuracy against an independent reference. This is partial (soft prior plus independent IMU/visual factors), and it does not affect the TII-RATM experiments, which use MoCap ground truth.
full rationale
The main derivation of ADR-VINS - an ESKF that fuses IMU with direct gate-corner reprojection residuals - is self-contained: the filter equations (Eqs. 5-17) follow standard ESKF and reprojection geometry, and the TII-RATM validation uses external MoCap ground truth, so those accuracy numbers are not circular. ADR-FGO's own accuracy is also checked against MoCap on TII-RATM (0.060 m), giving independent evidence that the smoother is useful. The genuine circularity is confined to the A2RL deployment evaluation: with no MoCap, the paper labels ADR-FGO as 'GT' while the FGO is initialized/softly anchored by ADR-VINS priors (Eq. 20) and consumes the same detections, so the resulting 'ADR-VINS error' numbers are consistency-with-smoothed-self measures. The paper discloses the lack of MoCap, but still reports the value as translational RMSE, which overstates independence. There is also a non-circular data inconsistency: the abstract's 0.134 m average does not match the arithmetic mean of the Table II per-sequence e_t values (sum 1.434 / 10 = 0.143 m), a separate reproducibility defect not scored here. Overall, because the central TII-RATM claims rest on MoCap and the self-reference is secondary and disclosed, the circularity is partial, not structural: score 4.
Assumptions & free parameters
free parameters (8)
- IMU noise stds σ_a, σ_ω, σ_ba, σ_bω =
not reported
- Visual measurement covariance R_cov =
not reported
- Huber threshold τ =
not reported
- Association thresholds (75 px, ρ>0.2) =
75 px, 0.2
- Gate distance cutoff 15 m =
15 m
- Reordering probe distance d=3 m =
3 m
- Keyframe time threshold τ_t =
not reported
- FGO covariance weights Σ_IMU, Σ_corner, Σ_prior, Σ_ext =
not reported
assumptions (5)
- domain assumption Known, fixed, accurate gate map G in world frame
- domain assumption IMU model with bias random walk and Gaussian noise (Eq. 8)
- domain assumption RTMO-nano detections are unbiased and training sequences suffice for deployment (Sec. III-A)
- domain assumption Gravity-aligned z-axis in world frame (Eq. 1, g_z)
- domain assumption Factor graph optimization converges to a global minimum and soft priors are sufficiently weak
Cite this review
Pith. "Pith review of Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing." pith.science (2026). https://pith.science/paper/FGNKI3ZZ
@misc{pith2026260302742,
author = {Pith},
title = {Pith review of: Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing},
year = {2026},
howpublished = {\url{https://pith.science/paper/FGNKI3ZZ}},
note = {Machine review of arXiv:2603.02742}
}
read the original abstract
Autonomous drone racing (ADR) demands state estimation that is simultaneously computationally efficient and resilient to the perceptual degradation experienced during extreme velocity and maneuvers. Traditional frameworks typically rely on conventional visual-inertial pipelines with loosely-coupled gate-based Perspective-n-Points (PnP) corrections that suffer from a rigid requirement for four visible features and information loss in intermediate steps. Furthermore, the absence of GNSS and Motion Capture systems in uninstrumented, competitive racing environments makes the objective evaluation of such systems remarkably difficult. To address these limitations, we propose ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing. Our approach integrates direct pixel reprojection errors from gate corners features as innovation terms within the filter. By bypassing intermediate PnP solvers, ADR-VINS maintains valid state updates with as few as two visible corners and utilizes robust reweighting instead of RANSAC-based schemes to handle outliers, enhancing computational efficiency. Furthermore, we introduce ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments. The proposed system is validated using TII-RATM dataset, where ADR-VINS achieves an average RMS translation error of 0.143 m, while ADR-FGO yields 0.060 m as a smoothing-based reference. Finally, ADR-VINS was successfully deployed in the A2RL Drone Championship Season 2, maintaining stable and robust estimation despite noisy detections during high-agility flight at top speeds of 20.9 m/s. We further utilize ADR-FGO for post-flight evaluation in uninstrumented racing environments.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Autonomous drone racing: A survey,
D. Hanover, A. Loquercio, L. Bauersfeld, A. Romero, R. Penicka, Y . Song, G. Cioffi, E. Kaufmann, and D. Scaramuzza, “Autonomous drone racing: A survey,”IEEE Transactions on Robotics, 2024
2024
-
[2]
Alphapilot: Autonomous drone racing,
P. Foehn, D. Brescianini, E. Kaufmann, T. Cieslewski, M. Gehrig, M. Muglikar, and D. Scaramuzza, “Alphapilot: Autonomous drone racing,”Autonomous Robots, vol. 46, no. 1, pp. 307–320, 2022
2022
-
[3]
The sensing, state-estimation, and control behind the winning entry to the 2019 artificial intelligence robotic racing competition,
C. De Wagter, F. Paredes-Vall ´es, N. Sheth, and G. C. de Croon, “The sensing, state-estimation, and control behind the winning entry to the 2019 artificial intelligence robotic racing competition,”Field Robotics, vol. 2, pp. 1263–1290, 2022
2019
-
[4]
Champion-level drone racing using deep reinforce- ment learning,
E. Kaufmann, L. Bauersfeld, A. Loquercio, M. M ¨uller, V . Koltun, and D. Scaramuzza, “Champion-level drone racing using deep reinforce- ment learning,”Nature, vol. 620, no. 7976, pp. 982–987, 2023
2023
-
[5]
On your own: Pro-level autonomous drone racing in uninstrumented arenas,
M. Bosello, F. Pinzarrone, S. Kiade, D. Aguiari, Y . Keuter, A. AlShe- hhi, G. Caminati, K. L. Wong, K. S. Chou, J. Halepota, F. Alneyadi, J. Panerati, and G. Pau, “On your own: Pro-level autonomous drone racing in uninstrumented arenas,”IEEE Robotics and Automation Letters, vol. 11, no. 3, pp. 2674–2681, 2026
2026
-
[6]
F. Nov ´ak, M. Petrl ´ık, M. Novosad, P. M. Gupta, R. P ˇeniˇcka, and M. Saska, “Vision-only uav state estimation for fast flights without external localization systems: A2rl drone racing finalist approach,” arXiv, preprint arXiv:2602.01860, 2026
arXiv 2026
-
[7]
Drift-corrected monocular vio and perception-aware planning for autonomous drone racing,
M. B. Azhari, D. Han, J. I. You, S. Park, and D. H. Shim, “Drift-corrected monocular vio and perception-aware planning for autonomous drone racing,”arXiv, preprint arXiv:2512.20475, 2025
arXiv 2025
-
[8]
Monorace: Winning champion-level drone racing with robust monocular ai,
S. A. Bahnam, R. Ferede, T. M. Blaha, A. E. Lang, E. Lucassen, Q. Missinne, A. E. C. Verraest, C. D. Wagter, and G. C. H. E. de Croon, “Monorace: Winning champion-level drone racing with robust monocular ai,”arXiv, preprint arXiv:2601.15222, 2026
arXiv 2026
Show all 24 references
-
[9]
Robust visual in- ertial odometry using a direct ekf-based approach,
M. Bloesch, S. Omari, M. Hutter, and R. Siegwart, “Robust visual in- ertial odometry using a direct ekf-based approach,” in2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 298–304, IEEE, 2015
2015
-
[10]
Openvins: A research platform for visual-inertial estimation,
P. Geneva, K. Eckenhoff, W. Lee, Y . Yang, and G. Huang, “Openvins: A research platform for visual-inertial estimation,” in2020 IEEE Inter- national Conference on Robotics and Automation (ICRA), pp. 4666– 4672, IEEE, 2020
2020
-
[11]
Vins-mono: A robust and versatile monoc- ular visual-inertial state estimator,
T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monoc- ular visual-inertial state estimator,”IEEE transactions on robotics, vol. 34, no. 4, pp. 1004–1020, 2018
2018
-
[12]
Quaternion kinematics for the error-state kalman filter,
J. Sola, “Quaternion kinematics for the error-state kalman filter,”arXiv preprint arXiv:1711.02508, 2017
2017 arXiv
-
[13]
Factor graphs for robot perception,
F. Dellaert and M. Kaess, “Factor graphs for robot perception,” Foundations and Trends® in Robotics, vol. 6, no. 1-2, pp. 1–139, 2017
2017
-
[14]
maplab 2.0–a modular and multi-modal mapping framework,
A. Cramariuc, L. Bernreiter, F. Tschopp, M. Fehr, V . Reijgwart, J. Nieto, R. Siegwart, and C. Cadena, “maplab 2.0–a modular and multi-modal mapping framework,”IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 520–527, 2022
2022
-
[15]
On-manifold preintegration for real-time visual–inertial odometry,
C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, “On-manifold preintegration for real-time visual–inertial odometry,”IEEE transac- tions on robotics, vol. 33, no. 1, pp. 1–21, 2016
2016
-
[16]
Rtmo: Towards high-performance one-stage real-time multi-person pose estimation,
P. Lu, T. Jiang, Y . Li, X. Li, K. Chen, and W. Yang, “Rtmo: Towards high-performance one-stage real-time multi-person pose estimation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1491–1500, 2024
2024
-
[17]
SymForce: Symbolic Computation and Code Generation for Robotics,
H. Martiros, A. Miller, N. Bucki, B. Solliday, R. Kennedy, J. Zhu, T. Dang, D. Pattison, H. Zheng, T. Tomic, P. Henry, G. Cross, J. VanderMey, A. Sun, S. Wang, and K. Holtz, “SymForce: Symbolic Computation and Code Generation for Robotics,” inProceedings of Robotics: Science a...
2022
-
[18]
Race against the machine: A fully-annotated, open-design dataset of autonomous and piloted high-speed flight,
M. Bosello, D. Aguiari, Y . Keuter, E. Pallotta, S. Kiade, G. Caminati, F. Pinzarrone, J. Halepota, J. Panerati, and G. Pau, “Race against the machine: A fully-annotated, open-design dataset of autonomous and piloted high-speed flight,”IEEE Robotics and Automation Letters, vol...
2024
-
[19]
Self-supervised monocular visual drone model identification through improved oc- clusion handling,
S. A. Bahnam, C. De Wagter, and G. C. De Croon, “Self-supervised monocular visual drone model identification through improved oc- clusion handling,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 18977–18984, 2025
2025
-
[20]
Pampc: Perception- aware model predictive control for quadrotors,
D. Falanga, P. Foehn, P. Lu, and D. Scaramuzza, “Pampc: Perception- aware model predictive control for quadrotors,” inIEEE International Conference on Intelligent Robots and Systems, IEEE, 2018
2018
-
[21]
Time-optimal gate- traversing planner for autonomous drone racing,
C. Qin, M. S. Michet, J. Chen, and H. H.-T. Liu, “Time-optimal gate- traversing planner for autonomous drone racing,” in2024 IEEE Inter- national Conference on Robotics and Automation (ICRA), pp. 8693– 8699, 2024
2024
-
[22]
Pegasus simulator: An isaac sim framework for multiple aerial vehicles simulation,
M. Jacinto, J. Pinto, J. Patrikar, J. Keller, R. Cunha, S. Scherer, and A. Pascoal, “Pegasus simulator: An isaac sim framework for multiple aerial vehicles simulation,” in2024 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 917–922, 2024
2024
-
[23]
Learned inertial odometry for autonomous drone racing,
G. Cioffi, L. Bauersfeld, E. Kaufmann, and D. Scaramuzza, “Learned inertial odometry for autonomous drone racing,”IEEE Robotics and Automation Letters, vol. 8, no. 5, pp. 2684–2691, 2023
2023
-
[24]
Reaching the limit in autonomous racing: Optimal control versus reinforcement learning,
Y . Song, A. Romero, M. M ¨uller, V . Koltun, and D. Scaramuzza, “Reaching the limit in autonomous racing: Optimal control versus reinforcement learning,”Science Robotics, vol. 8, no. 82, p. eadg1462, 2023
2023
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.