Pith. sign in

REVIEW 3 major objections 4 minor 27 references

The paper claims that a drone with only one camera and an IMU can estimate metric depth for obstacle avoidance by fitting a monotonic spline that maps relative depth predictions to sparse 3D feature points from a visual-inertial navigation

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A monotonic spline fitted to sparse visual-inertial features rescales monocular relative depth to metric depth, enabling collision avoidance with a single camera and IMU.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Solid MAV depth-rescaling paper with a real hardware demo; the AbsRel metric in Eq. (3) is non-standard and the GT baseline failures in Table I need explanation before trusting the quantitative claims. the 3 major comments →

arxiv 2509.08159 v1 pith:WAW6ZXHZ submitted 2025-09-09 cs.RO cs.AI

Zero-Shot Metric Depth Estimation via Monocular Visual-Inertial Rescaling for Autonomous Aerial Navigation

Steven Yang , Xiaoyu Tian , Kshitij Goel , Wennie Tabib This is my paper
classification cs.RO cs.AI
keywords metric depth estimationmonocular visual-inertial navigationzero-shot rescalingrelative depthcollision avoidancequadrotormonotonic splinesparse 3D features
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a quadrotor can navigate cluttered, a priori unknown environments using only a monocular camera and an IMU, without heavy depth sensors or environment-specific fine-tuning. It does this by taking relative depth images from a pretrained monocular depth network and rescaling them into metric depth using a monotonic spline fitted to the sparse 3D feature map that a visual-inertial navigation system already produces. The central claim is that this online rescaling generalizes across the image well enough to support collision avoidance, with depth estimates running onboard at 15 Hz on a compute-constrained quadrotor. If true, this provides a zero-shot, sensor-light path to metric depth for small aerial robots in unknown environments.

Core claim

The paper's central claim is that relative monocular depth estimates can be turned into metric depth without retraining by fitting a monotonic spline from the sparse 3D feature map produced by a sliding-window visual-inertial navigation system to the relative disparity values in the same image. The fitted function is applied to every pixel of the depth image, and the result—metric depth in meters—is used directly by a motion-primitives planner for collision avoidance. In simulation, the monotonic spline is the most consistent rescaling strategy across both confined and open environments, and on hardware it enabled a quadrotor with only a forward camera and IMU to complete two obstacle-avoida

What carries the argument

The central object is the monotonic spline: a cubic B-spline function fitted by penalized least squares to pairs of relative disparity (from the depth network) and metric disparity (from the VINS sparse feature map). Monotonicity is enforced through a penalty on negative first differences, so the fitted curve preserves the ordering of depths. The spline is fitted online per frame from the sparse feature set and then applied densely to the whole image, which is what converts a scale-ambiguous relative depth image into a metric depth image.

Load-bearing premise

The rescaling curve is fitted only where the visual-inertial system has metrically accurate 3D features, and those features are assumed accurate and spread widely enough to represent the whole scene, including depths beyond their range.

What would settle it

Fly in a straight line toward a flat wall in a low-texture corridor while the tracked VINS features are all on nearby floor texture; record ground-truth distance to the wall with a rangefinder or motion capture and compare the rescaled depth at the wall's pixels. If the predicted wall distance deviates from the true distance by more than the planner's tolerance at the moment the wall enters the image, the assumption that the spline extrapolates beyond the feature range fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Aerial robots can run collision avoidance with only a monocular camera and IMU, since the metric depth image is produced onboard at 15 Hz and feeds the planner directly.
  • No environment-specific fine-tuning or ground-truth depth is needed at deployment; the rescaling is computed on the fly from the VINS feature map.
  • The method shows consistent accuracy in both confined and open simulation environments, with the monotonic spline performing best across both.
  • Existing planners and costmaps that assume metric depth can be used unchanged, since the output is a dense metric image rather than relative depth.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same spline-fitting procedure should transfer to other relative-depth networks, because it only sees disparity pairs and does not depend on the internals of the specific depth network used.
  • A safety-focused planner could treat pixels whose relative disparity lies outside the range covered by current VINS features as unknown, since the paper's limitation section states that extrapolation beyond the sampled feature range is unreliable.
  • Weighting the spline fit by VINS feature covariance would be a natural next step; the paper mentions uncertainty-aware rescaling as future work, and simulation benchmarks could quantify the benefit in low-texture scenes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a zero-shot method for metric depth estimation from a monocular camera and IMU. A VINS sliding-window optimizer produces a sparse 3D feature map; the relative disparity output of DepthAnythingV2 is rescaled into metric disparity by fitting one of several functions (polynomial, exponential, smoothing spline, monotonic spline) to the sparse feature correspondences. The monotonic spline version is selected after simulation ablations, integrated with a motion-primitive planner, and deployed on a quadrotor in a dusty tunnel, where it reportedly runs at 15 Hz and enables collision avoidance. The manuscript also releases software and includes a real-world hardware validation.

Significance. If the reported results are reliable, this is a practical contribution: it offers a lightweight, fine-tuning-free way to obtain metric depth for MAV navigation using sensors already present on many platforms, with a real hardware demonstration in a challenging environment. The open-source release and the careful treatment of real-world operational constraints (dust, compute limits, feature scarcity) are strengths. However, the quantitative evaluation currently contains a nonstandard error metric and implausible ground-truth baseline results, so the central accuracy claim is not yet supported as written.

major comments (3)
  1. [§IV.B, Eq. (3)] The paper defines AbsRel as (1/N)Σ|z_pred−z_gt|/z_pred, while the surrounding text says it is normalized by ground-truth depth. The standard absolute relative error normalizes by z_gt. With the printed denominator, overestimates (z_pred>z_gt) are discounted and underestimates are heavily penalized; for obstacle avoidance this is the wrong asymmetry, because an overestimate is the dangerous case. Since Table I is the primary quantitative evidence, the evaluation should be redone with the standard metric or the choice must be strongly justified.
  2. [Table I] The columns labeled GT are described as an expected upper bound because they fit the rescaling using ground-truth sparse depths. Yet GT often performs much worse than the VINS-based rescaling: Drone Dome deg-1 polynomial GT AbsRel=2.613 vs. VINS=0.894, and deg-3 GT=20.259 vs. VINS=0.457; the weighted-average deg-1 GT=1.081 vs. VINS=0.436. This is implausible under a correct evaluation pipeline. It may result from the nonstandard metric in Eq. (3), or from a fitting/evaluation bug. Either way, the Table I results must be corrected and re-reported before the quantitative comparison can be trusted.
  3. [§III.C and §V] The rescaling pipeline depends on several hand-set hyperparameters (spline knot count, smoothing S, penalties λ and κ, minimum sparse-feature count) with no sensitivity study, so the reported monotonic-spline performance is tied to those choices. More importantly, the Limitation section admits that depth estimates are unreliable outside the depth range spanned by the sampled sparse features. Since this is a core premise of the method, the manuscript should quantify how often and how severely this occurs in the test environments and should temper the 'zero-shot' claim accordingly.
minor comments (4)
  1. [§IV.B, Eq. (4)] The δ1 condition is written as two separate inequalities; this is equivalent to the standard max-ratio condition, but the notation should be clarified. Table headers use 'Delta1' rather than 'δ1'.
  2. [§III.B.3 and §III.B.5] The notation 'C 2 spline' should read 'C^2 spline'; 'preceeding' should be 'preceding'. Please polish these and similar typos.
  3. [Fig. 4] Please clarify what the percentages represent (e.g., fraction of trials) and what 'Total' aggregates across the three sewer maps. The current bar chart is hard to interpret without this detail.
  4. [§IV.C] State explicitly whether the metrics in Table I are computed over all valid image pixels or only at the sparse feature pixels used for fitting. This distinction is important for interpreting the rescaling accuracy and for comparing with prior work such as [10].

Circularity Check

0 steps flagged

No significant circularity: the metric rescaling is a supervised regression on external VINS 3D features, with independent full-image evaluation.

full rationale

The derivation chain is self-contained. The paper fits a monotonic spline f mapping DepthAnythingV2 relative disparity drel to metric disparity dmet, with target values dmet(i)=1/zmet(i) taken from sparse 3D features produced by the VINS sliding-window optimizer, not from the MDE network. The objectives in Section III.B are ordinary penalized least-squares regressions against those external reference points; applying the fitted f to all pixels is interpolation/extrapolation, not an input-output tautology. The evaluation in Table I compares full-image predictions against simulator ground-truth depth, which is independent of the sparse fitted values, so the reported AbsRel and Delta1 are not equal to the fitting residuals by construction. The hardware collision-avoidance trials provide an independent behavioral check of the central claim. Cited prior work—VINS [18], the motion-primitives planner [27], and DepthAnythingV2—is used as components, not as a source of the paper's conclusion; no uniqueness theorem or ansatz is imported from the authors' own prior work in a load-bearing way. The paper explicitly acknowledges the main extrapolation limitation: 'Depth estimation becomes unreliable when predictions are made outside the depth range represented by the sampled sparse features,' which is an honest scope restriction rather than an internal circularity. The non-standard AbsRel denominator in Eq. (3) and the surprising GT-baseline failures in Table I are evaluation-correctness concerns, not circular-reasoning defects.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The method's core computation is a per-frame regression from relative disparity to metric disparity. The only external signal that carries metric scale is the sparse 3D feature map from VINS; the MDE provides shape. The central assumptions are the accuracy and distribution of those VINS features, the affine-invariance of the MDE output, and known calibration. No new physical entities are introduced.

free parameters (6)
  • spline knot count = 10
    Section III-C: 'we find that 10 knots works well.' This is a hand-chosen hyperparameter selected without a reported validation procedure.
  • smoothing hyperparameter S = not reported
    Section III-B.3: the smoothing spline objective requires S, controlling the trade-off between error and smoothness. The paper does not report how S is set.
  • smoothing penalty lambda = not reported
    Section III-B.4: the monotonic smoothing spline objective includes lambda for the third-derivative penalty. The value is not given.
  • non-monotonicity penalty kappa = not reported
    Section III-B.4 and B.5: the objective includes kappa, the weight on the non-monotonicity penalty V^(1/2)D^(1)beta. The value is not reported.
  • minimum sparse feature count = 10
    Section III-C: 'we require that there be at least 10 sparse features, which is empirically determined.' This threshold affects which frames produce output.
  • depth clipping bounds = 0.05 m to 65 m
    Section III-C: depth values outside this range are clipped and only pixels with ground truth depth within the range are used. This is an implementation choice that changes the fitted mapping.
axioms (4)
  • domain assumption The sparse 3D feature map from the VINS sliding-window optimizer is metrically accurate up to the VINS estimation error.
    Section III-A: the 3D feature positions are used as the metric reference values for fitting f. If the VINS scale or positions are biased, the fitted curve is biased. This is acknowledged in the Limitations section.
  • domain assumption DepthAnythingV2 outputs affine-invariant relative depth (or disparity) predictions.
    Section III-B: the problem is formulated as mapping relative disparity to metric disparity, which relies on the known affine-invariance property of relative MDE models (cited from [15]). The paper assumes this property holds for the specific model used.
  • domain assumption The camera intrinsics and extrinsics between camera, IMU, and body frames are known.
    Section III-A: feature projection from the VINS map to the current image plane uses the pinhole camera model and known calibration (the quantities R_WBi, R_BiCi, etc. are said to be known).
  • standard math Spline fitting via penalized least squares is well-posed and produces a usable continuous mapping.
    Section III-B.4 and B.5: the monotonic spline approach uses B-spline basis functions and penalized least squares, a standard numerical technique.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Zero-Shot Metric Depth Estimation via Monocular Visual-Inertial Rescaling for Autonomous Aerial Navigation." pith.science (2026). https://pith.science/paper/WAW6ZXHZ

@misc{pith2026250908159,
  author       = {Pith},
  title        = {Pith review of: Zero-Shot Metric Depth Estimation via Monocular Visual-Inertial Rescaling for Autonomous Aerial Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WAW6ZXHZ}},
  note         = {Machine review of arXiv:2509.08159}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents a methodology to predict metric depth from monocular RGB images and an inertial measurement unit (IMU). To enable collision avoidance during autonomous flight, prior works either leverage heavy sensors (e.g., LiDARs or stereo cameras) or data-intensive and domain-specific fine-tuning of monocular metric depth estimation methods. In contrast, we propose several lightweight zero-shot rescaling strategies to obtain metric depth from relative depth estimates via the sparse 3D feature map created using a visual-inertial navigation system. These strategies are compared for their accuracy in diverse simulation environments. The best performing approach, which leverages monotonic spline fitting, is deployed in the real-world on a compute-constrained quadrotor. We obtain on-board metric depth estimates at 15 Hz and demonstrate successful collision avoidance after integrating the proposed method with a motion primitives-based planner.

Figures

Figures reproduced from arXiv: 2509.08159 by Kshitij Goel, Steven Yang, Wennie Tabib, Xiaoyu Tian.

Figure 1
Figure 1. Figure 1: Image and data corresponding to one hardware experiment [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the approach to rescale predicted depth from an MDE network using a metrically accurate 3D sparse feature map [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Examples of images used for ablation study derived from [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance comparison of autonomous navigation in the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Representative simulated scenes used to evaluated the proposed approach. The colors ranging from red (closer) to purple (further [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Images and data from one of the hardware experiments. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of environmental details during hardware ex [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages · 1 internal anchor

  1. [1]

    Autonomous cave surveying with an aerial robot,

    W. Tabib, K. Goel, J. Yao, C. Boirum, and N. Michael, “Autonomous cave surveying with an aerial robot,” IEEE Transactions on Robotics , vol. 38, no. 2, pp. 1016–1032, 2022

    work page 2022

  2. [2]

    Real-time information-theoretic exploration with gaussian mixture model maps

    W. Tabib, K. Goel, J. W. Yao, M. Dabhi, C. Boirum, and N. Michael, “Real-time information-theoretic exploration with gaussian mixture model maps.” in Robotics: Science and Systems , vol. 2, 2019

    work page 2019

  3. [3]

    Real-time monocular dense mapping on aerial robots using visual-inertial fusion,

    Z. Yang, F. Gao, and S. Shen, “Real-time monocular dense mapping on aerial robots using visual-inertial fusion,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 4552– 4559

    work page 2017

  4. [4]

    Autonomous aerial navigation using monocular visual- inertial fusion,

    Y . Lin, F. Gao, T. Qin, W. Gao, T. Liu, W. Wu, Z. Yang, and S. Shen, “Autonomous aerial navigation using monocular visual- inertial fusion,” Journal of Field Robotics , vol. 35, no. 1, pp. 23–51, 2018

    work page 2018

  5. [5]

    Unidepthv2: Universal monocular metric depth estimation made simpler,

    L. Piccinelli, C. Sakaridis, Y .-H. Yang, M. Segu, S. Li, W. Abbeloos, and L. Van Gool, “Unidepthv2: Universal monocular metric depth estimation made simpler,” arXiv preprint arXiv:2502.20110 , 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Depth Anything V2,

    L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao, “Depth Anything V2,” Advances in Neural Information Processing Systems, vol. 37, pp. 21 875–21 911, Dec. 2024

    work page 2024

  7. [7]

    Reactive Collision Avoidance for Safe Agile Navigation

    A. Saviolo, N. Picello, J. Mao, R. Verma, and G. Loianno, “Re- active collision avoidance for safe agile navigation,” arXiv preprint arXiv:2409.11962, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    Time-optimized safe navigation in unstructured environments through learning based depth completion,

    J. Mao, R. C. Srinivas, S. Nogar, and G. Loianno, “Time-optimized safe navigation in unstructured environments through learning based depth completion,” arXiv preprint arXiv:2506.14975 , 2025

    work page arXiv 2025

  9. [9]

    Monocular Visual- Inertial Depth Estimation,

    D. Wofk, R. Ranftl, M. M ¨uller, and V . Koltun, “Monocular Visual- Inertial Depth Estimation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , May 2023, pp. 6095–6101

    work page 2023

  10. [10]

    A simple yet effective test-time adaptation for zero-shot monocular metric depth estimation,

    R. Marsal, A. Chapoutot, P. Xu, and D. Filliat, “A simple yet effective test-time adaptation for zero-shot monocular metric depth estimation,” arXiv preprint arXiv:2412.14103 , 2024

    arXiv 2024

  11. [11]

    Zoedepth: Zero-shot transfer by combining relative and metric depth,

    S. F. Bhat, R. Birkl, D. Wofk, P. Wonka, and M. M ¨uller, “Zoedepth: Zero-shot transfer by combining relative and metric depth,” arXiv preprint arXiv:2302.12288, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Unidepth: Universal monocular metric depth estimation,

    L. Piccinelli, Y .-H. Yang, C. Sakaridis, M. Segu, S. Li, L. Van Gool, and F. Yu, “Unidepth: Universal monocular metric depth estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 10 106–10 116

    work page 2024

  13. [13]

    Metric3d: Towards zero-shot metric 3d prediction from a single image,

    W. Yin, C. Zhang, H. Chen, Z. Cai, G. Yu, K. Wang, X. Chen, and C. Shen, “Metric3d: Towards zero-shot metric 3d prediction from a single image,” in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 9043–9053

    work page 2023

  14. [14]

    Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation,

    M. Hu, W. Yin, C. Zhang, Z. Cai, X. Long, H. Chen, K. Wang, G. Yu, C. Shen, and S. Shen, “Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, pp. 10 579–10 596, Dec. 2024

    work page 2024

  15. [15]

    Depth anything: Unleashing the power of large-scale unlabeled data,

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 10 371–10 381

    2024

  16. [16]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,

    M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM , vol. 24, no. 6, pp. 381–395, 1981

    work page 1981

  17. [17]

    Distinctive image features from scale-invariant key- points,

    D. G. Lowe, “Distinctive image features from scale-invariant key- points,” International Journal of Computer Vision , vol. 60, no. 2, pp. 91–110, 2004

    2004

  18. [18]

    Resource-constrained state estimation with multi-modal sensing,

    J. W. Yao, “Resource-constrained state estimation with multi-modal sensing,” Ph.D. dissertation, Carnegie Mellon University, Pittsburgh, PA, April 2020

    work page 2020

  19. [19]

    Good features to track,

    J. Shi et al. , “Good features to track,” in 1994 Proceedings of IEEE conference on computer vision and pattern recognition . IEEE, 1994, pp. 593–600

    work page 1994

  20. [20]

    An iterative image registration technique with an application to stereo vision,

    B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in IJCAI’81: 7th international joint conference on Artificial intelligence , vol. 2, 1981, pp. 674–679

    work page 1981

  21. [21]

    Robust real-time visual odometry with a single camera and an imu,

    L. Kneip, M. Chli, and R. Siegwart, “Robust real-time visual odometry with a single camera and an imu,” in Proceedings of the British Ma- chine Vision Conference 2011 . British Machine Vision Association, 2011

    work page 2011

  22. [22]

    An algorithm for smoothing, differentiation and in- tegration of experimental data using spline functions,

    P. Dierckx, “An algorithm for smoothing, differentiation and in- tegration of experimental data using spline functions,” Journal of Computational and Applied Mathematics , vol. 1, no. 3, pp. 165–184, 1975

    work page 1975

  23. [23]

    Unimodal smoothing,

    P. H. Eilers, “Unimodal smoothing,” Journal of Chemometrics: A Journal of the Chemometrics Society , vol. 19, no. 5-7, pp. 317–328, 2005

    work page 2005

  24. [24]

    De Boor, A practical guide to splines

    C. De Boor, A practical guide to splines . springer New York, 1978, vol. 27

    work page 1978

  25. [25]

    Flightmare: A flexible quadrotor simulator,

    Y . Song, S. Naji, E. Kaufmann, A. Loquercio, and D. Scaramuzza, “Flightmare: A flexible quadrotor simulator,” in Conference on Robot Learning. PMLR, 2021, pp. 1147–1157

    work page 2021

  26. [26]

    Depth map prediction from a single image using a multi-scale deep network,

    D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” Advances in neural information processing systems , vol. 27, 2014

    2014

  27. [27]

    Rapid quadrotor navigation in diverse environments using an onboard depth camera,

    J. Lee, A. Rathod, K. Goel, J. Stecklein, and W. Tabib, “Rapid quadrotor navigation in diverse environments using an onboard depth camera,” in 2024 IEEE International Symposium on Safety Security Rescue Robotics (SSRR) , 2024, pp. 18–25

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.