pith. sign in

arxiv: 2606.21669 · v1 · pith:AXTOMIGWnew · submitted 2026-06-19 · 💻 cs.RO

Online Learning of Robust Legged Odometry with Minimal Exteroceptive Supervision

Abhijeet M. Kulkarni , Yuze Du , Guoquan Huang This is my paper

Pith reviewed 2026-06-26 14:08 UTC · model grok-4.3

classification 💻 cs.RO
keywords legged odometryonline learningproprioceptive velocityexteroceptive supervisionneural networkquadruped robotssensor fusionInvariant EKF
0
0 comments X

The pith

A neural network learns proprioceptive velocity online under exteroceptive supervision to deliver calibration-free legged odometry that falls back gracefully when external sensors degrade.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that exteroceptive motion pipelines can serve as a continuous training signal for a velocity neural network driven only by proprioceptive inputs such as joint encoders and IMU readings. Once trained, the network supplies velocity estimates that an Invariant EKF fuses with IMU data, removing any requirement for explicit sensor-to-sensor calibration or platform-specific kinematic models. When environmental conditions cause exteroception to fail, the system switches automatically to the learned proprioceptive model. Demonstrations on multiple quadruped platforms show that the same trained network maintains motion estimates across hardware changes and degraded scenes.

Core claim

By treating established exteroceptive motion pipelines as a supervisory signal, an online learned velocity neural network can be trained directly from proprioceptive data; the resulting velocity estimates are fused with IMU measurements inside an Invariant EKF to produce odometry that requires neither exteroceptive-to-proprioceptive calibration nor explicit kinematic modeling and that seamlessly reverts to the learned model whenever exteroception becomes unavailable or unreliable.

What carries the argument

The online learned velocity neural network, trained by continuous exteroceptive supervision and fused with IMU data inside an Invariant EKF.

If this is right

  • No explicit calibration between exteroceptive and proprioceptive sensors is required.
  • Platform-specific kinematic models are unnecessary for the velocity estimator.
  • The system switches automatically to the learned proprioceptive velocity when exteroception degrades.
  • The same training procedure applies across different quadruped hardware without modification.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could shorten deployment time for new legged robots by removing per-platform modeling steps.
  • Intermittent availability of exteroception during operation might allow continued online refinement of the velocity network.
  • Similar supervision patterns could be explored for other proprioceptive quantities such as contact forces.

Load-bearing premise

Exteroceptive motion pipelines must remain accurate enough during training periods to provide reliable supervision for the proprioceptive velocity network.

What would settle it

A measurable increase in position drift on a new quadruped platform after the network has been trained on a different platform, or a sharp accuracy drop once exteroception is removed, would show the learned model does not generalize without calibration.

Figures

Figures reproduced from arXiv: 2606.21669 by Abhijeet M. Kulkarni, Guoquan Huang, Yuze Du.

Figure 1
Figure 1. Figure 1: The proposed legged odometry system. The InEKF propagates with IMU and updates with velocity [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evolution of 10 randomly chosen (out of D “ 8227) readout MLP weights trained with full ground-truth velocity supervision on the ATRIUM sequence from the Spot dataset [38]. The readout MLP converges quickly, as evident from Fig￾ure 2, suggesting that partial su￾pervision suffices to learn a re￾liable map from the ESN state to the robot’s base velocity. We therefore evaluate the effect of limiting supervisi… view at source ↗
Figure 3
Figure 3. Figure 3: Hallway exploration with lighting degradation. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Lavatory exploration with the lights turned off mid-spin. In [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
read the original abstract

Robust locomotion and navigation for legged robots relies heavily on dependable odometry. Traditional multi-sensor fusion for such state estimation requires meticulous sensor calibration and platform-specific kinematic modeling, which complicates deployment. Industrially packaged exteroceptive sensors can provide accurate motion tracking but remain vulnerable to perceptually degraded conditions. We thus develop a plug-and-play, robust legged odometry system that eliminates the need for explicit exteroceptive-to-proprioceptive calibration or system kinematic modeling. Our approach leverages established exteroceptive motion pipelines as a continuous supervisory signal to train an online learned velocity neural network directly from proprioceptive data. An Invariant EKF (InEKF) is then used to fuse the learned proprioceptive or exteroceptive velocity (if any) and IMU data. When exteroception fails due to environmental degradation, the system seamlessly falls back to using the learned proprioceptive model, yielding a resilient legged odometry that readily adapts to new hardware. We demonstrate the platform-agnostic, easily deployable nature of our approach on different quadruped platforms, showcasing promising results in maintaining robust motion estimation across challenging scenarios.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims to develop a plug-and-play robust legged odometry system that uses established exteroceptive motion pipelines as continuous supervisory signals to train an online velocity neural network from proprioceptive data, eliminating explicit exteroceptive-to-proprioceptive calibration and kinematic modeling. An InEKF fuses the learned or exteroceptive velocity with IMU data, enabling seamless fallback to the proprioceptive model when exteroception degrades. The approach is demonstrated as platform-agnostic on multiple quadruped platforms in challenging scenarios.

Significance. If the results hold with sufficient validation, the method could lower barriers to deploying legged robots by removing calibration and modeling overhead while improving resilience via learned proprioceptive fallback. The online learning with minimal supervision and InEKF integration represent practical strengths if supported by quantitative evidence of generalization and noise tolerance.

major comments (2)
  1. [Abstract] The central claim of robustness via fallback to the learned proprioceptive model (Abstract) depends on exteroceptive pipelines supplying sufficiently accurate continuous supervision during online training; no quantitative bounds on acceptable supervision noise, bias, or required convergence time before exteroception removal are provided, leaving the generalization guarantee unverified.
  2. [Abstract and Experiments] The abstract states 'promising results' on different platforms but provides no error metrics, ablation studies, baseline comparisons, or derivation details for the velocity NN or InEKF; this absence undermines assessment of whether the elimination of calibration and kinematic modeling is achieved without inheriting supervision errors.
minor comments (1)
  1. [Method] Clarify the precise proprioceptive inputs to the neural network and the exact state representation used in the InEKF fusion step.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their insightful comments, which help us improve the clarity and rigor of our work. We provide point-by-point responses to the major comments and indicate the revisions planned for the manuscript.

read point-by-point responses

  1. Referee: [Abstract] The central claim of robustness via fallback to the learned proprioceptive model (Abstract) depends on exteroceptive pipelines supplying sufficiently accurate continuous supervision during online training; no quantitative bounds on acceptable supervision noise, bias, or required convergence time before exteroception removal are provided, leaving the generalization guarantee unverified.

    Authors: The manuscript presents empirical evidence of the fallback mechanism's effectiveness through experiments across various platforms and scenarios. However, we agree that providing quantitative bounds would enhance the claims. In the revised version, we will add a subsection analyzing the impact of supervision noise and bias on learning convergence and odometry accuracy, including observed convergence times. revision: yes

  2. Referee: [Abstract and Experiments] The abstract states 'promising results' on different platforms but provides no error metrics, ablation studies, baseline comparisons, or derivation details for the velocity NN or InEKF; this absence undermines assessment of whether the elimination of calibration and kinematic modeling is achieved without inheriting supervision errors.

    Authors: Detailed error metrics, ablation studies, baseline comparisons, and derivation details are provided in the full manuscript's Experiments and Methods sections. To address the concern, we will update the abstract to include representative quantitative results highlighting the performance and the fact that ablations confirm the approach does not simply inherit supervision errors. revision: yes

Circularity Check

0 steps flagged

No circularity; supervision is external and derivation is self-contained

full rationale

The paper presents an online learning method that uses established exteroceptive motion pipelines as an external supervisory signal to train a proprioceptive velocity network, followed by InEKF fusion and fallback. No equations, fitted parameters, or self-citations appear in the abstract or description that reduce any claimed result to its own inputs by construction. The approach is a standard supervised learning setup with an independent assumption on supervisor accuracy during training periods; this does not constitute circularity under the defined patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no free parameters, axioms, or invented entities are specified in the text.

pith-pipeline@v0.9.1-grok · 5732 in / 1289 out tokens · 26906 ms · 2026-06-26T14:08:12.288994+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 16 canonical work pages

  1. [1]

    Z. Wu, K. Zheng, Z. Ding, and H. Gao. A survey on legged robots: Advances, tech- nologies and applications.Engineering Applications of Artificial Intelligence, 138:109418,

  2. [2]

    doi:https://doi.org/10.1016/j.engappai.2024.109418

    ISSN 0952-1976. doi:https://doi.org/10.1016/j.engappai.2024.109418. URLhttps: //www.sciencedirect.com/science/article/pii/S0952197624015768

    work page doi:10.1016/j.engappai.2024.109418 1976

  3. [3]

    Di Carlo, P

    J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim. Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In2018 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), pages 1–9, 2018. doi:10.1109/IROS.2018. 8594448

    work page doi:10.1109/iros.2018 2018

  4. [4]

    Kumar, Z

    A. Kumar, Z. Fu, D. Pathak, and J. Malik. RMA: Rapid Motor Adaptation for Legged Robots. InProceedings of Robotics: Science and Systems, Virtual, July 2021. doi:10.15607/RSS.2021. XVII.011

    work page doi:10.15607/rss.2021 2021

  5. [5]

    Y . Peng, C. Chen, K. Wu, and G. Huang. sqrt-vins: Robust and ultrafast square-root filter- based 3d motion tracking.IEEE Transactions on Robotics (TRO), oct 2025. URLhttps: //ieeexplore.ieee.org/document/11219345

    arXiv 2025

  6. [6]

    Xu and F

    W. Xu and F. Zhang. Fast-lio: A fast, robust lidar-inertial odometry package by tightly-coupled iterated kalman filter.IEEE Robotics and Automation Letters, 6(2):3317–3324, 2021. doi: 10.1109/LRA.2021.3064227

    work page doi:10.1109/lra.2021.3064227 2021

  7. [7]

    Furgale, J

    P. Furgale, J. Rehder, and R. Siegwart. Unified temporal and spatial calibration for multi-sensor systems. In2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1280–1286. IEEE, 2013

    2013

  8. [8]

    Rehder, J

    J. Rehder, J. Nikolic, T. Schneider, T. Hinzmann, and R. Siegwart. Extending kalibr: Cal- ibrating the extrinsics of multiple imus and of individual axes. In2016 IEEE international conference on robotics and automation (ICRA), pages 4304–4311. IEEE, 2016

    2016

  9. [9]

    Camurri, M

    M. Camurri, M. Ramezani, S. Nobili, and M. Fallon. Pronto: A multi-sensor state estimator for legged robots in real-world scenarios.Frontiers in Robotics and AI, 7:68, 2020

    2020

  10. [10]

    a fer, Andrew Wing Keung To, Kuan-Ho Lao, Murat Cubuktepe, Matthew Haley, Peter B \

    C. Burgul, W. Lee, P. Geneva, and G. Huang. Online determination of legged kinematics. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9043–9049, 2024. doi:10.1109/IROS58592.2024.10801595

    work page doi:10.1109/iros58592.2024.10801595 2024

  11. [11]

    Wisth, M

    D. Wisth, M. Camurri, and M. Fallon. VILENS: Visual, inertial, lidar, and leg odometry for all-terrain legged robots.IEEE Transactions on Robotics, 39(1):309–326, 2023. doi:10.1109/ TRO.2022.3193788

    arXiv 2023

  12. [12]

    In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp

    V . Dh´edin, H. Li, S. Khorshidi, L. Mack, A. K. C. Ravi, A. Meduri, P. Shah, F. Grimminger, L. Righetti, M. Khadiv, and J. Stueckler. Visual-inertial and leg odometry fusion for dynamic locomotion. InProceedings of the IEEE International Conference on Robotics and Automa- tion, pages 9966–9972, 2023. doi:10.1109/ICRA48891.2023.10160898

    work page doi:10.1109/icra48891.2023.10160898 2023

  13. [13]

    Hartley, M

    R. Hartley, M. Ghaffari, R. M. Eustice, and J. W. Grizzle. Contact-aided invariant extended kalman filtering for robot state estimation.The International Journal of Robotics Research, 39(4):402–430, 2020. doi:10.1177/0278364919894385. URLhttps://doi.org/10.1177/ 0278364919894385

    work page doi:10.1177/0278364919894385 2020

  14. [14]

    T.-Y . Lin, R. Zhang, J. Yu, and M. Ghaffari. Legged robot state estimation using invari- ant kalman filtering and learned contact events. InProceedings of the 5th Conference on Robot Learning, volume 164 ofProceedings of Machine Learning Research, pages 1057–

  15. [15]

    URLhttps://proceedings.mlr.press/v164/lin22b.html

    PMLR, 2022. URLhttps://proceedings.mlr.press/v164/lin22b.html. 9

    2022

  16. [16]

    Buchanan, M

    R. Buchanan, M. Camurri, F. Dellaert, and M. Fallon. Learning inertial odometry for dynamic legged robot state estimation. InProceedings of the 5th Conference on Robot Learning, volume 164 ofProceedings of Machine Learning Research, pages 1575–1584. PMLR, 2022. URL https://proceedings.mlr.press/v164/buchanan22a.html

    2022

  17. [17]

    Wasserman, A

    J. Wasserman, A. Agarwal, R. Jangir, G. Chowdhary, D. Pathak, and A. Gupta. Legolas: Deep leg-inertial odometry. InProceedings of the 8th Conference on Robot Learning, volume 270 ofProceedings of Machine Learning Research, pages 2928–2947. PMLR, 2025. URL https://proceedings.mlr.press/v270/wasserman25a.html

    2025

  18. [18]

    A. M. Kulkarni, I. Poulakakis, and G. Huang. Learning neural observer-predictor models for limb-level sampling-based locomotion planning, 2025. URLhttps://arxiv.org/abs/ 2510.22789

    arXiv 2025

  19. [19]

    Lesort, V

    T. Lesort, V . Lomonaco, A. Stoian, D. Maltoni, D. Filliat, and N. D ´ıaz-Rodr´ıguez. Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Inf. Fusion, 58(C):52–68, June 2020. ISSN 1566-2535. doi:10.1016/j.inffus.2019.12.004. URLhttps://doi.org/10.1016/j.inffus.2019.12.004

    work page doi:10.1016/j.inffus.2019.12.004 2020

  20. [20]

    Klanke, S

    S. Klanke, S. Vijayakumar, and S. Schaal. A library for locally weighted projection regres- sion.Journal of Machine Learning Research, 9(21):623–626, 2008. URLhttp://jmlr. org/papers/v9/klanke08a.html

    2008

  21. [21]

    Nguyen-tuong, J

    D. Nguyen-tuong, J. Peters, and M. Seeger. Local gaussian process regression for real time online model learning. In D. Koller, D. Schuurmans, Y . Bengio, and L. Bot- tou, editors,Advances in Neural Information Processing Systems, volume 21. Curran As- sociates, Inc., 2008. URLhttps://proceedings.neurips.cc/paper_files/paper/ 2008/file/01161aaa0b6d1345dd8...

    2008

  22. [22]

    T. Z. Jiahao, K. Y . Chee, and M. A. Hsieh. Online dynamics learning for predictive control with an application to aerial robots. In6th Annual Conference on Robot Learning, 2022. URL https://openreview.net/forum?id=8-8e18idYLD

    2022

  23. [23]

    Okawara, K

    T. Okawara, K. Koide, S. Oishi, M. Yokozuka, A. Banno, K. Uno, and K. Yoshida. Tightly- coupled lidar-imu-wheel odometry with an online neural kinematic model learning via factor graph optimization.Robotics and Autonomous Systems, 187:104929, 2025. ISSN 0921-8890. doi:https://doi.org/10.1016/j.robot.2025.104929. URLhttps://www.sciencedirect.com/ science/a...

    work page doi:10.1016/j.robot.2025.104929 2025

  24. [24]

    Tanaka, T

    G. Tanaka, T. Yamane, J. B. H ´eroux, R. Nakane, N. Kanazawa, S. Takeda, H. Numata, D. Nakano, and A. Hirose. Recent advances in physical reservoir computing: A re- view.Neural Networks, 115:100–123, 2019. ISSN 0893-6080. doi:https://doi.org/10.1016/ j.neunet.2019.03.005. URLhttps://www.sciencedirect.com/science/article/pii/ S0893608019300784

    2019

  25. [25]

    Kim and A

    S. Kim and A. Billard. Estimating the non-linear dynamics of free-flying objects.Robotics and Autonomous Systems, 60(9):1108–1122, 2012. ISSN 0921-8890. doi:https://doi.org/10. 1016/j.robot.2012.05.022. URLhttps://www.sciencedirect.com/science/article/ pii/S092188901200084X

    2012

  26. [26]

    A. S. Polydoros and L. Nalpantidis. A reservoir computing approach for learning forward dy- namics of industrial manipulators. In2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 612–618, 2016. doi:10.1109/IROS.2016.7759116

    work page doi:10.1109/iros.2016.7759116 2016

  27. [27]

    Maddigan and T

    M. Folgheraiter, A. Yskak, and S. Yessirkepov. One-shot bipedal robot dynamics identification with a reservoir-based rnn.IEEE Access, 11:50180–50194, 2023. doi:10.1109/ACCESS.2023. 3277977. 10

    work page doi:10.1109/access.2023 2023

  28. [28]

    Luko ˇseviˇcius and H

    M. Luko ˇseviˇcius and H. Jaeger. Reservoir computing approaches to recurrent neural net- work training.Computer Science Review, 3(3):127–149, 2009. ISSN 1574-0137. doi:https:// doi.org/10.1016/j.cosrev.2009.03.005. URLhttps://www.sciencedirect.com/science/ article/pii/S1574013709000173

    work page doi:10.1016/j.cosrev.2009.03.005 2009

  29. [29]

    Singhal and L

    S. Singhal and L. Wu. Training multilayer perceptrons with the extended kalman al- gorithm. In D. Touretzky, editor,Advances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann, 1988. URLhttps://proceedings.neurips.cc/paper_ files/paper/1988/file/38b3eff8baf56627478ec76a704e9b52-Paper.pdf

    1988

  30. [30]

    Puskorius and L

    G. Puskorius and L. Feldkamp. Decoupled extended kalman filter training of feedforward layered networks. InIJCNN-91-Seattle International Joint Conference on Neural Networks, volume i, pages 771–777 vol.1, 1991. doi:10.1109/IJCNN.1991.155276

    work page doi:10.1109/ijcnn.1991.155276 1991

  31. [31]

    P. G. Chang, G. Dur ´an-Mart´ın, A. Shestopaloff, M. Jones, and K. P. Murphy. Low-rank ex- tended kalman filtering for online learning of neural networks from streaming data. InConfer- ence on Lifelong Learning Agents, pages 1025–1071. PMLR, 2023

    2023

  32. [32]

    C. Sun, M. Song, D. Cai, B. Zhang, S. Hong, and H. Li. A systematic review of echo state networks from design to application.IEEE Transactions on Artificial Intelligence, 5(1):23–37,

  33. [33]

    doi:10.1109/TAI.2022.3225780

    work page doi:10.1109/tai.2022.3225780 2022

  34. [34]

    A. Hart, J. Hook, and J. Dawes. Embedding and approximation theorems for echo state net- works.Neural Networks, 128:234–247, 2020. ISSN 0893-6080. doi:https://doi.org/10.1016/ j.neunet.2020.05.013. URLhttps://www.sciencedirect.com/science/article/pii/ S0893608020301830

    2020

  35. [35]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Te- jani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learning library. InAdvances in Neural Information Processing Sy...

    2019

  36. [36]

    K. Wang, F. Cuzzolin, K. Shariatmadar, D. Moens, and H. Hallez. A Review of Uncer- tainty Representation and Quantification in Neural Networks .IEEE Transactions on Pat- tern Analysis & Machine Intelligence, 48(03):2476–2495, Mar. 2026. ISSN 1939-3539. doi:10.1109/TPAMI.2025.3626645. URLhttps://doi.ieeecomputersociety.org/10. 1109/TPAMI.2025.3626645

    work page doi:10.1109/tpami.2025.3626645 2026

  37. [37]

    H. Song, M. Kim, D. Park, Y . Shin, and J.-G. Lee. Learning from noisy labels with deep neural networks: A survey.IEEE Transactions on Neural Networks and Learning Systems, 2022

    2022

  38. [38]

    Rolnick, A

    D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne. Experience replay for con- tinual learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch ´e-Buc, E. Fox, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URLhttps://proceedings.neurips.cc/paper_files/paper/ 2019/file...

    2019

  39. [39]

    Lee, H.-B

    S. Lee, H.-B. Kim, and K.-S. Kim. Legged robot state estimation using invariant neural- augmented kalman filter with a neural compensator. In2025 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pages 15445–15452, 2025. doi:10.1109/ IROS60139.2025.11247668

    arXiv 2025

  40. [40]

    In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp

    B. Joshi, H. Damron, S. Rahman, and I. Rekleitis. Sm/vio: Robust underwater state es- timation switching between model-based and visual inertial odometry. In2023 IEEE In- ternational Conference on Robotics and Automation (ICRA), pages 5192–5199, 2023. doi: 10.1109/ICRA48891.2023.10161407. 11

    work page doi:10.1109/icra48891.2023.10161407 2023

  41. [41]

    C. Noh, S. Jung, H. Kim, Y . Hu, L. Herlant, and A. Kim. Garlileo: Gravity-aligned radar-leg- inertial enhanced odometry, 2025. URLhttps://arxiv.org/abs/2511.13216. Appendices A Switching Manager The system health monitor computes a normalized health scoreHP r0,1sas a weighted combina- tion of MSCKF feature quality, SLAM feature quality, and covariance qu...

    arXiv 2025