CoCo-InEKF: State Estimation with Learned Contact Covariances in Dynamic, Contact-Rich Scenarios
Pith reviewed 2026-06-30 20:02 UTC · model grok-4.3
The pith
CoCo-InEKF replaces binary contact states with learned continuous covariances in an invariant EKF for legged robot velocity estimation.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By training a neural network to output contact velocity covariances for predefined candidate points and integrating them into an invariant EKF, the filter dynamically adjusts for nuanced contact conditions ranging from firm contact to slippage or none, yielding superior accuracy-efficiency tradeoffs and consistency for linear velocity estimation without binary decisions or manual labels.
What carries the argument
Differentiable invariant extended Kalman filter modulated by continuous contact velocity covariances predicted by a lightweight neural network trained end-to-end on state-error loss.
If this is right
- Enables robust execution of challenging motions such as dancing and complex ground interactions in simulation and the real world.
- Delivers a superior accuracy-efficiency tradeoff for linear velocity estimation compared to baseline methods.
- Improves overall filter consistency relative to traditional binary contact approaches.
- Eliminates dependence on heuristic ground-truth contact labels for training.
- Remains insensitive to the precise placement of contact candidate points.
Where Pith is reading between the lines
- The covariance prediction could extend to other legged platforms by adapting the automated candidate selection procedure.
- The uncertainty information from covariances might integrate into motion planning for safer navigation in uncertain terrain.
- Collecting diverse real-world interaction data could further reduce simulation-to-reality gaps in training.
- Onboard efficiency gains may support higher-frequency control loops in high-speed locomotion tasks.
Load-bearing premise
A lightweight neural network trained end-to-end on state-error loss can produce contact velocity covariances that correctly capture nuanced contact conditions and improve the invariant EKF without introducing instability.
What would settle it
A controlled test showing higher velocity estimation error or filter divergence when using the learned covariances versus binary contact baselines during a partial-contact or slippage scenario.
Figures
read the original abstract
Robust state estimation for highly dynamic motion of legged robots remains challenging, especially in dynamic, contact-rich scenarios. Traditional approaches often rely on binary contact states that fail to capture the nuances of partial contact or directional slippage. This paper presents CoCo-InEKF, a differentiable invariant extended Kalman filter that utilizes continuous contact velocity covariances instead of binary contact states. These learned covariances allow the method to dynamically modulate contact confidence, accounting for more nuanced conditions ranging from firm contact to directional slippage or no contact. To predict these covariances for a set of predefined contact candidate points, we employ a lightweight neural network trained end-to-end using a state-error loss. This approach eliminates the need for heuristic ground-truth contact labels. In addition, we propose an automated contact candidate selection procedure and demonstrate that our method is insensitive to their exact placement. Experiments on a bipedal robot demonstrate a superior accuracy-efficiency tradeoff for linear velocity estimation, as well as improved filter consistency compared to baseline methods. This enables the robust execution of challenging motions, including dancing and complex ground interactions -- both in simulation and in the real world.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents CoCo-InEKF, a differentiable invariant extended Kalman filter (InEKF) for state estimation in legged robots that replaces binary contact states with continuous contact velocity covariances predicted by a lightweight neural network. The network is trained end-to-end on a state-error loss without requiring heuristic contact labels. An automated contact candidate selection method is proposed, and the approach is demonstrated to be robust to candidate placement. Experiments on a bipedal robot in simulation and hardware show improved accuracy and consistency in linear velocity estimation over baselines, supporting complex motions such as dancing and ground interactions.
Significance. This work addresses a significant challenge in robust state estimation for dynamic, contact-rich legged locomotion by learning to modulate contact confidence continuously. If the empirical results are reproducible, it offers an efficient way to integrate data-driven components into invariant filters. The manuscript earns credit for its end-to-end training procedure that avoids labeled data, the ablation studies on contact candidate placement and training, and the real-world experiments validating performance on challenging tasks.
minor comments (2)
- [Abstract] Abstract: the claim of a 'superior accuracy-efficiency tradeoff' would be strengthened by including one or two key quantitative metrics (e.g., velocity RMSE reduction or consistency metric) rather than qualitative description alone.
- The description of how the neural network outputs are mapped to the covariance matrices in the InEKF update equations would benefit from an explicit equation or pseudocode block to improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. No specific major comments were listed in the report.
Circularity Check
No significant circularity in derivation chain
full rationale
The paper presents a differentiable InEKF that incorporates continuous contact velocity covariances produced by a lightweight NN. The NN is trained end-to-end on a state-error loss to output those covariances; this is a conventional learning procedure that maps sensor inputs to filter parameters, not a derivation that reduces by construction to its own fitted values or to a self-citation. No equations are shown that equate a claimed prediction to a fitted input, no uniqueness theorem is invoked via self-citation, and no ansatz is smuggled. The central claims rest on simulation/hardware experiments and ablations that directly test accuracy, consistency, and robustness; these are externally falsifiable and independent of the training loop itself. The approach therefore remains self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (1)
- neural network weights
axioms (1)
- domain assumption State-error loss is sufficient to train contact covariances without heuristic ground-truth contact labels
Forward citations
Cited by 2 Pith papers
-
Learning Contact Representation for Leg Odometry
Self-supervised contact detection for legged robot odometry using joint encoders outperforms supervised methods that require force sensors.
-
Constrained Whole-Body Tracking for Humanoid Robots
ConstrainedMimic integrates operational space control and control barrier functions into RL tracking policies to enforce arbitrary runtime constraints on humanoid kinematics and dynamics while preserving contact modes...
Reference graph
Works this paper leans on
-
[1]
Rong Li, and Thiagalingam Kirubarajan.Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software
Yaakov Bar-Shalom, X. Rong Li, and Thiagalingam Kirubarajan.Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software. John Wiley & Sons, New York, 2001. ISBN 978-0-471-41655-5
2001
-
[2]
Contact model fusion for event-based locomotion in unstructured terrains
Gerardo Bledt, Patrick M Wensing, Sam Ingersoll, and Sangbae Kim. Contact model fusion for event-based locomotion in unstructured terrains. In2018 IEEE International Conference on Robotics and Automation (ICRA), pages 4399–4406, 2018
2018
-
[3]
David Remy, and Roland Siegwart
Michael Bloesch, Marco Hutter, Mark Hoepflinger, Stefan Leutenegger, Christian Gehring, C. David Remy, and Roland Siegwart. State Estimation for Legged Robots - Consistent Fusion of Leg Kinematics and IMU. In Proceedings of Robotics: Science and Systems, 2012
2012
-
[4]
State estimation for legged robots on unstable and slippery terrain
Michael Bloesch, Christian Gehring, P ´eter Fankhauser, Marco Hutter, Mark A Hoepflinger, and Roland Siegwart. State estimation for legged robots on unstable and slippery terrain. In2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 6058–6064, 2013
2013
-
[5]
AI- IMU dead-reckoning.IEEE Transactions on Intelligent Vehicles, 5(4):585–595, 2020
Martin Brossard, Axel Barrau, and Silv `ere Bonnabel. AI- IMU dead-reckoning.IEEE Transactions on Intelligent Vehicles, 5(4):585–595, 2020
2020
-
[6]
Denoising IMU gyroscopes with deep learning for open- loop attitude estimation.IEEE Robotics and Automation Letters, 5(3):4796–4803, 2020
Martin Brossard, Silvere Bonnabel, and Axel Barrau. Denoising IMU gyroscopes with deep learning for open- loop attitude estimation.IEEE Robotics and Automation Letters, 5(3):4796–4803, 2020
2020
-
[7]
Learning Inertial Odometry for Dynamic Legged Robot State Estimation
Russell Buchanan, Marco Camurri, Frank Dellaert, and Maurice Fallon. Learning Inertial Odometry for Dynamic Legged Robot State Estimation. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors,Proceedings of the 5th Conference on Robot Learning, volume 164 ofProceedings of Machine Learning Research, pages 1575–1584. PMLR, 2022
2022
-
[8]
Caldwell, and Claudio Semini
Marco Camurri, Maurice Fallon, St ´ephane Bazeille, An- dreea Radulescu, Victor Barasuol, Darwin G. Caldwell, and Claudio Semini. Probabilistic Contact Estimation and Impact Detection for State Estimation of Quadruped Robots.IEEE Robotics and Automation Letters, 2(2): 1023–1030, 2017
2017
-
[9]
Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios.Frontiers in Robotics and AI, 7, 2020
Marco Camurri, Milad Ramezani, Simona Nobili, and Maurice Fallon. Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios.Frontiers in Robotics and AI, 7, 2020. Publisher: Frontiers
2020
-
[10]
IONet: learning to cure the curse of drift in inertial odometry
Changhao Chen, Xiaoxuan Lu, Andrew Markham, and Niki Trigoni. IONet: learning to cure the curse of drift in inertial odometry. InProceedings of the Thirty-Second AAAI Conference on Artificial Intelli- gence and Thirtieth Innovative Applications of Artifi- cial Intelligence Conference and Eighth AAAI Sympo- sium on Educational Advances in Artificial Intell...
2018
-
[11]
Learned inertial odometry for autonomous drone racing.IEEE Robotics and Automation Letters, 8(5):2684–2691, 2023
Giovanni Cioffi, Leonard Bauersfeld, Elia Kaufmann, and Davide Scaramuzza. Learned inertial odometry for autonomous drone racing.IEEE Robotics and Automation Letters, 8(5):2684–2691, 2023
2023
-
[12]
Proprioceptive Sensor Fusion for Quadruped Robot State Estimation
Geoff Fink and Claudio Semini. Proprioceptive Sensor Fusion for Quadruped Robot State Estimation. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10914–10920, 2020
2020
-
[13]
Enhancing Leg Odometry in Legged Robots with Learned Contact Bias: An LSTM Recurrent Neural Network Approach
Yaru Gu, Ze Liu, and Ting Zou. Enhancing Leg Odometry in Legged Robots with Learned Contact Bias: An LSTM Recurrent Neural Network Approach. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6832–6839, 2024
2024
-
[14]
Backprop KF: learning discriminative deterministic state estimators
Tuomas Haarnoja, Anurag Ajay, Sergey Levine, and Pieter Abbeel. Backprop KF: learning discriminative deterministic state estimators. InProceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 4383–4391. Curran Associates Inc., 2016
2016
-
[15]
Legged robot state-estimation through combined forward kinematic and preintegrated contact factors
Ross Hartley, Josh Mangelson, Lu Gan, Maani Ghaffari Jadidi, Jeffrey M Walls, Ryan M Eustice, and Jessy W Grizzle. Legged robot state-estimation through combined forward kinematic and preintegrated contact factors. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 4422–4429, 2018
2018
-
[16]
Contact-aided invariant extended Kalman filtering for robot state estimation.The Inter- national Journal of Robotics Research, 39(4):402–430,
Ross Hartley, Maani Ghaffari, Ryan M Eustice, and Jessy W Grizzle. Contact-aided invariant extended Kalman filtering for robot state estimation.The Inter- national Journal of Robotics Research, 39(4):402–430,
-
[17]
Publisher: SAGE Publications Ltd STM
-
[18]
Ronin: Robust neural inertial navigation in the wild: Benchmark, evaluations, & new methods
Sachini Herath, Hang Yan, and Yasutaka Furukawa. Ronin: Robust neural inertial navigation in the wild: Benchmark, evaluations, & new methods. In2020 IEEE international conference on robotics and automation (ICRA), pages 3146–3152, 2020
2020
-
[19]
InEKFormer: A Hybrid State Estimator for Humanoid Robots
Lasse Hohmeyer, Mihaela Popescu, Ivan Bergonzani, Dennis Mronga, and Frank Kirchner. InEKFormer: A Hybrid State Estimator for Humanoid Robots. In2025 IEEE International Conference on Advanced Robotics (ICAR), pages 833–840, 2025
2025
-
[20]
Probabilistic foot contact estimation by fusing information from dynamics and differential/forward kinematics
Jemin Hwangbo, Carmine Dario Bellicoso, P ´eter Fankhauser, and Marco Hutter. Probabilistic foot contact estimation by fusing information from dynamics and differential/forward kinematics. In2016 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS), pages 3872–3878, 2016
2016
-
[21]
Dynamic locomo- tion on slippery ground.IEEE Robotics and Automation Letters, 4(4):4170–4176, 2019
Fabian Jenelten, Jemin Hwangbo, Fabian Tresoldi, C Dario Bellicoso, and Marco Hutter. Dynamic locomo- tion on slippery ground.IEEE Robotics and Automation Letters, 4(4):4170–4176, 2019
2019
-
[22]
Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion.IEEE Robotics and Automation Letters, 7 (2):4630–4637, 2022
Gwanghyeon Ji, Juhyeok Mun, Hyeongjun Kim, and Jemin Hwangbo. Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion.IEEE Robotics and Automation Letters, 7 (2):4630–4637, 2022
2022
-
[23]
Legged Robot State Estimation With Dynamic Con- tact Event Information.IEEE Robotics and Automation Letters, 6(4):6733–6740, 2021
Joon-Ha Kim, Seungwoo Hong, Gwanghyeon Ji, Se- unghun Jeon, Jemin Hwangbo, Jun-Ho Oh, and Hae-Won Park. Legged Robot State Estimation With Dynamic Con- tact Event Information.IEEE Robotics and Automation Letters, 6(4):6733–6740, 2021
2021
-
[24]
Adaptive Invariant Extended Kalman Filter for Legged Robot State Esti- mation
Kyung-Hwan Kim, DongHyun Ahn, Dong-hyun Lee, JuYoung Yoon, and Dong Jin Hyun. Adaptive Invariant Extended Kalman Filter for Legged Robot State Esti- mation. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3063–3068, 2025
2025
-
[25]
Multimodal sensor fusion with differentiable filters
Michelle A Lee, Brent Yi, Roberto Mart ´ın-Mart´ın, Silvio Savarese, and Jeannette Bohg. Multimodal sensor fusion with differentiable filters. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10444–10451, 2020
2020
-
[26]
Legged Robot State Estimation Using Invariant Neural- Augmented Kalman Filter with a Neural Compensator
Seokju Lee, Hyun-Bin Kim, and Kyung-Soo Kim. Legged Robot State Estimation Using Invariant Neural- Augmented Kalman Filter with a Neural Compensator. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 15445–15452, 2025
2025
-
[27]
Legged Robot State Estimation using Invariant Kalman Filtering and Learned Contact Events
Tzu-Yuan Lin, Ray Zhang, Justin Yu, and Maani Ghaffari. Legged Robot State Estimation using Invariant Kalman Filtering and Learned Contact Events. InConference on Robot Learning, pages 1057–1066. PMLR, 2022
2022
-
[28]
Proprioceptive Invariant Robot State Estimation,
Tzu-Yuan Lin, Tingjun Li, Wenzhe Tong, and Maani Ghaffari. Proprioceptive Invariant Robot State Estimation,
- [29]
-
[30]
Debiasing 6-DOF IMU via Hierarchical Learning of Continuous Bias Dynamics
Ben Liu, Tzu-Yuan Lin, Wei Zhang, and Maani Ghaffari. Debiasing 6-DOF IMU via Hierarchical Learning of Continuous Bias Dynamics. InProceedings of Robotics: Science and Systems, 2025
2025
-
[31]
Tlio: Tight learned inertial odometry.IEEE Robotics and Automation Letters, 5(4):5653–5660, 2020
Wenxin Liu, David Caruso, Eddy Ilg, Jing Dong, Anas- tasios I Mourikis, Kostas Daniilidis, Vijay Kumar, and Jakob Engel. Tlio: Tight learned inertial odometry.IEEE Robotics and Automation Letters, 5(4):5653–5660, 2020
2020
-
[32]
α-MDF: An Attention-based Multimodal Differen- tiable Filter for Robot State Estimation
Xiao Liu, Yifan Zhou, Shuhei Ikemoto, and Heni Ben Amor. α-MDF: An Attention-based Multimodal Differen- tiable Filter for Robot State Estimation. InProceedings of The 7th Conference on Robot Learning, volume 229 ofProceedings of Machine Learning Research, pages 3870–3893. PMLR, 2023
2023
-
[33]
Piga, Ugo Pattacini, and Lorenzo Natale
Nicola A. Piga, Ugo Pattacini, and Lorenzo Natale. A Differentiable Extended Kalman Filter for Object Tracking Under Sliding Regime.Frontiers in Robotics and AI, 8, 2021. Publisher: Frontiers
2021
-
[34]
3d animation and 2d cartoons made sim- ple., 2023
Reallusion. 3d animation and 2d cartoons made sim- ple., 2023. URL https://www.reallusion.com. studio- mocap-girl-dance, studio-mocap-evolution-of-dance-vol- 1, studio-mocap-evolution-of-dance-vol-2, iclone-motion- pack—street-dance-locking
2023
-
[35]
Guy Revach, Nir Shlezinger, Xiaoyong Ni, Adri `a L ´opez Escoriza, Ruud J. G. van Sloun, and Yonina C. Eldar. KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics.IEEE Transactions on Signal Processing, 70:1532–1547, 2022
2022
-
[36]
VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters
Agon Serifi, Ruben Grandia, Espen Knoop, Markus Gross, and Moritz B ¨acher. VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters. Computer Graphics Forum, 43(8):e15175, 2024
2024
-
[37]
Robot Crash Course: Learning Soft and Stylized Falling,
Pascal Strauch, David M ¨uller, Sammy Christen, Agon Serifi, Ruben Grandia, Espen Knoop, and Moritz B ¨acher. Robot Crash Course: Learning Soft and Stylized Falling,
- [38]
-
[39]
Proprioceptive slip detection and state estimation of multi-legged robots in slippery scenarios
Peng Sun, Qi Li, Hao Hu, Junjie Qiang, Weiwei Wu, and Xin Luo. Proprioceptive slip detection and state estimation of multi-legged robots in slippery scenarios. Frontiers of Mechanical Engineering, 20(5):36, 2025
2025
-
[40]
Valenti, Ivan Dryanovski, and Jizhong Xiao
Roberto G. Valenti, Ivan Dryanovski, and Jizhong Xiao. Keeping a Good Attitude: A Quaternion-Based Orien- tation Filter for IMUs and MARGs.Sensors, 15(8): 19302–19330, 2015. Publisher: Multidisciplinary Digital Publishing Institute
2015
-
[41]
Legged Robot State Estimation with Invariant Extended Kalman Filter Using Neural Measurement Network
Donghoon Youm, Hyunsik Oh, Suyoung Choi, Hyeongjun Kim, Seunghun Jeon, and Jemin Hwangbo. Legged Robot State Estimation with Invariant Extended Kalman Filter Using Neural Measurement Network. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 670–676, 2025
2025
-
[42]
State estimation transformers for agile legged locomotion
Chen Yu, Yichu Yang, Tianlin Liu, Yangwei You, Mingliang Zhou, and Diyun Xiang. State estimation transformers for agile legged locomotion. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6810–6817, 2024
2024
-
[43]
IMU data processing for inertial aided navigation: A recurrent neural network based approach
Ming Zhang, Mingming Zhang, Yiming Chen, and Mingyang Li. IMU data processing for inertial aided navigation: A recurrent neural network based approach. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 3992–3998, 2021
2021
-
[44]
Robust State Estimation for Legged Robots With Dual Beta Kalman Filter.IEEE Robotics and Automation Letters, 10(8):7955–7962, 2025
Tianyi Zhang, Wenhan Cao, Chang Liu, Tao Zhang, Jiang- tao Li, and Shengbo Eben Li. Robust State Estimation for Legged Robots With Dual Beta Kalman Filter.IEEE Robotics and Automation Letters, 10(8):7955–7962, 2025
2025
-
[45]
A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry
Zichao Zhang and Davide Scaramuzza. A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry. In2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7244– 7251, 2018
2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.