pith. sign in

arxiv: 2407.05717 · v12 · submitted 2024-07-08 · 📡 eess.SY · cs.RO· cs.SY· eess.SP

Mitigating Overconfidence in Nonlinear Kalman Filters via Covariance Recalibration

Shida Jiang , Junzhe Shi , Scott Moura This is my paper

Pith reviewed 2026-05-23 23:07 UTC · model grok-4.3

classification 📡 eess.SY cs.ROcs.SYeess.SP
keywords nonlinear Kalman filtercovariance estimationoverconfidencestate estimationextended Kalman filterunscented Kalman filtermeasurement model
0
0 comments X

The pith

Nonlinear measurements cause the standard Kalman filter update to underestimate true posterior covariance.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proves that nonlinear Kalman filters retain the linear update equations even when measurements are nonlinear, and that this choice produces a systematic underestimation of posterior covariance. A sympathetic reader would care because overconfident covariance estimates can lead filters to trust bad measurements and diverge. The authors supply the first general mathematical demonstration of the bias. They then introduce a recalibration step that re-linearizes the measurement model after the state update so the reported covariance better reflects the actual Kalman gain effect; harmful updates can be rejected outright. Simulations on four common nonlinear filters and five applications show large gains in both state and covariance accuracy.

Core claim

Under nonlinear measurements the conventional Kalman filter update framework inherently tends to underestimate the true posterior covariance. The covariance-recalibrated framework re-approximates the measurement model after the state update to capture the actual effect of the Kalman gain on the posterior covariance; when the recalibration indicates that an update is harmful the update is withdrawn. The framework combines with any existing nonlinear KF and improves state and covariance estimation accuracy.

What carries the argument

The covariance recalibration procedure that re-approximates the measurement model after the state update to better capture the actual effect of the Kalman gain.

If this is right

  • The recalibrated framework applies unchanged to the extended, unscented, and cubature Kalman filters.
  • An update is withdrawn when recalibration shows it would increase covariance error.
  • Both state estimates and covariance estimates improve when the framework is added to existing nonlinear filters.
  • Error reductions of several orders of magnitude occur in the tested applications.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same underestimation mechanism may help explain observed divergence of nonlinear filters on strongly nonlinear problems.
  • A similar post-update recalibration step could be tested on particle filters and other sequential Bayesian estimators.
  • Direct comparison of reported covariances against empirical covariances obtained from repeated Monte Carlo runs on standard nonlinear benchmarks would provide an immediate empirical check.

Load-bearing premise

The conventional Kalman filter update equations can be applied directly to nonlinear measurement models without introducing systematic bias into the reported posterior covariance.

What would settle it

Compute the true posterior covariance by Monte Carlo sampling on a nonlinear measurement model and show that it is larger than the covariance returned by an unmodified nonlinear KF on the same data.

Figures

Figures reproduced from arXiv: 2407.05717 by Junzhe Shi, Scott Moura, Shida Jiang.

Figure 1
Figure 1. Figure 1: The basic idea of the new framework for nonlinear [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The comparison of the covariance estimation after the [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The root mean squared error of the state estimations [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 6
Figure 6. Figure 6: The root mean squared error of the state estimations [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The root mean squared error of the state estimations [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The actual and estimated root mean squared error of [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The root mean squared error of the state estimations [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of the computational time of different [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
read the original abstract

The Kalman filter (KF) is an optimal linear state estimator for linear systems, and numerous extensions, including the extended Kalman filter (EKF), unscented Kalman filter (UKF), and cubature Kalman filter (CKF), have been developed for nonlinear systems. Although these nonlinear KFs differ in how they approximate nonlinear transformations, they all retain the same update framework as the linear KF. In this paper, we show that, under nonlinear measurements, this conventional framework inherently tends to underestimate the true posterior covariance, leading to overconfident covariance estimates. To the best of our knowledge, this is the first work to provide a mathematical proof of this systematic covariance underestimation in a general nonlinear KF framework. Motivated by this analysis, we propose a covariance-recalibrated framework that re-approximates the measurement model after the state update to better capture the actual effect of the Kalman gain on the posterior covariance; when recalibration indicates that an update is harmful, the update can be withdrawn. The proposed framework can be combined with essentially any existing nonlinear KF, and simulations across four nonlinear KFs and five applications show that it substantially improves both state and covariance estimation accuracy, often reducing errors by several orders of magnitude. The code and supplementary material are available at https://github.com/Shida-Jiang/A-new-framework-for-nonlinear-Kalman-filters.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript claims that the standard Kalman update equations retained by nonlinear filters (EKF, UKF, CKF, etc.) systematically underestimate the true posterior covariance whenever the measurement function is nonlinear, provides a mathematical proof of this underestimation, and introduces a covariance-recalibration procedure that re-approximates the measurement model after the state update (optionally withdrawing the update). The recalibrated framework is asserted to be compatible with any existing nonlinear KF; simulations across four filters and five applications are reported to yield substantial gains in both state and covariance accuracy.

Significance. A general, assumption-light proof that the conventional update produces overconfident covariances under nonlinearity, together with a practical fix and open code, would be a meaningful contribution to nonlinear filtering. The empirical claim of order-of-magnitude error reductions, if supported by proper baselines and statistics, would strengthen the practical case.

major comments (2)
  1. [Proof of underestimation (likely §3 or §4)] The central claim of a 'mathematical proof of this systematic covariance underestimation in a general nonlinear KF framework' requires an explicit statement of the function class for which the inequality holds. For arbitrary nonlinear h(·) the true posterior covariance is obtained by integrating a non-Gaussian likelihood against the prior and does not admit a closed-form comparison to the KF update; any strict underestimation result must invoke additional structure (convexity, bounded Hessian, local linearity, etc.). The manuscript must identify this structure and verify that it is not tacitly assumed.
  2. [Simulation results (likely §5)] The simulation section reports 'often reducing errors by several orders of magnitude' across four filters and five applications, yet the abstract and available description give no indication of error bars, statistical significance tests, or explicit baseline comparisons (e.g., against existing covariance-inflation or robust KF variants). Without these, the magnitude of the reported gains cannot be evaluated and the cross-application claim is not yet secured.
minor comments (1)
  1. [Notation and framework definition] Notation for the recalibrated quantities (e.g., P_r^+, K_r) should be introduced once and used consistently; currently the distinction between original and recalibrated variables is occasionally ambiguous.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. The comments help clarify the scope of the theoretical result and strengthen the empirical evaluation. We address each major comment below.

read point-by-point responses

  1. Referee: [Proof of underestimation (likely §3 or §4)] The central claim of a 'mathematical proof of this systematic covariance underestimation in a general nonlinear KF framework' requires an explicit statement of the function class for which the inequality holds. For arbitrary nonlinear h(·) the true posterior covariance is obtained by integrating a non-Gaussian likelihood against the prior and does not admit a closed-form comparison to the KF update; any strict underestimation result must invoke additional structure (convexity, bounded Hessian, local linearity, etc.). The manuscript must identify this structure and verify that it is not tacitly assumed.

    Authors: We agree that a strict inequality for completely arbitrary nonlinear h(·) cannot be stated in closed form without additional structure, as the true posterior is generally non-Gaussian. Our derivation is performed inside the standard Gaussian approximation framework employed by nonlinear KFs (EKF linearization, UKF/CKF sigma-point moment matching). Under these approximations the conventional update produces a covariance that is smaller than the covariance implied by the same approximated measurement model after the gain is applied. We will revise §3 to explicitly state the function class (C² functions whose Hessian is bounded in a neighborhood of the operating point) and add a short verification paragraph confirming these conditions are the same ones already required for the consistency of EKF/UKF/CKF. A clarifying sentence will also be added that the claimed underestimation holds relative to the filter’s own Gaussian approximation rather than the exact Bayesian posterior. revision: yes

  2. Referee: [Simulation results (likely §5)] The simulation section reports 'often reducing errors by several orders of magnitude' across four filters and five applications, yet the abstract and available description give no indication of error bars, statistical significance tests, or explicit baseline comparisons (e.g., against existing covariance-inflation or robust KF variants). Without these, the magnitude of the reported gains cannot be evaluated and the cross-application claim is not yet secured.

    Authors: We accept that the current presentation of the numerical results lacks the statistical detail needed to substantiate the magnitude of the improvements. In the revised manuscript we will (i) report mean and standard deviation over 100 Monte-Carlo runs for every metric, (ii) include paired statistical tests (Wilcoxon signed-rank) against the baseline filters, and (iii) add direct comparisons with two representative covariance-inflation schemes and one robust KF variant. These additions will be placed in §5 and summarized in the abstract. revision: yes

Circularity Check

0 steps flagged

No significant circularity; proof and recalibration are independently derived

full rationale

The paper's central chain consists of (1) a claimed mathematical proof that the standard KF update underestimates posterior covariance for nonlinear measurements, and (2) a recalibration procedure motivated by that analysis. No equations or steps reduce by construction to fitted parameters, self-definitions, or self-citations. The recalibration re-approximates the measurement model post-update and is not statistically forced by the underestimation proof itself. The provided abstract and reader summary contain no load-bearing self-citations or ansatz smuggling. This is the common case of an independent derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that the linear KF update equations apply unchanged to nonlinear cases; the paper introduces a new recalibration framework whose validity rests on the simulations described.

axioms (1)
  • domain assumption Nonlinear Kalman filters retain the same update framework as the linear KF
    Explicitly stated in the abstract as the retained structure that produces the underestimation.
invented entities (1)
  • covariance-recalibrated framework no independent evidence
    purpose: Re-approximates the measurement model after the state update to capture the actual effect of the Kalman gain on posterior covariance
    Newly proposed method introduced to address the identified underestimation; no independent evidence outside the paper's simulations.

pith-pipeline@v0.9.0 · 5784 in / 1275 out tokens · 26776 ms · 2026-05-23T23:07:33.628039+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CAR-EnKF: A Covariance-Adaptive and Recalibrated Ensemble Kalman Filter Framework

    eess.SY 2026-04 unverdicted novelty 6.0

    CAR-EnKF reduces RMSE versus standard EnKF by recalibrating the Kalman gain effect and adding an online-tuned covariance compensation term that activates only under measurement nonlinearity.

  2. Assumed Density Filtering and Smoothing with Neural Network Surrogate Models

    eess.SY 2025-11 unverdicted novelty 5.0

    Assumed density filtering and smoothing for neural network surrogate models is enabled by analytic computation of output moments, yielding more accurate state estimates and improved LQR performance on stochastic Loren...

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages · cited by 2 Pith papers

  1. [1]

    Optimal filtering

    Brian DO Anderson and John B Moore. Optimal filtering. Courier Corporation, 2005

    work page 2005

  2. [2]

    Critical remarks on the linearised and extended kalman filters with geodetic navigation examples

    Dah-Jing Jwo and Ta-Shun Cho. Critical remarks on the linearised and extended kalman filters with geodetic navigation examples. Measurement, 43(9):1077–1089, 2010

    work page 2010

  3. [3]

    Disturbance observer and kalman filter based motion control realization

    Thao Tran Phuong, Kiyoshi Ohishi, Chowarit Mitsantisuk, Yuki Yokokura, Kouhei Ohnishi, Roberto Oboe, and Asif Sabanovic. Disturbance observer and kalman filter based motion control realization. IEEJ Journal of Industry Applications, 7(1):1–14, 2018

    work page 2018

  4. [4]

    [9] application of the kalman filter to computational problems in statistics

    Emery N Brown and Christopher H Schmid. [9] application of the kalman filter to computational problems in statistics. In Methods in enzymology, volume 240, pages 171–181. Elsevier, 1994

    work page 1994

  5. [5]

    The ensemble kalman filter: a signal processing perspective

    Michael Roth, Gustaf Hendeby, Carsten Fritsche, and Fredrik Gustafsson. The ensemble kalman filter: a signal processing perspective. EURASIP Journal on Advances in Signal Processing, 2017:1–16, 2017

    work page 2017

  6. [6]

    Nonlinear Filtering: Methods and Applications

    Kumar Pakki Bharani Chandra and Da-Wei Gu. Nonlinear Filtering: Methods and Applications . Springer, 2019

    work page 2019

  7. [7]

    Applied Optimal Estimation

    A Gelb. Applied Optimal Estimation . MIT Press, 1974

    work page 1974

  8. [8]

    New extension of the kalman filter to nonlinear systems

    Simon J Julier and Jeffrey K Uhlmann. New extension of the kalman filter to nonlinear systems. In Signal processing, sensor fusion, and target recognition VI, volume 3068, pages 182–193. Spie, 1997

    work page 1997

  9. [9]

    The unscented kalman filter for nonlinear estimation

    Eric A Wan and Rudolph Van Der Merwe. The unscented kalman filter for nonlinear estimation. In Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium (Cat. No. 00EX373), pages 153–158. Ieee, 2000

    work page 2000

  10. [10]

    Cubature kalman filters

    Ienkaran Arasaratnam and Simon Haykin. Cubature kalman filters. IEEE Transactions on automatic control, 54(6):1254– 1269, 2009. 11

    work page 2009

  11. [11]

    Discrete-time nonlinear filtering algorithms using gauss– hermite quadrature

    Ienkaran Arasaratnam, Simon Haykin, and Robert J Elliott. Discrete-time nonlinear filtering algorithms using gauss– hermite quadrature. Proceedings of the IEEE, 95(5):953–977, 2007

    work page 2007

  12. [12]

    Square-root quadrature kalman filtering

    Ienkaran Arasaratnam and Simon Haykin. Square-root quadrature kalman filtering. IEEE Transactions on Signal Processing, 56(6):2589–2593, 2008

    work page 2008

  13. [13]

    Some relations between extended and unscented kalman filters

    Fredrik Gustafsson and Gustaf Hendeby. Some relations between extended and unscented kalman filters. IEEE Transactions on Signal Processing, 60(2):545–555, 2011

    work page 2011

  14. [14]

    Analysis of kalman filter approximations for nonlinear measurements

    Mark R Morelande and Angel F Garcia-Fernandez. Analysis of kalman filter approximations for nonlinear measurements. IEEE Transactions on Signal Processing, 61(22):5477–5484, 2013

    work page 2013

  15. [15]

    Convergence analysis of nonlinear kalman filters with novel innovation-based method

    Shiyuan Wang, Wanli Wang, Badong Chen, and K Tse Chi. Convergence analysis of nonlinear kalman filters with novel innovation-based method. Neurocomputing, 289:188– 194, 2018

    work page 2018

  16. [16]

    Nonlinear filtering

    Kumar Pakki Bharani Chandra and Da-Wei Gu. Nonlinear filtering. Cham, Switzerland: Springer , 2019

    work page 2019

  17. [17]

    Why does enkf suffer from analysis overconfidence? an insight into exploiting the ever- increasing volume of observations

    Daisuke Hotta and Yoichiro Ota. Why does enkf suffer from analysis overconfidence? an insight into exploiting the ever- increasing volume of observations. Quarterly Journal of the Royal Meteorological Society, 147(735):1258–1277, 2021

    work page 2021

  18. [18]

    Performance evaluation of ukf-based nonlinear filtering

    Kai Xiong, HY Zhang, and CW Chan. Performance evaluation of ukf-based nonlinear filtering. Automatica, 42(2):261–270, 2006

    work page 2006

  19. [19]

    The iterated kalman filter update as a gauss-newton method

    Bradley M Bell and Frederick W Cathey. The iterated kalman filter update as a gauss-newton method. IEEE Transactions on Automatic Control, 38(2):294–297, 1993

    work page 1993

  20. [20]

    Iterated unscented kalman filter for passive target tracking

    Ronghui Zhan and Jianwei Wan. Iterated unscented kalman filter for passive target tracking. IEEE Transactions on Aerospace and Electronic Systems, 43(3):1155–1163, 2007

    work page 2007

  21. [21]

    Iterated cubature kalman filter and its application

    Jing Mu and Yuan-li Cai. Iterated cubature kalman filter and its application. In 2011 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems, pages 33–37. IEEE, 2011

    work page 2011

  22. [22]

    Performance evaluation of iterated extended kalman filter with variable step-length

    Jindˇ rich Havl´ ık and Ondˇ rej Straka. Performance evaluation of iterated extended kalman filter with variable step-length. In Journal of Physics: Conference Series , volume 659, page 012022. IOP Publishing, 2015

    work page 2015

  23. [23]

    Marginalised iterated unscented kalman filter

    Lubin Chang, Baiqing Hu, Guobin Chang, and An Li. Marginalised iterated unscented kalman filter. IET Control Theory & Applications, 6(6):847–854, 2012

    work page 2012

  24. [24]

    Alternative framework for the iterated unscented kalman filter

    Guobin Chang, Tianhe Xu, and Qianxin Wang. Alternative framework for the iterated unscented kalman filter. IET Signal Processing, 11(3):258–264, 2017

    work page 2017

  25. [25]

    On unscented kalman filtering for state estimation of continuous-time nonlinear systems

    Simo Sarkka. On unscented kalman filtering for state estimation of continuous-time nonlinear systems. IEEE Transactions on automatic control, 52(9):1631–1641, 2007

    work page 2007

  26. [26]

    Introduction to kalman filter and its applications

    Youngjoo Kim, Hyochoong Bang, et al. Introduction to kalman filter and its applications. Introduction and Implementations of the Kalman Filter , 1:1–16, 2018

    work page 2018

  27. [27]

    Online state estimation of a synchronous generator using unscented kalman filter from phasor measurements units

    Esmaeil Ghahremani and Innocent Kamwa. Online state estimation of a synchronous generator using unscented kalman filter from phasor measurements units. IEEE Transactions on Energy Conversion, 26(4):1099–1108, 2011

    work page 2011

  28. [28]

    x1,k x2,k # =

    Shida Jiang, Junzhe Shi, Manashita Borah, and Scott Moura. Weaknesses and improvements of the extended kalman filter for battery state-of-charge and state-of-health estimation. In 2024 American Control Conference (ACC), pages 1441–1448. IEEE, 2024. A Supplementary proof for Theorem 1 Lemma 3 Let (Ω, F , P) be an underlying probability space. Assume the ra...

    work page 2024