Pith. sign in

REVIEW 3 major objections 4 minor 28 references

Unobservable Systems: No Problem for Noise Identification

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read The paper shows that the measurement difference method can estimate state and measurement noise covariances in linear time-varying systems even when the state is unobservable and the input is unknown.

desk verdict Genuinely useful MDM generalization for unobservable and unknown-input LTV systems, but the consistency claim is under-supported and false as stated; a persistence-of-excitation condition is needed. read the letter →

arxiv 2505.23983 v2 pith:MWQ7THUB submitted 2025-05-29 eess.SP

classification eess.SP
keywords noisecovarianceidentificationmeasurementdifferencemethod(MDM)unobservablelinearsystemsunknowninputstime-varyingannihilationmatrixcorrelationmethodsKalmanfiltering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the noise covariance matrices of a linear time-varying stochastic state-space model can be estimated even when the model is not observable and the input sequence is unknown. The key move is to replace the pseudo-inverse of the observability matrix with an annihilation matrix that maps the augmented measurement vector to a residual depending only on the process and measurement noises. A least-squares fit of the residual covariances then recovers the parameters of the noise covariances, provided the stacked design matrix has full column rank. The paper argues that this generalized measurement difference method gives unbiased and consistent estimates, and demonstrates it on an unobservable clock-ensemble model and on an unobservable model with unknown input. If true, this removes two common restrictions that currently force practitioners to assume observability and known inputs before estimating noise covariances.

What carries the argument

The annihilation matrix $\mathrm{am}(O_k)$ is the central object: any matrix whose rows span a left null space of the observability matrix, so $\mathrm{am}(O_k)O_k = 0$. Multiplying the augmented measurement equation by this matrix eliminates the unknown state $x_k$ in one step, producing the residual $\tilde{eZ}_k$ that is exactly linear in the noises $W_k, V_k$. The paper builds the argument on this residual: its covariance is related to the sought noise covariances through the Kronecker-product identity $\mathcal{R}_{eZ_k^2} = \Xi_{eZ_k^2} A_k^{\otimes 2} C_k^{\otimes 2} \Upsilon_{E^2}\,\alpha$, and stacking over time yields a linear system solved by least squares. The structure-defining matrices $B_Q^{(i)}, B_R^{(i)}$ convert the unknown matrices $Q, R$ into a parameter vector $\alpha$, and the unification matrix $\Xi$ removes redundant entries. The same annihilation principle, applied to $[O_k, \Gamma_k G_k]$, extends the mechanism to unknown inputs.

What would settle it

Pick a fixed unobservable LTV model with known noise covariances and compute $A$ in (19) for increasing horizon $\tau$ at a given $L$; if a full-column-rank $A$ is found but the ordinary MDM estimate from simulated data does not converge to the true $\alpha$ as $\tau$ grows, the consistency claim is contradicted. Conversely, finding a rank-deficient $A$ for which simulation still gives unbiased estimates would show the stated condition is sufficient rather than necessary.

Watch

Extended reading notes

Core claim

The paper's central claim is that the measurement difference method, previously derived under observability and known-input assumptions, remains valid for unobservable linear time-varying systems and for systems with unknown inputs, provided the rank-deficient observability matrix is annihilated from the left rather than inverted. The residue $\tilde{eZ}_k = \mathrm{am}(O_k)(Z_k - \Gamma_k G_k U_k)$ is constructed so that $\mathrm{am}(O_k)O_k = 0$; because $O_k$ need not have full rank, the state term disappears regardless of observability. The residue is then a linear function of the stacked state and measurement noises, so its covariance is affine in the unknown noise covariance parameters $\alpha$ after parametrizing $Q$ and $R$ with structure-defining matrices. Stacking these relations over time gives $\mathcal{R}_{eZ^2} = A\alpha + L\eta$, and the ordinary least-squares estimate $\hat\alpha_o = (A^\top A)^{-1}A^\top \mathcal{R}_{eZ^2}$ is claimed to be unbiased and consistent whenever $A$ has full column rank. For unknown inputs, the annihilator is applied to $[O_k, \Gamma_k G_k]$ in place of $O_k$, which removes both the state and the input contributions at the cost of possibly fewer identifiable parameters.

Load-bearing premise

The load-bearing premise is that the stacked linear system in (19) has full column rank and that the sample averages of the residual products converge to their true covariances as the number of measurements grows; if either fails, the claimed unbiasedness and consistency of the ordinary MDM no longer follow.

Editorial extensions

If this is right

  • The ordinary MDM gives unbiased and consistent noise-covariance estimates for unobservable LTV systems as long as the stacked design matrix $A$ in (19) has full column rank.
  • Unknown input sequences can be accommodated by annihilating the joint state-input matrix; the price is a possible reduction in the number of identifiable noise-covariance parameters, with the extreme case $G_k = E_k$ making $Q$ unidentifiable.
  • The weighted MDM produces lower-variance estimates and a realistic estimate of its own covariance, at the cost of a small bias that vanishes as the data record grows.
  • The number of identifiable noise-covariance parameters for a given unobservable LTV model can be read off numerically from the rank of $A$, even though an analytical formula in terms of $n_x$, $n_{zk}$, and $\mathrm{rank}(O_k)$ remains open.
  • The method inherits prior MDM extensions, including correlated noises, noise mean and higher-moment identification, and sensor calibration with unknown state dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct practical consequence the paper does not spell out: because identifiability is decided by the numeric rank of $A$, a practitioner can pre-test a planned experiment design (horizon $L$ and time span $\tau$) before collecting data, and choose $L$ just large enough to make $A$ full rank.
  • The annihilation-matrix viewpoint suggests a broader class of identification residuals: any left annihilator of the state contribution yields a valid statistic, so one could optimize the annihilator to maximize Fisher information or inferential precision instead of taking any fixed null-space basis.
  • For non-Gaussian noises, the consistency claim relies on convergence of sample averages of fourth-order products; making the moment conditions explicit would give practitioners a testable check beyond the Gaussian examples shown in the paper.
  • For the clock-ensemble application, this implies that noise covariances can be calibrated from the same phase-difference measurements used for time-scale generation, without a separate observable reference.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper extends the measurement difference method (MDM) for estimating state and measurement noise covariance parameters in linear time-varying state-space models to two settings not covered by previous MDM versions: rank-deficient observability matrices and unknown input sequences. The key device is to replace the pseudo-inverse of the observability matrix with an annihilation matrix am(O_k), so the residue eZ_k becomes a linear function of the noises even when O_k is rank deficient; for unknown inputs the annihilator is applied to [O_k, Gamma_k G_k]. Noise covariances are parametrized through structure-defining matrices and unknown weights alpha, and ordinary and weighted least-squares estimators are derived. The paper claims unbiased and consistent estimates, provides a MATLAB implementation, and reports Monte Carlo studies for three examples including an unobservable clock-ensemble model.

Significance. If the consistency claim can be made rigorous, the paper is a useful and well-scoped extension of a known correlation method. The algebraic derivation of the annihilator-based residue is clean, the unknown-input extension is natural, and the public MATLAB toolbox is a concrete asset that supports reproducibility. The numerical examples cover a realistic unobservable application (atomic-clock ensembles) and an unobservable LTV system with unknown inputs. However, the paper's central asymptotic claim currently rests on a finite-rank condition that is not sufficient for consistency; this is a load-bearing gap that requires either a new theorem with explicit persistence-of-excitation and moment conditions or a retreat to a weaker claim.

major comments (3)
  1. [Section V-D, Eq. (21)] The statement that the ordinary MDM gives unbiased and consistent estimates 'provided that the matrix A (19) has full column rank' is not supported. Full column rank of A for a finite horizon guarantees identifiability of alpha from the expected normal equations, but it does not control the asymptotic behaviour of the sample average in (21). A concrete counterexample is the scalar model F_k=1, E_k=D_k=1, G_k=0, H_k=1/k. For every finite tau >= 2 the matrix A in (19) has full column rank, but the regressor column corresponding to Q decays like 1/k^2, so (1/tau)A^T A tends to a singular limit and the OLS estimate of Q is not consistent. Consistency requires a uniform lower bound on the smallest eigenvalue of (1/tau)A^T A and conditions under which (1/tau)A^T L Y converges to zero in probability (e.g., moment and weak-dependence assumptions on eta_k). The sentence 'the proof is similar to [14]' is not a substitute, because [14] assumes an observable model and therefore its proof does not cover rank-deficient O_k or the unknown-input annihilator in Eq. (31).
  2. [Section V-C] The paper explicitly states that no analytical identifiability condition has been established for unobservable LTV systems and that the number of identifiable parameters can only be inferred numerically from the rank of A. This admission is in tension with the unqualified consistency claim in the abstract and in Section V-D. The H_k=1/k example illustrates the gap: finite-horizon rank of A can be full while the asymptotic design matrix is singular, so numerical rank at a chosen horizon does not certify consistency. The authors should either state a formal identifiability and consistency condition (for example, a uniform excitation condition on the stacked design A) or explicitly qualify the consistency claim to settings satisfying that condition.
  3. [Section V-D, weighted MDM] The sentence that the weighted MDM estimate is biased but 'the estimate bias goes to zero with an increasing number of data' is asserted without conditions. Because the weighting matrix P in Eq. (22) is estimated from the same data used in Eq. (23), the feasible weighted LS estimator is subject to the standard conditions for asymptotic unbiasedness of two-step estimators. If this claim is retained, it should be justified by a standard result or by explicit assumptions on the noise moments and on the rate at which the estimated P converges to P.
minor comments (4)
  1. [Section V-E] The existence condition for the annihilation matrix is stated only for the known-input case as n_{z,k,L} > rank(O_k). For the unknown-input residue in Eq. (31), the corresponding condition should be n_{z,k,L} > rank([O_k, Gamma_k G_k]); otherwise the annihilator may not exist even when the known-input condition is satisfied.
  2. [Section VII-D] The sentence 'The numerical illustrations confirm that the ordinary MDM provides unbiased estimates for any linear model' is too broad; the simulations demonstrate unbiasedness for the three tested models, not for any linear model. The theoretical statement in Section V-D already carries the full column rank and moment conditions, so the concluding sentence should be qualified accordingly.
  3. [Abstract and Introduction] There are several missing spaces in the typeset text ('correlationmeasurementdifference', 'unobservablesystem', 'availableMDMsource codes', 'which further reads', 'weightedLS'). These appear to be LaTeX formatting artifacts and should be corrected in the final version.
  4. [Section IV-B, Eq. (22)] The block matrix expression for E[YY^T] is visually ambiguous because the off-diagonal blocks are not explicitly indexed; stating that the (i,j) block is (R_{eta_i,eta_j})_M would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the generalized MDM estimate is derived from the model equations by linear algebra; the only flagged issue is a consistency-proof gap, not a circular reduction.

full rationale

The paper's central derivation is self-contained. Equation (5) defines the residue by annihilating the observability matrix O_k in the augmented measurement equation (2), eliminating the state while leaving a linear function of the noises. Equations (8), (10), and (16) propagate this into a linear map from the unknown covariance parameters alpha to the residual covariance, and (21) is the ordinary least-squares estimate of alpha. At no point is alpha fitted to data and then relabeled as a prediction of Q and R; alpha is the unknown vector the method is designed to identify, and the numerical examples compare the estimates with independently chosen ground-truth matrices. The structure-defining parametrization from [12] and the MDM machinery from [14] are external methodology, not assumptions equivalent to the result. The one passage that delegates a proof to prior work is Section V-D ('The proof is similar to the one presented in [14]'), but this is a proof-similarity claim about a least-squares consistency argument, not a reduction of the new unobservable/unknown-input claim to its own inputs. The gap between finite-horizon full-column-rank A and a persistence-of-excitation condition, and the absence of an analytical identifiability condition for unobservable systems noted in Section V-C, are missing technical assumptions or correctness risks rather than circularity. Accordingly, no circular step was found.

Assumptions & free parameters 1 free parameters · 8 assumptions · 0 invented entities

The central estimation problem is well-posed only if the user can specify the covariance structure and if the stacked regressor is identifiable. The paper is explicit that an analytical identifiability formula is missing and that an unknown input can make Q unidentifiable when the input and state-noise shaping matrices coincide (Section V-C). No new physical entities are introduced.

free parameters (1)
  • L (window length, l_L = L-1) = L=10 in the clock example, L=2 in the unknown-input and observable examples
    User-selected parameter in the augmented measurement vector (2). It must satisfy n_z,k,L > rank(O_k), or the analogous condition for the unknown-input case, and it trades the number of equations against computational complexity. No data-driven selection rule is given.
assumptions (8)
  • standard math There exists an annihilation matrix am(O_k) with am(O_k) O_k = 0 whenever n_z,k,L > rank(O_k).
    Used in Eqs. (4)-(5) to remove the state from the augmented measurement; existence is a linear algebra fact.
  • standard math Kronecker product and vectorization identities used in Eqs. (8)-(12) are valid.
    Standard matrix algebra used to express residue covariance in terms of the noise covariance parameters.
  • domain assumption State noise w_k and measurement noise v_k are zero-mean and have time-invariant covariance matrices Q and R.
    Section II defines the model; the method estimates these time-invariant Q and R.
  • domain assumption True Q and R can be represented as linear combinations of user-selected structure-defining matrices B_Q(i) and B_R(i) with unknown weights alpha.
    Equation (9); if the selected structure is wrong, the estimated alpha does not correspond to the true covariance matrices.
  • domain assumption System matrices F_k, G_k, E_k, H_k, D_k are known and bounded for all k.
    The method requires these matrices to build A and Upsilon in Eqs. (19)-(20); stated in Section II.
  • domain assumption The stacked least-squares regressor A in (19) has full column rank.
    Needed for the ordinary LS estimate (21). The paper checks this rank numerically in examples but gives no analytical identifiability condition (Section V-C).
  • domain assumption The time-varying design is persistently exciting and the noise has enough finite moments and decaying dependence for consistency as tau grows.
    Section V-D claims consistency by reference to [14] without stating these conditions for the LTV, unobservable, unknown-input case.
  • domain assumption For the weighted MDM, noises are Gaussian, or their fourth-order moments are otherwise available.
    Section IV-B step 2 states that for Gaussian noises the fourth moments can be computed from the estimated covariances; for other distributions they must be estimated separately.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unobservable Systems: No Problem for Noise Identification." pith.science (2026). https://pith.science/paper/MWQ7THUB

@misc{pith2026250523983,
  author       = {Pith},
  title        = {Pith review of: Unobservable Systems: No Problem for Noise Identification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWQ7THUB}},
  note         = {Machine review of arXiv:2505.23983}
}
read the original abstract

This paper deals with the noise identification of a linear time-varying stochastic dynamic system described by the state-space model. In particular, the stress is laid on the design of the correlation measurement difference method for estimation of the state and measurement noise covariance matrices for both observable and \textit{unobservable} systems with possibly unknown input sequence. The method provides unbiased and consistent estimates and is implemented in a publicly available MATLAB toolbox and numerically evaluated.

Figures

Figures reproduced from arXiv: 2505.23983 by the authors.

Figure 2
Figure 2. Ordinary MDM identification (unobservable, LTV, unknown input). [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Belanger’s method and (ordinary and weighted) MDM identification. ´ [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 25 canonical work pages

  1. [14]

    Measurement difference method: A universal tool for noise identification,

    O. Kost, J. Dun ´ık, and O. Straka, “Measurement difference method: A universal tool for noise identification,”IEEE Transactions on Automatic Control, vol. 68, no. 3, pp. 1792–1799, 2023

  2. [1]

    Autonomous tracking and state estimation with generalized group lasso,

    R. Gao, S. S ¨arkk¨a, R. Claveria-Vega, and S. Godsill, “Autonomous tracking and state estimation with generalized group lasso,”IEEE Transactions on Cybernetics, vol. 52, no. 11, pp. 12 056–12 070, 2022

  3. [2]

    Approaches to adaptive filtering,

    R. K. Mehra, “Approaches to adaptive filtering,”IEEE Transactions on Automatic Control, vol. 17, no. 10, pp. 693–698, 1972

  4. [3]

    Noise covariance matrices in state-space models: A survey and comparison - part I,

    J. Dun ´ık, O. Straka, O. Kost, and J. Havl ´ık, “Noise covariance matrices in state-space models: A survey and comparison - part I,”International Journal of Adaptive Control and Signal Processing, vol. 31, no. 11, pp. 1505–1543, 2017

  5. [4]

    Recursive noise adaptive Kalman filtering by variational Bayesian approximations,

    S. S ¨arkk¨a and A. Nummenmaa, “Recursive noise adaptive Kalman filtering by variational Bayesian approximations,”IEEE Transactions on Automatic Control, vol. 54, no. 3, pp. 596–600, 2009

  6. [5]

    Variational adaptive Kalman filter with Gaussian-Inverse-Wishart mixture distribution,

    Y . Huang, Y . Zhang, P. Shi, and J. Chambers, “Variational adaptive Kalman filter with Gaussian-Inverse-Wishart mixture distribution,”IEEE Transactions on Automatic Control, vol. 66, no. 4, pp. 1786–1793, 2021

  7. [6]

    A. R. Moghaddamjoo and R. L. Kirlin,Approximate Kalman Filtering, ser. Series in Approximations and Decompositions. World scientific, 1993, vol. 2, ch. Robust Adaptive Kalman Filtering, pp. 65–85, (Ed. Chen, G.)

  8. [7]

    Esti- mating model error covariance matrix parameters in extended Kalman filtering,

    A. Solonen, J. Hakkarainen, A. Ilin, M. Abbas, and A. Bibov, “Esti- mating model error covariance matrix parameters in extended Kalman filtering,”Nonlinear Processes in Geophysics, vol. 21, no. 5, pp. 919– 927, 2014

Show all 28 references
  1. [8]

    R. H. Shumway and D. S. Stoffer,Time Series Analysis and its Applications. Springer-Verlag, 2000

  2. [9]

    System identification of nonlinear state-space models,

    T. B. Sch ¨on, A. Wills, and B. Ninness, “System identification of nonlinear state-space models,”Automatica, vol. 47, no. 1, pp. 39–49, 2011

  3. [10]

    Model error estimation using the expectation maximization algorithm and a particle flow filter,

    M. M. Lucini, P. J. van Leeuwen, and M. Pulido, “Model error estimation using the expectation maximization algorithm and a particle flow filter,” SIAM/ASA Journal on Uncertainty Quantification, vol. 9, no. 2, pp. 681– 707, 2021

  4. [11]

    A-KIT: Adaptive Kalman-informed trans- former,

    N. Cohen and I. Klein, “A-KIT: Adaptive Kalman-informed trans- former,”arXiv: 2401.09987, 2024

  5. [12]

    Estimation of noise covariance matrices for a linear time-varying stochastic process,

    P. R. B ´elanger, “Estimation of noise covariance matrices for a linear time-varying stochastic process,”Automatica, vol. 10, no. 3, pp. 267– 275, 1974

  6. [13]

    A new autocovari- ance least-squares method for estimating noise covariances,

    B. J. Odelson, M. R. Rajamani, and J. B. Rawlings, “A new autocovari- ance least-squares method for estimating noise covariances,”Automatica, vol. 42, no. 2, pp. 303–308, 2006

  7. [15]

    A tutorial on the two-state model of the atomic clock noise,

    L. Galleani, “A tutorial on the two-state model of the atomic clock noise,”Metrologia, vol. 45, no. 6, pp. 175–182, 2008

  8. [16]

    Time and the Kalman Filter: Applications of optimal estimation to atomic timing,

    L. Galleani and P. Tavella, “Time and the Kalman Filter: Applications of optimal estimation to atomic timing,”IEEE Control Systems Magazine, vol. 30, no. 2, pp. 44–65, 2010

  9. [17]

    Bar-Shalom, X

    Y . Bar-Shalom, X. R. Li, and T. Kirubarajan,Estimation with Appli- cations to Tracking and Navigation: Theory Algorithms and Software. John Wiley & Sons, 2001

  10. [18]

    A fast distributed variational Bayesian filtering for multisensor ltv system with non-Gaussian noise,

    J. Li, F. Deng, and J. Chen, “A fast distributed variational Bayesian filtering for multisensor ltv system with non-Gaussian noise,”IEEE Transactions on Cybernetics, vol. 49, no. 7, pp. 2431–2443, 2019

  11. [19]

    Design of measurement difference autocovariance method for estimation of process and measurement noise covariances,

    J. Dun ´ık, O. Kost, and O. Straka, “Design of measurement difference autocovariance method for estimation of process and measurement noise covariances,”Automatica, vol. 90, pp. 16–24, 2018

  12. [20]

    Noise covariances identification by MDM: Weighting, recursion, and implementation,

    O. Kost, J. Dun ´ık, and O. Straka, “Noise covariances identification by MDM: Weighting, recursion, and implementation,” inAccepted for the 20th IFAC Symposium on System Identification (SYSID), Boston (MA), USA, 2024

  13. [21]

    J. R. Magnus and H. Neudecker,Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition, 3rd ed. John Wiley & Sons, 2019

  14. [22]

    B. P. Gibbs,Advanced Kalman Filtering, Least-Squares and Modelling. John Wiley & Sons, 2011

  15. [23]

    Identifiability analysis of noise covariances for LTI stochastic systems with unknown inputs,

    H. Kong, S. Sukkarieh, T. J. Arnold, T. Chen, B. Mu, and W. X. Zheng, “Identifiability analysis of noise covariances for LTI stochastic systems with unknown inputs,”IEEE Transactions on Automatic Control, vol. 68, no. 7, pp. 4459–4466, 2023

  16. [24]

    Identifiability of unique elements of noise covariances in state-space models,

    O. Kost, J. Dun ´ık, and O. Straka, “Identifiability of unique elements of noise covariances in state-space models,” inProceedings of 19th IFAC Symposium on System Identification (SYSID), Padova, Italy, 2021

  17. [25]

    Covariance estimation and gaussianity assessment for state and measurement noise,

    J. Dun ´ık, O. Kost, O. Straka, and E. Blasch, “Covariance estimation and gaussianity assessment for state and measurement noise,”Journal of Guidance, Control, and Dynamics, vol. 43, no. 1, pp. 132–139, 2020

  18. [26]

    Correlated noise characteristics estimation for linear time-varying systems,

    O. Kost, J. Dun ´ık, and O. Straka, “Correlated noise characteristics estimation for linear time-varying systems,” in57th Conference on Decision and Control, Miami, FL, USA, Dec 2018

  19. [27]

    Noise moment and parameter estimation of state-space model,

    ——, “Noise moment and parameter estimation of state-space model,”In Proceedings of 18th IFAC Symposium on System Identification (SYSID), vol. 51, no. 15, pp. 891–896, 2018

  20. [28]

    Identification of GNSS measurement error: From time to elevation dependency,

    O. Kost, J. Dun ´ık, O. Straka, and O. Daniel, “Identification of GNSS measurement error: From time to elevation dependency,”IEEE Trans- actions on Aerospace and Electronic Systems, vol. 59, no. 6, pp. 8931– 8943, 2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.