REVIEW 3 major objections 4 minor 28 references
Unobservable Systems: No Problem for Noise Identification
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper shows that the measurement difference method can estimate state and measurement noise covariances in linear time-varying systems even when the state is unobservable and the input is unknown.
desk verdict Genuinely useful MDM generalization for unobservable and unknown-input LTV systems, but the consistency claim is under-supported and false as stated; a persistence-of-excitation condition is needed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The annihilation matrix $\mathrm{am}(O_k)$ is the central object: any matrix whose rows span a left null space of the observability matrix, so $\mathrm{am}(O_k)O_k = 0$. Multiplying the augmented measurement equation by this matrix eliminates the unknown state $x_k$ in one step, producing the residual $\tilde{eZ}_k$ that is exactly linear in the noises $W_k, V_k$. The paper builds the argument on this residual: its covariance is related to the sought noise covariances through the Kronecker-product identity $\mathcal{R}_{eZ_k^2} = \Xi_{eZ_k^2} A_k^{\otimes 2} C_k^{\otimes 2} \Upsilon_{E^2}\,\alpha$, and stacking over time yields a linear system solved by least squares. The structure-defining matrices $B_Q^{(i)}, B_R^{(i)}$ convert the unknown matrices $Q, R$ into a parameter vector $\alpha$, and the unification matrix $\Xi$ removes redundant entries. The same annihilation principle, applied to $[O_k, \Gamma_k G_k]$, extends the mechanism to unknown inputs.
What would settle it
Pick a fixed unobservable LTV model with known noise covariances and compute $A$ in (19) for increasing horizon $\tau$ at a given $L$; if a full-column-rank $A$ is found but the ordinary MDM estimate from simulated data does not converge to the true $\alpha$ as $\tau$ grows, the consistency claim is contradicted. Conversely, finding a rank-deficient $A$ for which simulation still gives unbiased estimates would show the stated condition is sufficient rather than necessary.
Extended reading notes
Core claim
The paper's central claim is that the measurement difference method, previously derived under observability and known-input assumptions, remains valid for unobservable linear time-varying systems and for systems with unknown inputs, provided the rank-deficient observability matrix is annihilated from the left rather than inverted. The residue $\tilde{eZ}_k = \mathrm{am}(O_k)(Z_k - \Gamma_k G_k U_k)$ is constructed so that $\mathrm{am}(O_k)O_k = 0$; because $O_k$ need not have full rank, the state term disappears regardless of observability. The residue is then a linear function of the stacked state and measurement noises, so its covariance is affine in the unknown noise covariance parameters $\alpha$ after parametrizing $Q$ and $R$ with structure-defining matrices. Stacking these relations over time gives $\mathcal{R}_{eZ^2} = A\alpha + L\eta$, and the ordinary least-squares estimate $\hat\alpha_o = (A^\top A)^{-1}A^\top \mathcal{R}_{eZ^2}$ is claimed to be unbiased and consistent whenever $A$ has full column rank. For unknown inputs, the annihilator is applied to $[O_k, \Gamma_k G_k]$ in place of $O_k$, which removes both the state and the input contributions at the cost of possibly fewer identifiable parameters.
Load-bearing premise
The load-bearing premise is that the stacked linear system in (19) has full column rank and that the sample averages of the residual products converge to their true covariances as the number of measurements grows; if either fails, the claimed unbiasedness and consistency of the ordinary MDM no longer follow.
Editorial extensions
If this is right
- The ordinary MDM gives unbiased and consistent noise-covariance estimates for unobservable LTV systems as long as the stacked design matrix $A$ in (19) has full column rank.
- Unknown input sequences can be accommodated by annihilating the joint state-input matrix; the price is a possible reduction in the number of identifiable noise-covariance parameters, with the extreme case $G_k = E_k$ making $Q$ unidentifiable.
- The weighted MDM produces lower-variance estimates and a realistic estimate of its own covariance, at the cost of a small bias that vanishes as the data record grows.
- The number of identifiable noise-covariance parameters for a given unobservable LTV model can be read off numerically from the rank of $A$, even though an analytical formula in terms of $n_x$, $n_{zk}$, and $\mathrm{rank}(O_k)$ remains open.
- The method inherits prior MDM extensions, including correlated noises, noise mean and higher-moment identification, and sensor calibration with unknown state dynamics.
Reading between the lines
- A direct practical consequence the paper does not spell out: because identifiability is decided by the numeric rank of $A$, a practitioner can pre-test a planned experiment design (horizon $L$ and time span $\tau$) before collecting data, and choose $L$ just large enough to make $A$ full rank.
- The annihilation-matrix viewpoint suggests a broader class of identification residuals: any left annihilator of the state contribution yields a valid statistic, so one could optimize the annihilator to maximize Fisher information or inferential precision instead of taking any fixed null-space basis.
- For non-Gaussian noises, the consistency claim relies on convergence of sample averages of fourth-order products; making the moment conditions explicit would give practitioners a testable check beyond the Gaussian examples shown in the paper.
- For the clock-ensemble application, this implies that noise covariances can be calibrated from the same phase-difference measurements used for time-scale generation, without a separate observable reference.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper extends the measurement difference method (MDM) for estimating state and measurement noise covariance parameters in linear time-varying state-space models to two settings not covered by previous MDM versions: rank-deficient observability matrices and unknown input sequences. The key device is to replace the pseudo-inverse of the observability matrix with an annihilation matrix am(O_k), so the residue eZ_k becomes a linear function of the noises even when O_k is rank deficient; for unknown inputs the annihilator is applied to [O_k, Gamma_k G_k]. Noise covariances are parametrized through structure-defining matrices and unknown weights alpha, and ordinary and weighted least-squares estimators are derived. The paper claims unbiased and consistent estimates, provides a MATLAB implementation, and reports Monte Carlo studies for three examples including an unobservable clock-ensemble model.
Significance. If the consistency claim can be made rigorous, the paper is a useful and well-scoped extension of a known correlation method. The algebraic derivation of the annihilator-based residue is clean, the unknown-input extension is natural, and the public MATLAB toolbox is a concrete asset that supports reproducibility. The numerical examples cover a realistic unobservable application (atomic-clock ensembles) and an unobservable LTV system with unknown inputs. However, the paper's central asymptotic claim currently rests on a finite-rank condition that is not sufficient for consistency; this is a load-bearing gap that requires either a new theorem with explicit persistence-of-excitation and moment conditions or a retreat to a weaker claim.
major comments (3)
- [Section V-D, Eq. (21)] The statement that the ordinary MDM gives unbiased and consistent estimates 'provided that the matrix A (19) has full column rank' is not supported. Full column rank of A for a finite horizon guarantees identifiability of alpha from the expected normal equations, but it does not control the asymptotic behaviour of the sample average in (21). A concrete counterexample is the scalar model F_k=1, E_k=D_k=1, G_k=0, H_k=1/k. For every finite tau >= 2 the matrix A in (19) has full column rank, but the regressor column corresponding to Q decays like 1/k^2, so (1/tau)A^T A tends to a singular limit and the OLS estimate of Q is not consistent. Consistency requires a uniform lower bound on the smallest eigenvalue of (1/tau)A^T A and conditions under which (1/tau)A^T L Y converges to zero in probability (e.g., moment and weak-dependence assumptions on eta_k). The sentence 'the proof is similar to [14]' is not a substitute, because [14] assumes an observable model and therefore its proof does not cover rank-deficient O_k or the unknown-input annihilator in Eq. (31).
- [Section V-C] The paper explicitly states that no analytical identifiability condition has been established for unobservable LTV systems and that the number of identifiable parameters can only be inferred numerically from the rank of A. This admission is in tension with the unqualified consistency claim in the abstract and in Section V-D. The H_k=1/k example illustrates the gap: finite-horizon rank of A can be full while the asymptotic design matrix is singular, so numerical rank at a chosen horizon does not certify consistency. The authors should either state a formal identifiability and consistency condition (for example, a uniform excitation condition on the stacked design A) or explicitly qualify the consistency claim to settings satisfying that condition.
- [Section V-D, weighted MDM] The sentence that the weighted MDM estimate is biased but 'the estimate bias goes to zero with an increasing number of data' is asserted without conditions. Because the weighting matrix P in Eq. (22) is estimated from the same data used in Eq. (23), the feasible weighted LS estimator is subject to the standard conditions for asymptotic unbiasedness of two-step estimators. If this claim is retained, it should be justified by a standard result or by explicit assumptions on the noise moments and on the rate at which the estimated P converges to P.
minor comments (4)
- [Section V-E] The existence condition for the annihilation matrix is stated only for the known-input case as n_{z,k,L} > rank(O_k). For the unknown-input residue in Eq. (31), the corresponding condition should be n_{z,k,L} > rank([O_k, Gamma_k G_k]); otherwise the annihilator may not exist even when the known-input condition is satisfied.
- [Section VII-D] The sentence 'The numerical illustrations confirm that the ordinary MDM provides unbiased estimates for any linear model' is too broad; the simulations demonstrate unbiasedness for the three tested models, not for any linear model. The theoretical statement in Section V-D already carries the full column rank and moment conditions, so the concluding sentence should be qualified accordingly.
- [Abstract and Introduction] There are several missing spaces in the typeset text ('correlationmeasurementdifference', 'unobservablesystem', 'availableMDMsource codes', 'which further reads', 'weightedLS'). These appear to be LaTeX formatting artifacts and should be corrected in the final version.
- [Section IV-B, Eq. (22)] The block matrix expression for E[YY^T] is visually ambiguous because the off-diagonal blocks are not explicitly indexed; stating that the (i,j) block is (R_{eta_i,eta_j})_M would improve readability.
Circularity Check
No significant circularity: the generalized MDM estimate is derived from the model equations by linear algebra; the only flagged issue is a consistency-proof gap, not a circular reduction.
full rationale
The paper's central derivation is self-contained. Equation (5) defines the residue by annihilating the observability matrix O_k in the augmented measurement equation (2), eliminating the state while leaving a linear function of the noises. Equations (8), (10), and (16) propagate this into a linear map from the unknown covariance parameters alpha to the residual covariance, and (21) is the ordinary least-squares estimate of alpha. At no point is alpha fitted to data and then relabeled as a prediction of Q and R; alpha is the unknown vector the method is designed to identify, and the numerical examples compare the estimates with independently chosen ground-truth matrices. The structure-defining parametrization from [12] and the MDM machinery from [14] are external methodology, not assumptions equivalent to the result. The one passage that delegates a proof to prior work is Section V-D ('The proof is similar to the one presented in [14]'), but this is a proof-similarity claim about a least-squares consistency argument, not a reduction of the new unobservable/unknown-input claim to its own inputs. The gap between finite-horizon full-column-rank A and a persistence-of-excitation condition, and the absence of an analytical identifiability condition for unobservable systems noted in Section V-C, are missing technical assumptions or correctness risks rather than circularity. Accordingly, no circular step was found.
Assumptions & free parameters
free parameters (1)
- L (window length, l_L = L-1) =
L=10 in the clock example, L=2 in the unknown-input and observable examples
assumptions (8)
- standard math There exists an annihilation matrix am(O_k) with am(O_k) O_k = 0 whenever n_z,k,L > rank(O_k).
- standard math Kronecker product and vectorization identities used in Eqs. (8)-(12) are valid.
- domain assumption State noise w_k and measurement noise v_k are zero-mean and have time-invariant covariance matrices Q and R.
- domain assumption True Q and R can be represented as linear combinations of user-selected structure-defining matrices B_Q(i) and B_R(i) with unknown weights alpha.
- domain assumption System matrices F_k, G_k, E_k, H_k, D_k are known and bounded for all k.
- domain assumption The stacked least-squares regressor A in (19) has full column rank.
- domain assumption The time-varying design is persistently exciting and the noise has enough finite moments and decaying dependence for consistency as tau grows.
- domain assumption For the weighted MDM, noises are Gaussian, or their fourth-order moments are otherwise available.
Cite this review
Pith. "Pith review of Unobservable Systems: No Problem for Noise Identification." pith.science (2026). https://pith.science/paper/MWQ7THUB
@misc{pith2026250523983,
author = {Pith},
title = {Pith review of: Unobservable Systems: No Problem for Noise Identification},
year = {2026},
howpublished = {\url{https://pith.science/paper/MWQ7THUB}},
note = {Machine review of arXiv:2505.23983}
}
read the original abstract
This paper deals with the noise identification of a linear time-varying stochastic dynamic system described by the state-space model. In particular, the stress is laid on the design of the correlation measurement difference method for estimation of the state and measurement noise covariance matrices for both observable and \textit{unobservable} systems with possibly unknown input sequence. The method provides unbiased and consistent estimates and is implemented in a publicly available MATLAB toolbox and numerically evaluated.
Figures
Reference graph
Works this paper leans on
-
[14]
Measurement difference method: A universal tool for noise identification,
O. Kost, J. Dun ´ık, and O. Straka, “Measurement difference method: A universal tool for noise identification,”IEEE Transactions on Automatic Control, vol. 68, no. 3, pp. 1792–1799, 2023
work page 2023
-
[1]
Autonomous tracking and state estimation with generalized group lasso,
R. Gao, S. S ¨arkk¨a, R. Claveria-Vega, and S. Godsill, “Autonomous tracking and state estimation with generalized group lasso,”IEEE Transactions on Cybernetics, vol. 52, no. 11, pp. 12 056–12 070, 2022
work page 2022
-
[2]
Approaches to adaptive filtering,
R. K. Mehra, “Approaches to adaptive filtering,”IEEE Transactions on Automatic Control, vol. 17, no. 10, pp. 693–698, 1972
work page 1972
-
[3]
Noise covariance matrices in state-space models: A survey and comparison - part I,
J. Dun ´ık, O. Straka, O. Kost, and J. Havl ´ık, “Noise covariance matrices in state-space models: A survey and comparison - part I,”International Journal of Adaptive Control and Signal Processing, vol. 31, no. 11, pp. 1505–1543, 2017
work page 2017
-
[4]
Recursive noise adaptive Kalman filtering by variational Bayesian approximations,
S. S ¨arkk¨a and A. Nummenmaa, “Recursive noise adaptive Kalman filtering by variational Bayesian approximations,”IEEE Transactions on Automatic Control, vol. 54, no. 3, pp. 596–600, 2009
work page 2009
-
[5]
Variational adaptive Kalman filter with Gaussian-Inverse-Wishart mixture distribution,
Y . Huang, Y . Zhang, P. Shi, and J. Chambers, “Variational adaptive Kalman filter with Gaussian-Inverse-Wishart mixture distribution,”IEEE Transactions on Automatic Control, vol. 66, no. 4, pp. 1786–1793, 2021
work page 2021
-
[6]
A. R. Moghaddamjoo and R. L. Kirlin,Approximate Kalman Filtering, ser. Series in Approximations and Decompositions. World scientific, 1993, vol. 2, ch. Robust Adaptive Kalman Filtering, pp. 65–85, (Ed. Chen, G.)
work page 1993
-
[7]
Esti- mating model error covariance matrix parameters in extended Kalman filtering,
A. Solonen, J. Hakkarainen, A. Ilin, M. Abbas, and A. Bibov, “Esti- mating model error covariance matrix parameters in extended Kalman filtering,”Nonlinear Processes in Geophysics, vol. 21, no. 5, pp. 919– 927, 2014
work page 2014
Show all 28 references
-
[8]
R. H. Shumway and D. S. Stoffer,Time Series Analysis and its Applications. Springer-Verlag, 2000
2000
-
[9]
System identification of nonlinear state-space models,
T. B. Sch ¨on, A. Wills, and B. Ninness, “System identification of nonlinear state-space models,”Automatica, vol. 47, no. 1, pp. 39–49, 2011
2011
-
[10]
Model error estimation using the expectation maximization algorithm and a particle flow filter,
M. M. Lucini, P. J. van Leeuwen, and M. Pulido, “Model error estimation using the expectation maximization algorithm and a particle flow filter,” SIAM/ASA Journal on Uncertainty Quantification, vol. 9, no. 2, pp. 681– 707, 2021
2021
-
[11]
A-KIT: Adaptive Kalman-informed trans- former,
N. Cohen and I. Klein, “A-KIT: Adaptive Kalman-informed trans- former,”arXiv: 2401.09987, 2024
2024 arXiv
-
[12]
Estimation of noise covariance matrices for a linear time-varying stochastic process,
P. R. B ´elanger, “Estimation of noise covariance matrices for a linear time-varying stochastic process,”Automatica, vol. 10, no. 3, pp. 267– 275, 1974
1974
-
[13]
A new autocovari- ance least-squares method for estimating noise covariances,
B. J. Odelson, M. R. Rajamani, and J. B. Rawlings, “A new autocovari- ance least-squares method for estimating noise covariances,”Automatica, vol. 42, no. 2, pp. 303–308, 2006
2006
-
[15]
A tutorial on the two-state model of the atomic clock noise,
L. Galleani, “A tutorial on the two-state model of the atomic clock noise,”Metrologia, vol. 45, no. 6, pp. 175–182, 2008
2008
-
[16]
Time and the Kalman Filter: Applications of optimal estimation to atomic timing,
L. Galleani and P. Tavella, “Time and the Kalman Filter: Applications of optimal estimation to atomic timing,”IEEE Control Systems Magazine, vol. 30, no. 2, pp. 44–65, 2010
2010
-
[17]
Bar-Shalom, X
Y . Bar-Shalom, X. R. Li, and T. Kirubarajan,Estimation with Appli- cations to Tracking and Navigation: Theory Algorithms and Software. John Wiley & Sons, 2001
2001
-
[18]
A fast distributed variational Bayesian filtering for multisensor ltv system with non-Gaussian noise,
J. Li, F. Deng, and J. Chen, “A fast distributed variational Bayesian filtering for multisensor ltv system with non-Gaussian noise,”IEEE Transactions on Cybernetics, vol. 49, no. 7, pp. 2431–2443, 2019
2019
-
[19]
Design of measurement difference autocovariance method for estimation of process and measurement noise covariances,
J. Dun ´ık, O. Kost, and O. Straka, “Design of measurement difference autocovariance method for estimation of process and measurement noise covariances,”Automatica, vol. 90, pp. 16–24, 2018
2018
-
[20]
Noise covariances identification by MDM: Weighting, recursion, and implementation,
O. Kost, J. Dun ´ık, and O. Straka, “Noise covariances identification by MDM: Weighting, recursion, and implementation,” inAccepted for the 20th IFAC Symposium on System Identification (SYSID), Boston (MA), USA, 2024
2024
-
[21]
J. R. Magnus and H. Neudecker,Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition, 3rd ed. John Wiley & Sons, 2019
2019
-
[22]
B. P. Gibbs,Advanced Kalman Filtering, Least-Squares and Modelling. John Wiley & Sons, 2011
2011
-
[23]
Identifiability analysis of noise covariances for LTI stochastic systems with unknown inputs,
H. Kong, S. Sukkarieh, T. J. Arnold, T. Chen, B. Mu, and W. X. Zheng, “Identifiability analysis of noise covariances for LTI stochastic systems with unknown inputs,”IEEE Transactions on Automatic Control, vol. 68, no. 7, pp. 4459–4466, 2023
2023
-
[24]
Identifiability of unique elements of noise covariances in state-space models,
O. Kost, J. Dun ´ık, and O. Straka, “Identifiability of unique elements of noise covariances in state-space models,” inProceedings of 19th IFAC Symposium on System Identification (SYSID), Padova, Italy, 2021
2021
-
[25]
Covariance estimation and gaussianity assessment for state and measurement noise,
J. Dun ´ık, O. Kost, O. Straka, and E. Blasch, “Covariance estimation and gaussianity assessment for state and measurement noise,”Journal of Guidance, Control, and Dynamics, vol. 43, no. 1, pp. 132–139, 2020
2020
-
[26]
Correlated noise characteristics estimation for linear time-varying systems,
O. Kost, J. Dun ´ık, and O. Straka, “Correlated noise characteristics estimation for linear time-varying systems,” in57th Conference on Decision and Control, Miami, FL, USA, Dec 2018
2018
-
[27]
Noise moment and parameter estimation of state-space model,
——, “Noise moment and parameter estimation of state-space model,”In Proceedings of 18th IFAC Symposium on System Identification (SYSID), vol. 51, no. 15, pp. 891–896, 2018
2018
-
[28]
Identification of GNSS measurement error: From time to elevation dependency,
O. Kost, J. Dun ´ık, O. Straka, and O. Daniel, “Identification of GNSS measurement error: From time to elevation dependency,”IEEE Trans- actions on Aerospace and Electronic Systems, vol. 59, no. 6, pp. 8931– 8943, 2023
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.