On building the state error covariance from a state estimate

Pavel Sakov

REVIEW 3 major objections 5 minor 5 references

On building the state error covariance from a state estimate

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that the state error covariance can be built from the current state estimate alone without being carried between assimilation cycles: in the standard Lorenz-96 test the proposed Algorithm 2 reaches analysis RMSE 0.181…

desk verdict A short, honest note showing that a static rule can rebuild a flow-dependent covariance from a single L96 state and match a tuned EnKF; worth a serious referee, though the evidence is thin. read the letter →

arxiv 2411.14809 v3 pith:YEENBU6D submitted 2024-11-22 nlin.CD

Pavel Sakov This is my paper · ORCID

classification nlin.CD

keywords dataassimilationstateerrorcovarianceensembleKalmanfilterLorenz-96tangentlinearmodeldeeplearning4D-Varreconstruction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Data assimilation systems normally carry both a state estimate and an estimate of its error covariance, and propagating that covariance is the expensive part. This paper shows that in a standard Lorenz-96 test the covariance can instead be rebuilt from scratch at each cycle using only the current state estimate: a simple tangent-linear back-and-forth propagation gives RMSE 0.235, and adding an EnKF-style damping step gives 0.181, essentially matching the 0.180 of a well-tuned ensemble Kalman filter. The point is not that the proposed algorithms are practical — they are computationally expensive — but that they prove the covariance is recoverable from the state alone in this weakly nonlinear setting. That makes the recent deep-learning result, where a trained operator matches the EnKF while propagating a single state, less surprising: a static state-to-covariance map exists and could in principle be learned.

What carries the argument

The engine of the argument is the perturbation ensemble $A$ and the square-root damping factor $(I+S^T S)^{-1/2}$, where $S = H_k A R_k^{-1/2}/n^{1/2}$ is the scaled innovation sensitivity. The algorithm first back-propagates the state $T$ steps ($x_{-T} \leftarrow M_{-T}(x^f)$), initialises $A=\varepsilon I$, then for each step propagates $A$ with the tangent linear model, applies the damping factor, and advances the state. This reproduces, without observations, the covariance contraction the EnKF analysis would impose, so unstable modes are controlled and the window can be extended to $T=25$. Algorithm 1 omits the damping and uses $T=6$; its covariance is naturally localised because the TLM only spreads initially local perturbations over a short interval.

What would settle it

In the same Standard Test, increase the perturbation amplitude in Algorithm 2 from $\varepsilon = 0.8$ to $\varepsilon = 2$ (keeping $T=25$). If the analysis RMSE remains near 0.18, the tangent-linear assumption is not essential to the reconstruction; if it degrades toward 0.4, the claim depends on weak nonlinearity as stated.

Watch

Extended reading notes

Core claim

The paper's central claim is that a covariance built purely from a state estimate can be good enough for near-optimal filtering. In the Standard Test, the author constructs two static algorithms that take only the forecast state $x^f$ and return a covariance $P^f = P(x^f)$. Algorithm 1 propagates a scaled identity perturbation matrix backward in time and then forward with the tangent linear model over $T=6$ steps, reaching RMSE 0.235. Algorithm 2 repeats the propagation but damps the perturbations each step with the square-root factor $(I+S^T S)^{-1/2}$ that mimics the EnKF analysis update, reaching RMSE 0.181 with $T=25$, $\varepsilon=0.8$ — statistically indistinguishable from the EnKF's 0.180. Because both algorithms depend only on the state estimate and known observation operators, not on the system history or innovations, the author argues they could be reproduced by the deep-learning analysis operator studied by Bocquet et al. (2024).

Load-bearing premise

The state estimate is close enough to the true state that anomaly evolution over the covariance-generation interval is governed by the tangent linear model; if nonlinearity dominates, the built covariance is wrong.

Editorial extensions

If this is right

The deep-learning DA operator trained by Bocquet et al. (2024) does not need to carry a hidden covariance: a static rule based on the current state can supply one, so the near-EnKF accuracy of the single-state DLS is consistent with standard Kalman-filter logic.
In the Standard Test, a 4D-Var system with a window of only about one error-doubling time can already build a useful flow-dependent covariance (Algorithm 1's result), not just the 31-step window previously reported.
A static covariance builder must be tuned to the DA conditions (observation network, error variances, model forcing); in non-stationary conditions such tuning would not be possible, so a deep-learning system would face a much harder problem.
The algorithms are far more expensive than the EnKF (full-rank ensemble propagation over multiple cycles), so they demonstrate feasibility of state-only covariance reconstruction, not an efficient way to do data assimilation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If this reconstruction transfers to other weakly nonlinear models, then any deep-learning analysis operator trained on a stationary DA problem can be benchmarked against a hand-coded Algorithm 2 as a lower bound: a learned operator that cannot beat Algorithm 2 has not learned anything beyond tangent-linear covariance building.
Because Algorithm 2's damping uses only $H_k$ and $R_k$, not the innovations, the method predicts that in fully observed, stationary systems the state-error covariance is nearly independent of the particular observation values; this could be tested by comparing Algorithm 2 covariances with EnKF covariances over many cycles.
The back-and-forth propagation in Algorithm 2 implicitly requires an adjoint or backward model; a learned operator that matches the EnKF without any backward pass would imply that the same covariance can be inferred from local spatial patterns alone, which would be a stronger result than the paper's.
A natural extension is to test Algorithm 2 with partial observations or time-averaged observations, where innovations do carry information; the paper's argument suggests the state-only map would degrade, giving a target for when a learned operator would actually need more than the state.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, this note is more interesting than its size suggests. Sakov proposes two algorithms that build a forecast error covariance from a single state estimate in the Lorenz-96 standard test. Algorithm 1 back-propagates a scaled identity ensemble through the TLM and forward to analysis; Algorithm 2 adds an EnKF-style damping at every time step. A2 gets analysis RMSE 0.181 versus 0.180 for a tuned EnKF. That basically closes the gap and offers a concrete explanation for the Bocquet et al. deep-learning result: a static operator can indeed construct a good covariance from a state alone, at least in this stationary test.

What the paper does well: the algorithms are clearly specified, the claims are modest, and the limitations are stated plainly. The CLV growth-rate figures are a genuinely useful addition—they give a plausible reason why A1 works with a short T and why bursts matter. The writing is honest that these algorithms are not practical (full-rank TLM propagation, expensive) and that tuning is required. No circularity: the EnKF is an external baseline.

The soft spots are real but not disqualifying. There is a single experiment, no error bars, and no sensitivity analysis for the free parameters T and epsilon. The parameters are tuned to the benchmark, so the headline RMSE is not a prediction. The 'weakly nonlinear' assumption is asserted rather than measured. One stress-test worry does not hold up: A2 damps perturbations at every time step, not once per multi-step burst, so the 5-step CLV growth rates shown in Figure 4 do not directly test A2's propagation interval. That said, the damping itself relies on the same covariance being built, so a more direct check of the TLM regime would help. The visual covariance comparison in Figure 3 is suggestive, not quantitative. These are the things a referee should ask for: sensitivity runs, a second configuration, and code.

Who should read it: anyone trying to understand how a learned single-state filter can match the EnKF, and anyone working on ensemble-free data assimilation. It deserves a serious referee. As a note, it is a solid contribution; after adding sensitivity analysis and releasing code it would be a clean, citeable result.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes two deterministic algorithms (A1 and A2) that construct an estimate of the state-error covariance from a single state estimate in the Lorenz-96 standard data assimilation test. A1 back-propagates the forecast state T steps and then propagates a full-rank set of perturbations forward with the tangent linear model; A2 adds an EnKF-style damping of the perturbations at each intermediate step. In the standard test, A1 yields an analysis RMSE of 0.235 and A2 yields 0.181, compared with 0.180 for a well-tuned EnKF. The authors argue that this demonstrates that a static rule can build a flow-dependent covariance from the state alone, offering an interpretation of the deep-learning result of Bocquet et al. (2024). The paper is clearly written and candid about the stationarity and tuning limitations, but the central numerical claim rests on a single tuned run with no uncertainty quantification and no direct verification of the tangent-linear assumption.

Significance. If the reported performance is robust, the result is conceptually valuable for data assimilation: it shows that in a weakly nonlinear, stationary setting the state-error covariance can be reconstructed from the current state and the model, without carrying covariance across cycles. The algorithms are unambiguously specified, and the comparison against a tuned EnKF baseline is the right style of evidence. The paper also honestly notes that the algorithms are tuned to the stationary test and are much more expensive than the EnKF. The main caveats are that the evidence is a single trajectory with no error bars or sensitivity analysis, and the tangent-linear validity assumption is asserted rather than verified. These caveats currently limit the strength of the broader inference about deep-learning systems, but they are addressable within the scope of a revision.

major comments (3)

[Section 3, results] The reported RMSE values are point estimates from one experiment with T and epsilon tuned to the benchmark. The paper should provide the RMSE as a function of (T, epsilon) over a grid (e.g., contours), the standard deviation of the mean RMSE over independent observation draws or random seeds, and at least one repeated run to show that 0.181 versus 0.180 is not a chance fluctuation. Without such evidence, the 'basically matching' claim is not established.
[Section 2, preliminary considerations] The claim that anomalies evolve under the tangent linear model is load-bearing, but no diagnostic is given to show that the actual perturbations used in A1 (epsilon=0.925) and A2 (epsilon=0.8) remain in the TLM regime over the T-step interval. The paper's own Figures 4 and 5 show instantaneous CLV doubling times below 2 steps, so with T=25 and only one damping per step, perturbation bursts could leave the linear regime. I recommend adding a direct comparison between TLM and nonlinear propagation of the perturbations along the backward trajectory (e.g., relative error of the TLM prediction) and a record of the perturbation norm or magnitude over the interval.
[Section 3, Algorithms 1 and 2] The algorithm descriptions use M_{-T}, a backward integration of the chaotic model. The manuscript does not explain how this is computed or whether it is numerically stable. Since backward integration of a chaotic system is generally unstable and can leave the attractor, the reproducibility of the results depends on this detail. Please describe the numerical procedure (e.g., explicit RK4 with negative step, any regularization) and report whether the backward trajectory stays bounded for T=25.

minor comments (5)

[Throughout] There are several typos: 'Runger-Kutta' should be 'Runge-Kutta'; 'the the evolution' should be 'the evolution'; 'dumping' should be 'damping'; 'suplementary' should be 'supplementary'; '4-order' should be '4th-order'.
[Figure 2] Figure 2 would benefit from axis labels, a title, and a legend identifying the two curves, as the current image appears to lack these elements.
[Section 3, Figure 1 caption] The sentence 'The displayed range on the right plot is that from the left plot scaled by the ratio of the traces of the covariances' is confusing; please rephrase to clarify what is scaled and why.
[Section 4, last paragraph] The statement that 'the evolution of covariance in the Kalman filter depends on observation locations and error variances but is mainly independent from innovations' should be reconciled with the fact that Algorithm 2 explicitly uses H_k and R_k at every intermediate step. The paper should acknowledge more clearly that A2 is not a function of the state estimate alone, but also of the observation network and error statistics, even though these are fixed in the standard test.
[Section 4, footnote 2] The remark about SST ensemble spread appears unsupported and is not connected to the rest of the paper; either remove it or provide a reference and a substantive link to the covariance-building argument.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the algorithms are proposed constructions evaluated against an external EnKF benchmark, not predictions derived from their own fitted inputs.

full rationale

The paper's central claim is that two static algorithms can build a state-error covariance from a state estimate, and it supports this by running them in the standard Lorenz-96 test and comparing analysis RMSE with a well-tuned EnKF (0.235 and 0.181 versus 0.180). This is an external, empirical benchmark: the RMSE values are not algebraic consequences of the algorithm definitions. The tunable parameters T and epsilon are fitted to the benchmark, but the paper does not present the resulting RMSE as a prediction; it explicitly reports tuned runs. Algorithm 2 does borrow the EnKF/ETKF anomaly-damping formula A <- A(I+S^T S)^-1/2, so its covariance resembles the EnKF's by design, but this is algorithm construction rather than circularity: the match is still verified numerically against an independent EnKF implementation and is not a tautology. The citation to Bocquet and Sakov (2013) for 4D-Var performance is a self-citation, but it is only motivational; the paper's own algorithms provide the evidence for the possibility claim. The weak-nonlinearity/TLM assumption is a stated limitation and a correctness risk, not a circular step. No load-bearing step reduces, by definition or by fitted parameter, to its own inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on four hand-tuned parameters (T and epsilon for each algorithm), the weak nonlinearity assumption, the stationarity of the test, the perfect model assumption, and the standard Kalman filter covariance result. No new physical entities are introduced.

free parameters (4)

Algorithm 1 T = 6
Number of time steps for backward/forward TLM propagation in Algorithm 1; hand-tuned to the Standard Test to maximize performance.
Algorithm 1 epsilon = 0.925
Initial perturbation amplitude in Algorithm 1; hand-tuned to the Standard Test.
Algorithm 2 T = 25
Number of time steps in Algorithm 2; hand-tuned to the Standard Test.
Algorithm 2 epsilon = 0.8
Initial perturbation amplitude in Algorithm 2; hand-tuned to the Standard Test.

assumptions (4)

domain assumption The DA system is weakly nonlinear: the state estimate is close enough to the true state that anomalies evolve mainly under the tangent linear model.
Invoked in Section 2 to justify building covariance via TLM propagation; if this fails, the covariance reconstruction is invalid.
domain assumption The standard test is stationary (fixed model, observation network, observation error variance), allowing algorithm parameters to be tuned to these conditions.
Acknowledged in Section 4 as a condition for the algorithms' performance; limits generalization.
domain assumption Perfect model framework: the model used to generate truth is the same as the forecast model, and its TLM is available.
The standard test defines a perfect model framework; algorithms rely on exact model and TLM.
standard math The Kalman filter covariance update is independent of innovations, so the EnKF-style damping in Algorithm 2 is valid.
Standard result used in Algorithm 2; underlies the perturbation damping step.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On building the state error covariance from a state estimate." pith.science (2026). https://pith.science/paper/YEENBU6D

@misc{pith2026241114809,
  author       = {Pith},
  title        = {Pith review of: On building the state error covariance from a state estimate},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEENBU6D}},
  note         = {Machine review of arXiv:2411.14809}
}

read the original abstract

It was recently found with the aid of machine learning that for a variety of toy data assimilation systems with chaotic Lorenz-96 model it is possible to achieve a nearly-optimal data assimilation without carrying the state error covariance between cycles. This result does not look surprising on its own because not carrying covariance is the approach taken by standard 4D-Var, but it was found ``astonishing'' in the context of the machine learning-based system trained on the ensemble Kalman filter. This note proposes two algorithms for building the state error covariance from a state estimate that yield good performance and could be worked out by the deep learning-based system.

Figures

Figures reproduced from arXiv: 2411.14809 by the authors.

**Figure 2.** Analysis RMSE from the system with Algorithm 1 and from the EnKF during spin-up. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Comparison of covariances from Algorithm 2 and from the EnKF at the same cycle as in Fig. 1. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Growth rates of the first 3 CLVs of L96 for a random sequence of 100 5-timestep intervals. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Minimal, mean, and maximal growth rates of CLVs of L96 during a random sequence of 4000 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 5 canonical work pages

[1]

Farchi, T

Bocquet, M., A. Farchi, T. S. Finn, C. Durand, S. Cheng, Y. Chen, I. Pasmans, and A. Carrassi, 2024: Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemble. Chaos, 34, 091104

work page 2024
[2]

Bocquet, M. and P. Sakov, 2013: Joint state and parameter estimation with an iterative ensemble Kalman smoother. Nonlinear Proc. Geoph., 20, 803–818

work page 2013
[3]

Chat´ e, R

Ginelli, F., H. Chat´ e, R. Livi, and A. Politi, 2013: Covariant Lyapunov vectors.J. Phys. A-Math. Theor., 46, 254005

work page 2013
[4]

Lorenz, E. N. and K. A. Emanuel, 1998: Optimal sites for suplementary weather observations: simulation with a small model. J. Atmos. Sci., 55, 399–414

work page 1998
[5]

Whitaker, J. S. and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924. 8

work page 2002

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Farchi, T

Bocquet, M., A. Farchi, T. S. Finn, C. Durand, S. Cheng, Y. Chen, I. Pasmans, and A. Carrassi, 2024: Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemble. Chaos, 34, 091104

work page 2024

[2] [2]

Bocquet, M. and P. Sakov, 2013: Joint state and parameter estimation with an iterative ensemble Kalman smoother. Nonlinear Proc. Geoph., 20, 803–818

work page 2013

[3] [3]

Chat´ e, R

Ginelli, F., H. Chat´ e, R. Livi, and A. Politi, 2013: Covariant Lyapunov vectors.J. Phys. A-Math. Theor., 46, 254005

work page 2013

[4] [4]

Lorenz, E. N. and K. A. Emanuel, 1998: Optimal sites for suplementary weather observations: simulation with a small model. J. Atmos. Sci., 55, 399–414

work page 1998

[5] [5]

Whitaker, J. S. and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924. 8

work page 2002