Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Online learning of neural state-space models

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes an almost-sure convergence guarantee for a recursive Gauss–Newton online learning rule applied to encoder-based neural state-space models, enabling computationally cheap adaptation during operation.

desk verdict A credible recursive prediction-error scheme for encoder-based ANN-SS models with a convergence theorem imported from Ljung, but the theory's key condition is not verified for the simulation, and the empirical evidence is thinner than claimed. read the letter →

arxiv 2607.17614 v1 pith:ZAJMISSM submitted 2026-07-20 eess.SY cs.LGcs.SY

Bendegúz Györök , Tamás Péni , Maarten Schoukens , Roland Tóth This is my paper · ORCID
classification eess.SYcs.LGcs.SY
keywords onlineidentificationneuralstate-spacemodelsencodernetworkrecursivepredictionerrorconvergenceanalysissystemstochasticapproximationvehiclelateraldynamics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper extends classical recursive prediction-error identification to neural state-space models with encoder-based initial-state estimation. It proposes a batch-wise pipeline and a single-step recursive Gauss–Newton update, and proves that under standard conditions the parameter estimates converge almost surely to a stationary point of the limit criterion. The value is practical: encoder-based ANN-SS models, normally fit offline with multiple-shooting, could now adapt online with modest computation. The proof transfers known stochastic-approximation and ODE arguments, while the simulation on vehicle lateral dynamics shows the recursive scheme maintains accuracy while taking only about 1 ms per update.

What carries the argument

The recursion (4): θ_i = θ_{i−1} + μ_i R_i^{−1} ψ_i^T(θ_{i−1}) ϵ_i(θ_{i−1}), with R_i updated as a weighted running covariance. The encoder network Ψ_θ reconstructs the initial state of each batch from past IO data, making the predictor depend only on a finite memory window, which is what allows the classical convergence analysis to apply. The ODE-based argument identifies V̄ as a Lyapunov function for the limiting trajectory, giving the almost-sure result.

What would settle it

Run recursion (4) on a system with slow or no exponential forgetting (e.g., an oscillator with zero damping) and observe whether θ_i diverges or cycles; the theorem predicts convergence only when Condition 1 holds, so a clear failure on such a plant would define the boundary of the claim. Additionally, test the fully unknown-output-map setting (learning h_θ from data) and compare convergence with the vehicle simulation.

Watch

Extended reading notes

Core claim

The central claim is Theorem 2: the recursion (4) — a Gauss–Newton update with step size, inverse Hessian approximation R_i, gradient ψ_i of the predictor, and prediction error ϵ_i — converges with probability 1 to either a stationary point of the limiting cost V̄(θ) or to the boundary of the parameter set Θ, provided the data-generating system satisfies exponential forgetting of initial states and the predictor is Lipschitz. This gives encoder-based ANN-SS models the same kind of almost-sure convergence guarantee long available for classical recursive prediction-error methods, and it applies to a whole family of subspace-encoder model structures.

Load-bearing premise

The convergence proof assumes Condition 1 — that the real system exponentially forgets its initial state — which the paper neither verifies for the simulated vehicle nor discusses for nonlinear systems where it fails; the simulation also supplies the model with the true output structure, an unstated favorable prior.

Editorial extensions

If this is right

  • Online adaptation becomes feasible for encoder-based ANN-SS models at roughly 1 ms per batch update.
  • The convergence guarantee extends to the whole SUBNET family—LPV, Koopman, and model-augmentation structures—not just plain ANN-SS models.
  • The recursive scheme is robust to batch length, whereas the batch-wise method is sensitive to horizon and memory size.
  • With the recommended forgetting-factor choices, the scheme can track slowly time-varying dynamics.
  • The batch-wise pipeline provides a practical middle ground using a replay buffer when lower-latency recursive updates are not required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The simulation's choice to set the output map to the true linear structure ([1 0]x) is an unstated favorable prior; the claimed accuracy may not carry over to the fully unknown-output-map setting the paper frames as its target.
  • Condition 1 (exponential forgetting) is plausible for the stable Pacejka vehicle model but could fail for near-marginally-stable systems; a natural test is to estimate the forgetting exponent from data and check whether it satisfies the bound.
  • The proof leaves open the practical role of the regularization constant δ and the boundary-convergence case; users likely need to monitor whether iterations are in the interior of Θ.
  • Because the recursion only needs the current batch and a running R_i, the method should scale to embedded or resource-constrained controllers where storing a replay buffer is infeasible.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes two methods for online identification of encoder-based neural state-space (ANN-SS) models. The first is a batch-wise pipeline that retrains on a sliding replay buffer using a JAX-based Adam+L-BFGS optimizer and an encoder for state initialization. The second is a recursive Gauss-Newton scheme (Eq. (4)) that updates the parameters each time a batch arrives, using the same encoder-based predictor. The main theoretical claim (Theorems 1 and 2) is that, under a global exponential-forgetting condition on the data-generating system and a Lipschitz/uniform-boundedness condition on the predictor, the recursive parameter estimates converge almost surely to stationary points of a limiting mean-square prediction-error criterion, or to the boundary of the parameter set. The methods are demonstrated on a nonlinear vehicle lateral-dynamics benchmark, with reported convergence-speed and wall-clock comparisons.

Significance. If the convergence result is valid under its hypotheses, the recursive scheme provides a computationally cheap online counterpart to state-of-the-art offline encoder-based ANN-SS identification (SUBNET), with a classical recursive-prediction-error type guarantee. This is a useful extension of an active research area. The batch-wise pipeline is also a practical engineering contribution, and the authors provide a public implementation. However, the significance is limited by a substantial gap between the theoretical assumptions and the simulation study: Condition 1 is not verified, and in fact appears to be violated by the benchmark vehicle model. The theoretical framework itself is standard and, conditional on the assumptions, likely correct, but the paper as written does not establish that the theory covers its own validation example.

major comments (3)
  1. [IV-C, Eq. (10); V, Eqs. (18)-(20)] Condition 1 (Eq. (10)) requires the data-generating system to exponentially forget initial conditions uniformly over all initial states in any delta-ball. The simulation model with saturating Pacejka tire forces does not satisfy this. For two initial states deep in the saturated regime with the same input and noise, the tire forces become identical constants, so Delta(dot omega)=0 and Delta(dot v_y) = -v_x Delta(omega). Hence Delta(omega) remains constant and Delta(v_y) grows linearly; fourth moments of the output difference do not decay, contradicting (10). Thus Theorem 2's almost-sure convergence guarantee does not apply to the benchmark used for validation. The paper should either replace the example with a system satisfying Condition 1, or explicitly state and justify that the simulation is an empirical demonstration outside the theorem's coverage.
  2. [V, output-map definition] The state is defined as x_k = [v_{y,k}, omega_k]^T and the measured output as y_k = omega_k + e_k, so the true output map is h(x) = [0 1]x. The text then says h_theta is chosen to match the true output structure, i.e., \hat y_k = [1 0]\hat x_k, which extracts v_y, not omega. As written, the model output is the wrong state component, making the simulation an invalid test of identifying the described system. Correct the typo or clarify the state ordering/output map.
  3. [IV-C, proof of Theorem 1] The proof of Theorem 1 consists of stating that the batch loss satisfies Condition C1 of [21] and that `the proof of [21, Lemma 3.1] applies`. This is too terse for a central result. Conditions 1-2 do not transparently imply that the loss sequence is quasi-stationary with geometrically summable covariances, nor that the gradient sequence psi_i has the bounded moments needed for uniform convergence of g(theta) and \bar R(theta). Please provide a self-contained verification of the cited lemma's hypotheses, or state the intermediate mixing/moment assumptions explicitly.
minor comments (5)
  1. [III-B, paragraph after parameter estimation] The sentence claiming that `under mild assumptions, the convergence and consistency properties of the SUBNET approach are inherited as m -> infinity` is unsupported. Please either give precise assumptions and a theorem, or soften this to a heuristic remark.
  2. [IV-D, Eq. (17)] The implemented update uses (R_i + delta I)^{-1} instead of the R_i^{-1} analyzed in Theorem 2. The effect of the constant regularization delta on the convergence guarantee should be discussed; as it stands, the practical algorithm is not exactly the recursion covered by the theorem.
  3. [IV-C, Theorem 2(b)] The condition R_i > 0 is assumed, but with the recommended mu_0 = 1, Eq. (4b) gives R_1 = psi_1^T psi_1, which can be singular. State explicitly that mu_i in (0,1) and R_0 > 0, or note that regularization is needed to satisfy (b).
  4. [IV-C, Condition 2] Condition 2 requires the derivative d/dtheta \hat gamma to satisfy a Lipschitz/uniform-bound condition. This is immediate for smooth tanh networks, but not for ReLU activations. The paper should state that the analysis assumes smooth activations, given that the differentiability assumption is only called `technical`.
  5. [V, Fig. 2] Minor presentation issues: the y-axis label `T est RMSE` has an unintended space; also, the AUCC values in Table III for divergent runs are reported as infinity, which is fine, but the table caption should define AUCC more explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the convergence theorem is an application of external stochastic-approximation theory and the simulation is evaluated on separate test data.

full rationale

The paper's derivation chain is not circular. The recursive update (4) is a Gauss-Newton-type parameter recursion on the batch prediction error. Theorem 1 is established by invoking Ljung's classical convergence analysis ([21, Lemma 3.1]), and Theorem 2 is proven by applying Ljung's recursive prediction-error identification theorem ([19, Theorem 2]) through an ODE/Lyapunov argument. These are external, established results, not self-citations, and they do not assume the paper's conclusion. The limiting criterion \bar V(\theta) is not a fitted quantity; it is defined as the almost-sure limit of the running average loss, and convergence to its stationary points is the theorem's claim, not an input. Condition 1 (Eq. 10) is an exponential-forgetting assumption on the data-generating system; even if the simulation model may violate it, that is a validity/coverage gap between theory and experiment, not a circular step. The simulation's choice of h_theta matching the true output structure ([1 0] x) is stated explicitly and evaluated on a separate test set; it is a favorable modeling prior, not a fitted parameter renamed as a prediction. Hyperparameters such as the step size and regularization are hand-selected and not fitted to the test data. The overlapping-author citations ([4], [17], [18]) are used for the encoder architecture, optimizer pipeline, and model-family examples, but they do not carry the convergence argument. No equation in the paper reduces, by construction or by self-citation, to its own inputs, so no circular step is identified.

Assumptions & free parameters 7 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard recursive identification theory plus conditions on the data-generating system (exponential forgetting, quasi-stationarity) and on the model (Lipschitzness, differentiability, compact parameter set). No fundamentally new entity is postulated; the encoder is inherited from SUBNET. Algorithm hyperparameters (forgetting factors, regularization, initial covariance, batch settings) are hand-chosen and their values matter for the reported performance.

free parameters (7)
  • mu_0 (initial step size) = 1
    User-specified constant in Eq. (15) that controls transient adaptation of the recursive update; the paper recommends mu_0 = 1.
  • lambda_0 (initial forgetting factor) = 0.75 (recommended range 0.75-0.95)
    Starting value of the forgetting-factor schedule lambda_i = bar_lambda * lambda_{i-1} + (1 - bar_lambda); affects early convergence speed.
  • bar_lambda (forgetting factor target) = 0.99
    Design parameter controlling memory length in the step-size schedule; for time-varying systems a constant lambda_i = bar_lambda < 1 is suggested.
  • delta (regularization for R_i) = 10^-3
    Added to R_i to ensure positive definiteness before Cholesky factorization; hand-chosen value affects the update direction.
  • R_0 (initial information matrix) = 10^4 I_n_theta
    Initial covariance-like matrix; a large identity is recommended in Sect. IV-D.
  • n (encoder lag) = not specified
    Length of the past input-output window used by the encoder in Eq. (2b). The method depends on n, but the simulation section never states its value, which hurts reproducibility.
  • Batch length N and replay memory m = N=25, m=20 in the main simulation; varied in Table III
    Settings of the online learning problem; strongly affect batch-wise performance, as shown by the N=5 divergence cases.
assumptions (7)
  • domain assumption The data-generating system is a discrete-time nonlinear state-space model (1) with i.i.d. finite-variance output noise, and the input is quasi-stationary and independent of the noise.
    Stated in Sect. II and used in Theorem 1. If the noise or input violates this, the limit criterion barV may not exist or may not be differentiable.
  • domain assumption Condition 1: exponential forgetting of initial conditions, E_e{||y_k - tilde_y_k||^4} < C(delta) lambda^{k-k0} (Eq. 10).
    Assumed for Theorem 1 and not verified for the simulated vehicle model. If the system does not forget its initial state exponentially, uniform convergence of the loss does not follow.
  • standard math Condition 2: the predictor and its gradient are Lipschitz in the past IO window (Eq. 11), automatically satisfied by Lipschitz ANN activations on a compact parameter set.
    Used to make the batch loss uniformly Lipschitz and to apply [21, Lemma 3.1]. Reasonable for tanh/ReLU networks with bounded weights.
  • domain assumption The encoder reconstructability map exists, so an initial state can be estimated from n past IO samples (Eq. 2b), following [4].
    Central to both batch-wise and recursive formulations. If the system is not reconstructable from a finite window, the initialized simulation is biased.
  • standard math Ljung's convergence theorems for recursive prediction-error identification (Refs. [19] and [21]) apply to recursion (4).
    Theorem 2 is proved by direct appeal to [19, Theorem 2] after identifying the associated ODE; the paper does not re-derive the martingale/ODE machinery.
  • ad hoc to paper Theta lies in a compact set Theta, projection keeps iterates inside, and the predictor is differentiable everywhere on Theta.
    Needed for Theorem 2 and for gradient computation. Standard in practice but not verified as a property of the ANN initialization.
  • ad hoc to paper For the batch-wise pipeline, convergence and consistency properties of SUBNET are inherited as m tends to infinity under unspecified mild assumptions.
    Stated in Sect. III-B without proof or precise assumptions. This is an unverified assertion, though it is not central to the recursive contribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Online learning of neural state-space models." pith.science (2026). https://pith.science/paper/ZAJMISSM

@misc{pith2026260717614,
  author       = {Pith},
  title        = {Pith review of: Online learning of neural state-space models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZAJMISSM}},
  note         = {Machine review of arXiv:2607.17614}
}
read the original abstract

Recent advances in deep-learning-based nonlinear system identification have led to encoder-based estimation of neural state-space (ANN-SS) models that achieve state-of-the-art performance in offline settings by estimating initial model states from past input-output data. These methods are typically used in multiple-shooting-based offline identification, and online learning of these models remains largely unexplored. This paper presents a batch-wise learning pipeline and a direct recursive identification algorithm for subspace encoder-based ANN-SS models. We provide convergence analysis of the recursive formulation and validate its performance through extensive simulation studies. The results demonstrate that the proposed approach enables computationally efficient online adaptation with high model accuracy.

Figures

Figures reproduced from arXiv: 2607.17614 by the authors.

Figure 1
Figure 1. Illustration of the training phases for batch-wise learning. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Test errors after each training phase with three online learning [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Average wall time of each training phase with three online learning [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 2 linked inside Pith

  1. [21]

    Convergence analysis of parametric identification methods,

    ——, “Convergence analysis of parametric identification methods,” IEEE Transactions on Automatic Control, vol. 23, no. 5, pp. 770–783, 1978

    1978

  2. [1]

    Deep Learning and System Identification,

    L. Ljung, C. Andersson, K. Tiels, and T. B. Sch ¨on, “Deep Learning and System Identification,” inProc. of the 21st IFAC World Congress, 2020, pp. 1175–1181

    2020

  3. [2]

    An L-BFGS-B Approach for Linear and Nonlinear System Identification Underℓ 1 and Group-Lasso Regularization,

    A. Bemporad, “An L-BFGS-B Approach for Linear and Nonlinear System Identification Underℓ 1 and Group-Lasso Regularization,” IEEE Transactions on Automatic Control, vol. 70, no. 7, pp. 4857– 4864, 2025

    2025

  4. [3]

    Continuous-time system identification with neural networks: Model structures and fitting criteria,

    M. Forgione and D. Piga, “Continuous-time system identification with neural networks: Model structures and fitting criteria,”European Journal of Control, vol. 59, pp. 69–81, 2021

    2021

  5. [4]

    Deep subspace encoders for nonlinear system identification,

    G. I. Beintema, M. Schoukens, and R. T ´oth, “Deep subspace encoders for nonlinear system identification,”Automatica, vol. 156, p. 111210, 2023

    2023

  6. [5]

    Fast Recursive Identification of State Space Models via Exploitation of Displacement Structure,

    Y . M. Cho, G. Xu, and T. Kailath, “Fast Recursive Identification of State Space Models via Exploitation of Displacement Structure,” Automatica, vol. 30, no. 1, pp. 45–59, 1994

    1994

  7. [6]

    Recursive identification of a nonlinear state space model,

    T. Wigren, “Recursive identification of a nonlinear state space model,” International Journal of Adaptive Control and Signal Processing, vol. 37, no. 2, pp. 447–473, 2023

    2023

  8. [7]

    Online learning of nonlinear parametric models under non-smooth regularization using EKF and ADMM,

    L. Frascati and A. Bemporad, “Online learning of nonlinear parametric models under non-smooth regularization using EKF and ADMM,” Automatica, vol. 185, p. 112787, 2026

    2026

Show all 23 references
  1. [8]

    On the adaptation of recurrent neural networks for system identification,

    M. Forgione, A. Muni, D. Piga, and M. Gallieri, “On the adaptation of recurrent neural networks for system identification,”Automatica, vol. 155, p. 111092, 2023

    2023

  2. [9]

    Meta-Learning of Neural State-Space Models Using Data From Similar Systems,

    A. Chakrabarty, G. Wichern, and C. R. Laughman, “Meta-Learning of Neural State-Space Models Using Data From Similar Systems,” in Proc. of the 22nd IFAC World Congress, 2023, pp. 1490–1495

    2023

  3. [10]

    On the adaptation of in-context learners for system identification,

    D. Piga, F. Pura, and M. Forgione, “On the adaptation of in-context learners for system identification,” inProc. of the 20th IFAC Sympo- sium on System Identification, 2024, pp. 277–282

    2024

  4. [11]

    Ljung and T

    L. Ljung and T. S ¨oderstr¨om,Theory and Practice of Recursive Identifi- cation, ser. Signal Processing, Optimization, and Control. Cambridge, MA, USA: MIT Press, 1987

    1987

  5. [12]

    Deep-Learning-Based Identification of LPV Models for Nonlinear Systems,

    C. Verhoek, G. I. Beintema, S. Haesaert, M. Schoukens, and R. T ´oth, “Deep-Learning-Based Identification of LPV Models for Nonlinear Systems,” inProc. of the 61st IEEE Conference on Decision and Control, 2022, pp. 3274–3280

    2022

  6. [13]

    Deep Identification of Nonlinear Systems in Koopman Form,

    L. C. Iacob, G. I. Beintema, M. Schoukens, and R. T ´oth, “Deep Identification of Nonlinear Systems in Koopman Form,” inProc. of the 60th IEEE Conference on Decision and Control, 2021, pp. 2288–2293

    2021

  7. [14]

    Learning- based augmentation of first-principle models: A linear fractional representation-based approach,

    J. H. Hoekstra, B. M. Gy ¨or¨ok, R. T ´oth, and M. Schoukens, “Learning- based augmentation of first-principle models: A linear fractional representation-based approach,”arXiv preprint arXiv:2602.17297, 2026

    2026

  8. [15]

    Adam: A Method for Stochastic Opti- mization,

    D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Opti- mization,” inProc. of the 3rd International Conference on Learning Representations, 2015, pp. 1–15

    2015

  9. [16]

    On the limited memory BFGS method for large scale optimization,

    D. C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization,”Mathematical Programming, vol. 45, pp. 503–528, 1989

    1989

  10. [17]

    Efficient identification of linear, parameter-varying, and nonlinear systems with noise models,

    A. Bemporad and R. T ´oth, “Efficient identification of linear, parameter-varying, and nonlinear systems with noise models,” arXiv:2504.11982, 2025

    2025 arXiv

  11. [18]

    Data-driven augmentation of first-principles models under constraint-free well-posedness and stability guarantees,

    B. Gy ¨or¨ok, R. Drenth, C. Verhoek, T. P ´eni, M. Schoukens, and R. T ´oth, “Data-driven augmentation of first-principles models under constraint-free well-posedness and stability guarantees,”arXiv preprint arXiv:2604.11421, 2026

    2026 arXiv

  12. [19]

    Analysis of a general recursive prediction error identifica- tion algorithm,

    L. Ljung, “Analysis of a general recursive prediction error identifica- tion algorithm,”Automatica, vol. 17, no. 1, pp. 89–99, 1981

    1981

  13. [20]

    Pearson Education, 1998

    ——,System Identification: Theory for the User, 2nd ed. Pearson Education, 1998

    1998

  14. [22]

    C. E. Rasmussen and C. K. I. Williams,Gaussian Processes for Machine Learning. MIT Press, 2005

    2005

  15. [23]

    H. B. Pacejka,Tyre and Vehicle Dynamics, 3rd ed. Butterworth- Heinemann, 2012

    2012

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.