Pith. sign in

REVIEW 1 cited by

Efficient identification of linear, parameter-varying, and nonlinear systems with noise models

T0 review · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A unified prediction-error-minimization framework separates deterministic dynamics from innovation noise and estimates both with L-BFGS-B and automatic differentiation, with consistency guarantees under classical assumptions.

arxiv 2504.11982 v1 pith:E46NCLG7 submitted 2025-04-16 math.OC cs.LGcs.SYeess.SY

Alberto Bemporad , Roland Tóth This is my paper · ORCID
classification math.OCcs.LGcs.SYeess.SY
keywords modelmodelsnoisenonlineargeneralidentificationlinearapproach
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

System identification is the art of building a mathematical model of a system from measured inputs and outputs. For linear systems, engineers have a standard way to separate the predictable part of the dynamics from the random noise that corrupts measurements. For nonlinear systems, this separation has been less clear. The paper shows that any nonlinear state-space model with innovation noise can be split into a deterministic process and a stochastic noise model, and that the inverse of the noise model has a simple analytic form. This extends the classical linear framework to nonlinear, parameter-varying, and linear models in one unified setting.\n\nThe authors parameterize the unknown functions with neural networks and estimate the combined model by minimizing the one-step-ahead prediction error. They use a constrained quasi-Newton method (L-BFGS-B) with automatic differentiation, which is much faster than the stochastic gradient methods usually applied to recurrent neural networks. On two benchmark systems, the proposed method reaches training times of seconds to a minute, compared with about ten hours reported for the SUBNET deep-learning baseline, with comparable or slightly better fit.\n\nThe paper also states consistency guarantees under standard assumptions: the input must be persistently exciting, the true system must lie in the model class, and the predictor must be stable. These conditions are assumed rather than verified for the benchmarks.
Extended reading notes

Core claim

The paper's central assertion: any nonlinear state-space system with innovation noise (1) can be separated into a deterministic process Go and a noise model Ho (Theorem 1), and the one-step-ahead predictor takes the form yhat(k|k-1) = Gouk + (I - Ho^{-1})(yk - Gouk), enabling joint PEM estimation of process and noise models with NN parameterizations. If true, this unifies LTI/LPV/NL identification with noise modeling.

Load-bearing premise

The consistency proof requires the data-generating system to lie in the chosen NN model set (Theta_o nonempty, Sec 5.2) and the global minimizer of the nonconvex PEM criterion to be found (Theorem 3 uses the global argmin (24), while the algorithm runs L-BFGS-B from random starts). If the true system is not representable, or the optimizer lands in a bad local minimum, the claimed consistency and benchmark accuracy do not follow.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The theory rests on standard PEM assumptions (quasi-stationary input, white innovation noise, predictor stability, representability of the true system) and on the global-minimizer idealization in the consistency theorem. The experimental claims additionally depend on hand-chosen regularization weights, neural network architectures, and the number of random initializations; the paper does not introduce new physical entities.

free parameters (4)
  • Regularization weights rho_theta and rho_w = rho_theta: 2e-4 / 2e-9 / 2e-8 / 2e-3; rho_w: 2e-8 / 2e-4 (per example)
    Hand-tuned per benchmark in Section 6; they control the regularization term (16) and affect which model wins on test data.
  • Number of random initial guesses = 100 (Sec 6.1), 500 (Sec 6.3.4)
    The best model on test data is selected, so this parameter directly inflates reported test BFR.
  • Neural network architectures (layers, neurons, activations) = scheduling map 2x6 sigmoid/swish; CMG scheduling 2x5 swish; NL plant 2-layer 15-10; CMG NL fx 2x10, fy 2x5
    Manual architecture choices per example; no automated selection.
  • Adam warm-start iterations = 1000
    Used only in some examples (Sec 6.1); fixed value, affects the local minimum found.
assumptions (5)
  • domain assumption The data-generating system satisfies Condition 1 (exponential forgetting in fourth moment).
    Assumed in Theorem 2; not verified for the benchmarks; needed for convergence of the prediction-error criterion.
  • domain assumption The predictor satisfies Condition 2 (stability w.r.t. past data and differentiability).
    Assumed; NN predictors are differentiable but exponential forgetting is not guaranteed.
  • domain assumption The true system belongs to the model set (Theta_o nonempty).
    Stated in Sec 5.2; necessary for consistency; NN universal approximation is not proven.
  • domain assumption Input is quasi-stationary and independent of white noise e.
    Standard PEM assumption, stated in Sec 2.1 and 5.1.
  • standard math Ljung's Lemma 3.1 and Lemma 4.1 (1978) apply.
    External theorems used for Theorem 2 and Theorem 3; the paper checks conditions only briefly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient identification of linear, parameter-varying, and nonlinear systems with noise models." pith.science (2026). https://pith.science/paper/E46NCLG7

@misc{pith2026250411982,
  author       = {Pith},
  title        = {Pith review of: Efficient identification of linear, parameter-varying, and nonlinear systems with noise models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E46NCLG7}},
  note         = {Machine review of arXiv:2504.11982}
}
read the original abstract

We present a general system identification procedure capable of estimating of a broad spectrum of state-space dynamical models, including linear time-invariant (LTI), linear parameter-varying} (LPV), and nonlinear (NL) dynamics, along with rather general classes of noise models. Similar to the LTI case, we show that for this general class of model structures, including the NL case, the model dynamics can be separated into a deterministic process and a stochastic noise part, allowing to seamlessly tune the complexity of the combined model both in terms of nonlinearity and noise modeling. We parameterize the involved nonlinear functional relations by means of artificial neural-networks (ANNs), although alternative parametric nonlinear mappings can also be used. To estimate the resulting model structures, we optimize a prediction-error-based criterion using an efficient combination of a constrained quasi-Newton approach and automatic differentiation, achieving training times in the order of seconds compared to existing state-of-the-art ANN methods which may require hours for models of similar complexity. We formally establish the consistency guarantees for the proposed approach and demonstrate its superior estimation accuracy and computational efficiency on several benchmark LTI, LPV, and NL system identification problems.

Figures

Figures reproduced from arXiv: 2504.11982 by the authors.

Figure 1
Figure 1. Separation of the data-generating system to a deterministic and stochastic [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Considered benchmark systems, whose simulation model is used for [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Prediction results of the self-scheduled LPV model with LTI noise model [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Online learning of neural state-space models

    eess.SY 2026-07 conditional novelty 6.0 of 10

    An encoder-based neural state-space model can be adapted online by batch retraining or by a recursive Gauss-Newton update that converges almost surely to stationary points under standard conditions.

Reference graph

Works this paper leans on

31 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    Beintema, M

    G.I. Beintema, M. Schoukens, and R. Tóth. Deep subspace encoders for nonlinear system identification. Automatica, 156:111210, 2023

    work page 2023

  2. [2]

    Bemporad

    A. Bemporad. Training recurrent neural networks by sequential least squares and the alternating direction method of multipliers. Automatica, 156:111183, October 2023

    work page 2023

  3. [3]

    Bemporad

    A. Bemporad. An L-BFGS-B approach for linear and nonlinear system identification under ℓ1- and group-lasso regularization. IEEE Transactions on Automatic Control , 2025. In press. Also available on arXiv at http: //arxiv.org/abs/2403.03827

    arXiv 2025

  4. [4]

    Billings

    S.A. Billings. Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains. Wiley, 2013

    work page 2013

  5. [5]

    Bloemers and R

    T. Bloemers and R. Tóth. Equations of motion of a control moment gyro- scope. Technical report, Eindhoven University of Technology, 2019

    work page 2019

  6. [6]

    Bradbury, R

    J. Bradbury, R. Frostig, P. Hawkins, M.J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang. JAX: composable transformations of Python+NumPy programs. 2018. 25

    work page 2018

  7. [7]

    R.H. Byrd, P. Lu, J. Nocedal, and C. Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5):1190–1208, 1995

    work page 1995

  8. [8]

    Candès, M.B

    E.J. Candès, M.B. Wakin, and S.P. Boyd. Enhancing sparsity by reweighted ℓ1 minimization. Journal of Fourier Analysis and Applications , (14):877– 905, 2008

    work page 2008

Show all 31 references
  1. [9]

    den Boef, P.B

    P. den Boef, P.B. Cox, and R. Tóth. LPVcore: MATLAB toolbox for LPV modelling, identification and control of non-linear systems. In Proc. of the 19th IFAC Symposium System Identification: learning models for decision and control, Padova, Italy, 2021

    2021

  2. [10]

    Forgione and D

    M. Forgione and D. Piga. dynoNet: A neural network architecture for learn- ing dynamical systems. International Journal of Adaptive Control and Signal Processing, 35(4):612–626, 2021

    2021

  3. [11]

    Giri and E.-W

    F. Giri and E.-W. Bai. Lecture Notes in Control and Information Sciences . Block-oriented Nonlinear System Identification. Springer-Germany, 2010

    2010

  4. [12]

    Glorot and Y

    X. Glorot and Y . Bengio. Understanding the difficulty of training deep feed- forward neural networks. In Proc. 13th Int. Conference on Artificial Intelli- gence and Statistics, pages 249–256. JMLR Workshop and Conference Pro- ceedings, 2010

    2010

  5. [13]

    Gonzalez and W

    J. Gonzalez and W. Yu. Non-linear system modeling using LSTM neural networks. In Proc. of the 2nd IFAC Conference on Modelling, Identification and Control of Nonlinear Systems, volume 51, pages 485–489, 2018

    2018

  6. [14]

    M. Jansson. Subspace identification and ARX modeling. IFAC Proceedings Volumes, 36(16):1585–1590, 2003

    2003

  7. [15]

    Kingma and J

    D.P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  8. [16]

    L. Ljung. Convergence analysis of parametric identification methods. IEEE Transactions on Automatic Control, 23(5):770–783, 1978

    1978

  9. [17]

    L. Ljung. System Identification, theory for the user . Prentice-Hall, 2nd edi- tion, 1999

    1999

  10. [18]

    L. Ljung. System Identification Toolbox for MATLAB. The Mathworks, Inc.,

  11. [19]

    Ljung, C

    L. Ljung, C. Andersson, K. Tiels, and T.B. Schön. Deep learning and system identification. In Proc. of the 21th IFAC World Congress, pages 1175–1181, 2020. 26

    2020

  12. [20]

    Masti and A

    D. Masti and A. Bemporad. Learning nonlinear state–space models using autoencoders. Automatica, 129:109666, 2021

    2021

  13. [21]

    Mulagaleti and A

    S.K. Mulagaleti and A. Bemporad. Combined learning of linear parameter- varying models and robust control invariant sets. 2024. submitted for publica- tion. Available on arXiv at https://arxiv.org/abs/2411.18166

    2024

  14. [22]

    Nied´ zwiecki

    M. Nied´ zwiecki. Identification of time-varying processes . John Wiley and Sons, 2000

    2000

  15. [23]

    Pillonetto, A

    G. Pillonetto, A. Aravkin, D. Gedon, L. Ljung, A.H. Ribeiro, and T.B. Schön. Deep networks for system identification: A survey. Automatica, 171:111907, 2025

    2025

  16. [24]

    Pillonetto, F

    G. Pillonetto, F. Dinuzzo, T. Chen, G. De Nicolao, and L. Ljung. Kernel methods in system identification, machine learning and function estimation: A survey. Automatica, 50(3):657–682, 2014

    2014

  17. [25]

    Ribeiro, K

    A.H. Ribeiro, K. Tiels, J. Umenberger, T.B. Schön, and L.A. Aguirre. On the smoothness of nonlinear system identification. Automatica, 121:109158, 2020

    2020

  18. [26]

    Schoukens and L

    J. Schoukens and L. Ljung. Nonlinear system identification: A user-oriented road map. IEEE Control Systems Magazine, 39(6):28–99, 2019

    2019

  19. [27]

    Suykens, T

    J.A.K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vande- walle. Least Squares Support Vector Machines. World Scientific, 2002

    2002

  20. [28]

    Suykens, J.P.L

    J.A.K. Suykens, J.P.L. Vandewalle, and B. De Moor. Nonlinear system iden- tification using neural networks. In Artificial Neural Networks for Modelling and Control of Non-Linear Systems. Springer, 1996

    1996

  21. [29]

    R. Tóth. Modeling and Identification of Linear Parameter-Varying Systems. Lecture Notes in Control and Information Sciences, V ol. 403. Springer, Hei- delberg, 2010

    2010

  22. [30]

    Verhoek, G.I

    C. Verhoek, G.I. Beintema, S. Haesaert, M. Schoukens, and R. Tóth. Deep- learning-based identification of LPV models for nonlinear systems. In Proc. of the 61st IEEE Conference on Decision and Control , pages 3274–3280, Cancun, Mexico, 2022. A Proof of Theorem 1 By taking wk =...

    2022

  23. [2001]

    https://www.mathworks.com/help/ident

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.