REVIEW 3 major objections 6 minor 28 references
Introducing Interval Neural Networks for Uncertainty-Aware System Identification
T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read By widening a trained model's weights into intervals, a system-identification network produces prediction intervals that hit target coverage without Bayesian assumptions.
desk verdict Useful interval-extension of LSTM/NODE for SysID, but the reported coverage is teacher-forced and the state-reset design undercuts the closed-loop claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is interval arithmetic over interval-valued learnable parameters: for a pre-trained parameter $\theta^*$, the network learns an interval $[\theta^* - \Delta, \theta^* + \Delta]$, and every layer maps interval inputs to interval outputs using interval addition, subtraction, and multiplication. For recurrent and ODE architectures, the state and feedback intervals are collapsed back to the pre-trained point values at each time step, so the generated prediction interval reflects parameter-interval uncertainty rather than accumulated state uncertainty. Training minimizes the LRQR-W loss, a relaxed quantile regression loss with a width penalty, while the elasticity measure $\bar{r} = \lVert \theta - \underline{\theta} \rVert / \lVert \theta^* \rVert$ identifies which parameters carry the most uncertainty.
What would settle it
Run the interval model on a dynamical system with known feedback-amplified noise, such as an unstable or strongly recurrent linear system whose true output variance grows with horizon, and measure PICP at increasing horizons: if coverage falls as the horizon grows while the point model's RMSE stays flat, the state-reset mechanism is the cause.
Extended reading notes
Core claim
The paper's central claim is that interval-valued parameters, trained around a pre-trained point model, are enough to generate prediction intervals for recurrent and neural-ODE system identifiers without probabilistic assumptions. Each pretrained parameter is replaced by an interval centered on its point value, and interval arithmetic propagates these intervals through the network to produce an interval output. The interval widths are learned by minimizing a relaxed quantile regression loss that rewards coverage and penalizes width, with the nonnegativity of widths enforced by parameterizing them with either an absolute-value or ReLU function. The authors report that the resulting intervals track target coverage on three system-identification benchmarks, and they introduce an 'elasticity' measure that attributes uncertainty to specific parameters and lagged inputs.
Load-bearing premise
The load-bearing premise is that re-centering the interval network's internal state and feedback to the point model's values at every step still lets the output interval capture the real system's uncertainty; if the system's uncertainty grows through feedback, coverage can fail.
Editorial extensions
If this is right
- Users of LSTM or Neural-ODE system identifiers can attach prediction intervals to existing point models by training interval widths, without deriving a Bayesian posterior or running an ensemble.
- The loss hyperparameters $\alpha$ and $\lambda$ provide a direct dial between coverage probability and interval width, letting practitioners tune the model to a required miscoverage rate.
- The elasticity measure maps which weights or lagged inputs dominate the uncertainty, offering a diagnostic for where a dynamical model lacks information.
- The absolute-value parameterization is the recommended configuration for coverage, since it outperforms the ReLU-based variant in the reported experiments.
Reading between the lines
- Because the recurrent state and feedback output are pinned to pretrained point values at every step, the method captures parameter-interval uncertainty, not state-uncertainty propagation; a natural extension would let the state interval evolve and compare coverage on long-horizon predictions.
- Elasticity could serve as an input-lag selection heuristic for system identification, since it highlights which delayed outputs carry the most uncertainty.
- The same interval-parameterization trick could transfer to other recurrent or ODE-based models, such as latent ODEs or RNN controllers, and be tested for distribution-free uncertainty quantification in closed-loop settings.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a framework for constructing and learning interval neural networks (INNs) for uncertainty quantification in nonlinear system identification. The authors convert the learned parameters of pre-trained LSTM and Neural ODE models into interval-valued parameters, propagate intervals through the network using interval arithmetic, and train the interval widths with a quantile-style loss (LRQR-W) plus a width penalty. The resulting Interval LSTM (ILSTM) and Interval Neural ODE (INODE) are evaluated on three benchmark datasets (Heat Exchanger, MR-Damper, Hair Dryer), with Prediction Interval Coverage Probability (PICP) and Prediction Interval Normalized Average Width (PINAW) reported over five seeds. The paper also introduces an 'elasticity' metric that summarizes interval width per parameter, intended to identify which parameters contribute most to output uncertainty. The central claim is that ILSTM and INODE generate calibrated prediction intervals for recurrent and ODE-based system identification models without probabilistic assumptions, with INODE-2 showing the best coverage.
Significance. If the central claim held, the framework would be a useful lightweight alternative to probabilistic UQ for recurrent and ODE system identification models, requiring only interval arithmetic and a standard optimizer. The paper provides concrete architectural extensions, an explicit training algorithm, open-source code, and honest out-of-sample reporting of PICP/PINAW with standard deviations over five seeds. The out-of-sample evaluation is appropriate and not circular. However, the main empirical support for the central claim is weakened by a load-bearing design choice: the interval recurrent states and outputs are reset to point-model values at every time step, so the reported coverage is conditional one-step-ahead coverage rather than closed-loop simulation coverage. As a result, the current evidence does not support the claim that the generated intervals quantify the uncertainty of the dynamical system in simulation mode. The contribution is plausibly salvageable by either propagating interval states in closed loop or explicitly reframing the claim as one-step-ahead parameter-interval uncertainty.
major comments (3)
- [Section IV-B, Eqs. (28)-(30); Section IV-C, Eq. (34)]
- [Section VI-A, paragraph after hyperparameter table]
- [Section VI-B, Eq. (42) and Fig. 4]
minor comments (6)
- [Section VI-A]
- [Algorithm 2 and Eq. (39)]
- [Eq. (37)]
- [Section II, Eqs. (4)-(6)]
- [Fig. 4 caption]
- [Table III]
Circularity Check
Elasticity re-labels fitted interval widths; state reset makes reported PI a conditional one-step measure rather than closed-loop SysID uncertainty.
-
renaming known result
[Section VI-B, Eq. (42)]
"The term ”elasticity” is introduced to quantify the level of uncertainty in an INN parameter, defined as follows: ¯r =∥θ−θ∥ ∥θ∗∥ (42). Examining the elasticity of the parameters reveals which parts of the designed NN have a greater influence on the uncertainty of the output prediction."
By construction θ = θ* − Δ and θ = θ* + Δ, so r̄ = 2‖Δ‖/‖θ*‖: the normalized magnitudes of the very interval-width parameters optimized in Eq. (35). The later 'finding' that the y(k−2) weights are most uncertain is a direct read-off of the fitted Δ values, not an independent empirical discovery about underlying uncertainty causes. This is a description of the optimization result renamed as elasticity; it does not affect the coverage claim but is not an independent explanation.
-
self definitional
[Section III vs. Section IV-C, Eqs. (28)-(30) and (34)]
"During training and inference, the model operates in simulation mode, with x(k) updated after each prediction. ... As we have done in ILSTM, to generate a PI around the output of the pre-trained NODE, we update at each step INN output with: [y(k− 1),y(k− 1)] = [y(k− 1),y (k− 1)] (34)."
Eq. (34) (and Eqs. (28)–(30) for ILSTM) resets the interval hidden/cell state and output to the pre-trained point model's crisp values at every step. Hence the regressor x(k) in Eq. (8) is always crisp, and no interval state is ever fed back into the recurrence. The reported PICP is therefore, by construction, a conditional one-step-ahead coverage computed around point-model states, not the coverage of an autonomously simulated dynamical system whose uncertainty propagates through feedback. The 'simulation mode' framing claims more than the inference equations actually implement.
full rationale
The central coverage results in Table III are evaluated on held-out test data, so the PICP values are not merely the training loss restated; the LRQR-W objective is optimized in-sample and coverage is measured out-of-sample, which is legitimate evidence for the trained intervals. The self-citations ([8], [13], [14]) are motivational and do not carry the technical derivation. The main circular/definitional content is twofold. First, the 'elasticity' analysis in Section VI-B defines r̄ as the normalized interval width and then reports which weights have large r̄ as if this were a discovered property of the system; it is exactly the fitted Δ output of the optimization, so it is a renaming rather than an independent finding. Second, the inference equations for ILSTM and INODE deliberately reset the interval states to point-model states, so the generated prediction interval is defined as a conditional interval around crisp state trajectories; claiming 'simulation mode' coverage for the dynamical system is therefore not supported by the computed PICP as a closed-loop measure. These issues do not nullify the empirical coverage results, but they lower the claimed scope: the intervals capture one-step parameter-uncertainty around point-model states, and the elasticity narrative is a description of the fit. Score 3 reflects this partial circularity without treating the central coverage evaluation as fitting-labeled-as-prediction.
Assumptions & free parameters
free parameters (4)
- Interval widths Delta and Delta-bar for every learnable parameter of the pre-trained network =
not listed individually; inferred per dataset and seed by minimizing Eq. (35)
- Uncertainty rates r_h (hidden) and r_o (output) =
0.75/0.75, 1/1, or 1/0.2 depending on dataset and architecture (Table I)
- Width penalty weight lambda =
not reported
- Recurrent-weight intervals for MR-Damper =
0 for all recurrent weights U in ILSTM-1 and ILSTM-2
assumptions (5)
- standard math Interval arithmetic operations as defined in Section II provide a correct semantics for propagating parameter uncertainty through the network.
- domain assumption The sigmoid and tanh activations are applied to intervals as [sigma(lo), sigma(hi)] because they are monotonic.
- domain assumption A pre-trained point model theta* is a good representation of the system, and all predictive uncertainty can be captured by widening its parameters symmetrically.
- domain assumption Minimizing the LRQR-W loss (Eqs. 35-38) on training data yields prediction intervals with the desired coverage on test data.
- domain assumption Training and running the model in simulation mode, feeding the model's own outputs back as lagged inputs, is a valid protocol for these benchmarks.
invented entities (1)
-
Elasticity bar-r = ||theta-bar - theta-underline|| / ||theta*||
Cite this review
Pith. "Pith review of Introducing Interval Neural Networks for Uncertainty-Aware System Identification." pith.science (2026). https://pith.science/paper/GS4XF7GK
@misc{pith2026250418845,
author = {Pith},
title = {Pith review of: Introducing Interval Neural Networks for Uncertainty-Aware System Identification},
year = {2026},
howpublished = {\url{https://pith.science/paper/GS4XF7GK}},
note = {Machine review of arXiv:2504.18845}
}
read the original abstract
System Identification (SysID) is crucial for modeling and understanding dynamical systems using experimental data. While traditional SysID methods emphasize linear models, their inability to fully capture nonlinear dynamics has driven the adoption of Deep Learning (DL) as a more powerful alternative. However, the lack of uncertainty quantification (UQ) in DL-based models poses challenges for reliability and safety, highlighting the necessity of incorporating UQ. This paper introduces a systematic framework for constructing and learning Interval Neural Networks (INNs) to perform UQ in SysID tasks. INNs are derived by transforming the learnable parameters (LPs) of pre-trained neural networks into interval-valued LPs without relying on probabilistic assumptions. By employing interval arithmetic throughout the network, INNs can generate Prediction Intervals (PIs) that capture target coverage effectively. We extend Long Short-Term Memory (LSTM) and Neural Ordinary Differential Equations (Neural ODEs) into Interval LSTM (ILSTM) and Interval NODE (INODE) architectures, providing the mathematical foundations for their application in SysID. To train INNs, we propose a DL framework that integrates a UQ loss function and parameterization tricks to handle constraints arising from interval LPs. We introduce novel concept "elasticity" for underlying uncertainty causes and validate ILSTM and INODE in SysID experiments, demonstrating their effectiveness.
Figures
Reference graph
Works this paper leans on
-
[1]
Ljung, System Identification: Theory for the User
L. Ljung, System Identification: Theory for the User . Prentice Hall PTR, 1999
1999
-
[2]
A. Dankers, P. M. J. Van den Hof, X. Bombois, and P. S. C. Heuberger, “Identification of dynamic models in complex networks with prediction error methods: Predictor input selection,” IEEE Transactions on Auto- matic Control, vol. 61, no. 4, pp. 937–952, 2016
work page 2016
-
[3]
N. Zhou, J. Pierre, and J. Hauer, “Initial results in power system identification from injected probing signals using a subspace method,” IEEE Transactions on Power Systems , vol. 21, no. 3, pp. 1296–1302, 2006
work page 2006
-
[4]
Deep networks for system identification: A survey,
G. Pillonetto, A. Aravkin, D. Gedon, L. Ljung, A. H. Ribeiro, and T. B. Sch¨on, “Deep networks for system identification: A survey,”Automatica, vol. 171, p. 111907, 2025
2025
-
[5]
A novel federated learning framework for system identification,
E. Kec ¸eci, M. G¨uzelkaya, and T. Kumbasar, “A novel federated learning framework for system identification,” in 2024 8th International Artificial Intelligence and Data Processing Symposium (IDAP) . IEEE, 2024, pp. 1–6
work page 2024
-
[6]
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, p. 1735–1780, 1997
work page 1997
-
[7]
Neural ordinary differential equations,
R. T. Chen, Y . Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information pro- cessing systems, vol. 31, 2018
2018
-
[8]
Deep learning frameworks to learn prediction and simulation focused control system models,
T. Tuna, A. Beke, and T. Kumbasar, “Deep learning frameworks to learn prediction and simulation focused control system models,” Applied Intelligence, vol. 52, no. 1, pp. 662–679, 2022
2022
Show all 28 references
-
[9]
Neural ordinary differential equations for nonlinear system identification,
A. Rahman, J. Drgo ˇna, A. Tuor, and J. Strube, “Neural ordinary differential equations for nonlinear system identification,” in American Control Conference, 2022, pp. 3979–3984
2022
-
[10]
Enhancing continuous time series modelling with a latent ode-lstm approach,
C. Coelho, M. F. P. Costa, and L. L. Ferr ´as, “Enhancing continuous time series modelling with a latent ode-lstm approach,” Applied Mathematics and Computation, vol. 475, p. 128727, 2024
2024
-
[11]
A comprehensive framework for verification, validation, and uncertainty quantification in scientific com- puting,
C. J. Roy and W. L. Oberkampf, “A comprehensive framework for verification, validation, and uncertainty quantification in scientific com- puting,” Computer methods in applied mechanics and engineering , vol. 200, no. 25-28, pp. 2131–2144, 2011
2011
-
[12]
A review of uncertainty quantification in deep learning: Techniques, applications and challenges,
M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya et al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information fusion, vol. 76, pp. 243–297, 2021
2021
-
[13]
Exploring zadeh’s general type- 2 fuzzy logic systems for uncertainty quantification,
Y . G¨uven, A. K¨okl¨u, and T. Kumbasar, “Exploring zadeh’s general type- 2 fuzzy logic systems for uncertainty quantification,” IEEE Transactions on Fuzzy Systems , 2024
2024
-
[14]
Odyssey of interval type-2 fuzzy logic systems: Learning strategies for uncertainty quantification,
A. K ¨okl¨u, Y . G¨uven, and T. Kumbasar, “Odyssey of interval type-2 fuzzy logic systems: Learning strategies for uncertainty quantification,” IEEE Transactions on Fuzzy Systems , 2024
2024
-
[15]
High-quality prediction intervals for deep learning: A distribution-free, ensembled approach,
T. Pearce, A. Brintrup, M. Zaki, and A. Neely, “High-quality prediction intervals for deep learning: A distribution-free, ensembled approach,” in Proceedings of the International Conference on Machine Learning , vol. 80, 2018, pp. 4075–4084
2018
-
[16]
Compre- hensive review of neural network-based prediction intervals and new advances,
A. Khosravi, S. Nahavandi, D. Creighton, and A. F. Atiya, “Compre- hensive review of neural network-based prediction intervals and new advances,” IEEE Transactions on Neural Networks , vol. 22, no. 9, pp. 1341–1356, 2011
2011
-
[17]
Assessments of epistemic uncertainty using gaussian stochastic weight averaging for fluid-flow regression,
M. Morimoto, K. Fukami, R. Maulik, R. Vinuesa, and K. Fukagata, “Assessments of epistemic uncertainty using gaussian stochastic weight averaging for fluid-flow regression,” Physica D: Nonlinear Phenomena , vol. 440, p. 133454, 2022
2022
-
[18]
Universal approximation theorem for interval neural networks,
M. R. Baker and R. B. Patil, “Universal approximation theorem for interval neural networks,” Reliable Computing , vol. 4, pp. 235–239, 1998
1998
-
[19]
Understand- ing certified training with interval bound propagation,
Y . Mao, M. N. M ¨uller, M. Fischer, and M. Vechev, “Understand- ing certified training with interval bound propagation,” arXiv preprint arXiv:2306.10426, 2023
2023 arXiv
-
[20]
Neural network model for imprecise regression with interval dependent variables,
K. Tretiak, G. Schollmeyer, and S. Ferson, “Neural network model for imprecise regression with interval dependent variables,” Neural Networks, vol. 161, pp. 550–564, 2023
2023
-
[21]
Efficient training of interval neural networks for imprecise training data,
J. Sadeghi, M. de Angelis, and E. Patelli, “Efficient training of interval neural networks for imprecise training data,” Neural Networks, vol. 118, pp. 338–351, 2019
2019
-
[22]
Interval neural networks: Uncertainty scores,
L. Oala, C. Heiß, J. Macdonald, M. M ¨arz, W. Samek, and G. Ku- tyniok, “Interval neural networks: Uncertainty scores,” arXiv preprint arXiv:2003.11566, 2020
2003 arXiv
-
[23]
Interval arithmetic: From principles to implementation,
T. Hickey, Q. Ju, and M. H. Van Emden, “Interval arithmetic: From principles to implementation,” J. ACM , vol. 48, no. 5, p. 1038–1068, 2001
2001
-
[24]
Anodev2: A coupled neural ode framework,
T. Zhang, Z. Yao, A. Gholami, J. E. Gonzalez, K. Keutzer, M. W. Mahoney, and G. Biros, “Anodev2: A coupled neural ode framework,” Advances in Neural Information Processing Systems , vol. 32, 2019
2019
-
[25]
Relaxed quantile regression: Prediction intervals for asymmetric noise,
T. Pouplin, A. Jeffares, N. Seedat, and M. van der Schaar, “Relaxed quantile regression: Prediction intervals for asymmetric noise,” arXiv preprint arXiv:2406.03258, 2024
2024 arXiv
-
[26]
Identification of hammerstein systems without explicit parameterisation of non-linearity,
J. Wang, A. Sano, T. Chen, and B. Huang, “Identification of hammerstein systems without explicit parameterisation of non-linearity,”International Journal of Control , vol. 82, no. 5, pp. 937–952, 2009
2009
-
[27]
Daisy: Database for the identification of systems,
B. De Moor, R. D. Bie, I. Lemahieu, and J. Renders, “Daisy: Database for the identification of systems,” http://homes.esat.kuleuven.be/ smc/daisy/, 1997, accessed 13 Jan 2025
1997
-
[28]
Short-term load and wind power forecasting using neural network-based prediction intervals,
H. Quan, D. Srinivasan, and A. Khosravi, “Short-term load and wind power forecasting using neural network-based prediction intervals,” IEEE Transactions on Neural Networks and Learning Systems , vol. 25, no. 2, pp. 303–315, 2014
2014
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.