Pith. sign in

REVIEW 3 major objections 6 minor 28 references

Introducing Interval Neural Networks for Uncertainty-Aware System Identification

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read By widening a trained model's weights into intervals, a system-identification network produces prediction intervals that hit target coverage without Bayesian assumptions.

desk verdict Useful interval-extension of LSTM/NODE for SysID, but the reported coverage is teacher-forced and the state-reset design undercuts the closed-loop claim. read the letter →

arxiv 2504.18845 v1 pith:GS4XF7GK submitted 2025-04-26 cs.LG cs.AI

classification cs.LGcs.AI
keywords intervalneuralnetworksuncertaintyquantificationsystemidentificationpredictionintervalslongshort-termmemoryordinarydifferentialequationsarithmeticrelaxedquantileregression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that uncertainty in a learned dynamical model can be represented not by probability distributions over weights but by intervals around each pretrained weight, and that propagating these intervals through the network with interval arithmetic produces prediction intervals. It extends two standard system-identification architectures, LSTM and Neural ODE, into Interval LSTM and Interval Neural ODE, and trains the interval widths with a coverage-aware loss plus a width penalty. On three benchmark datasets, MR-Damper, Heat Exchanger, and Hair Dryer, the intervals reach about 89–93 percent empirical coverage for a 90 percent target and 94–98 percent for a 95 percent target, with the absolute-value parameterization giving the most consistent results. A sympathetic reader would care because this offers distribution-free, calibration-controlled uncertainty estimates for safety-critical control models.

What carries the argument

The load-bearing mechanism is interval arithmetic over interval-valued learnable parameters: for a pre-trained parameter $\theta^*$, the network learns an interval $[\theta^* - \Delta, \theta^* + \Delta]$, and every layer maps interval inputs to interval outputs using interval addition, subtraction, and multiplication. For recurrent and ODE architectures, the state and feedback intervals are collapsed back to the pre-trained point values at each time step, so the generated prediction interval reflects parameter-interval uncertainty rather than accumulated state uncertainty. Training minimizes the LRQR-W loss, a relaxed quantile regression loss with a width penalty, while the elasticity measure $\bar{r} = \lVert \theta - \underline{\theta} \rVert / \lVert \theta^* \rVert$ identifies which parameters carry the most uncertainty.

What would settle it

Run the interval model on a dynamical system with known feedback-amplified noise, such as an unstable or strongly recurrent linear system whose true output variance grows with horizon, and measure PICP at increasing horizons: if coverage falls as the horizon grows while the point model's RMSE stays flat, the state-reset mechanism is the cause.

Watch

Extended reading notes

Core claim

The paper's central claim is that interval-valued parameters, trained around a pre-trained point model, are enough to generate prediction intervals for recurrent and neural-ODE system identifiers without probabilistic assumptions. Each pretrained parameter is replaced by an interval centered on its point value, and interval arithmetic propagates these intervals through the network to produce an interval output. The interval widths are learned by minimizing a relaxed quantile regression loss that rewards coverage and penalizes width, with the nonnegativity of widths enforced by parameterizing them with either an absolute-value or ReLU function. The authors report that the resulting intervals track target coverage on three system-identification benchmarks, and they introduce an 'elasticity' measure that attributes uncertainty to specific parameters and lagged inputs.

Load-bearing premise

The load-bearing premise is that re-centering the interval network's internal state and feedback to the point model's values at every step still lets the output interval capture the real system's uncertainty; if the system's uncertainty grows through feedback, coverage can fail.

Editorial extensions

If this is right

  • Users of LSTM or Neural-ODE system identifiers can attach prediction intervals to existing point models by training interval widths, without deriving a Bayesian posterior or running an ensemble.
  • The loss hyperparameters $\alpha$ and $\lambda$ provide a direct dial between coverage probability and interval width, letting practitioners tune the model to a required miscoverage rate.
  • The elasticity measure maps which weights or lagged inputs dominate the uncertainty, offering a diagnostic for where a dynamical model lacks information.
  • The absolute-value parameterization is the recommended configuration for coverage, since it outperforms the ReLU-based variant in the reported experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the recurrent state and feedback output are pinned to pretrained point values at every step, the method captures parameter-interval uncertainty, not state-uncertainty propagation; a natural extension would let the state interval evolve and compare coverage on long-horizon predictions.
  • Elasticity could serve as an input-lag selection heuristic for system identification, since it highlights which delayed outputs carry the most uncertainty.
  • The same interval-parameterization trick could transfer to other recurrent or ODE-based models, such as latent ODEs or RNN controllers, and be tested for distribution-free uncertainty quantification in closed-loop settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a framework for constructing and learning interval neural networks (INNs) for uncertainty quantification in nonlinear system identification. The authors convert the learned parameters of pre-trained LSTM and Neural ODE models into interval-valued parameters, propagate intervals through the network using interval arithmetic, and train the interval widths with a quantile-style loss (LRQR-W) plus a width penalty. The resulting Interval LSTM (ILSTM) and Interval Neural ODE (INODE) are evaluated on three benchmark datasets (Heat Exchanger, MR-Damper, Hair Dryer), with Prediction Interval Coverage Probability (PICP) and Prediction Interval Normalized Average Width (PINAW) reported over five seeds. The paper also introduces an 'elasticity' metric that summarizes interval width per parameter, intended to identify which parameters contribute most to output uncertainty. The central claim is that ILSTM and INODE generate calibrated prediction intervals for recurrent and ODE-based system identification models without probabilistic assumptions, with INODE-2 showing the best coverage.

Significance. If the central claim held, the framework would be a useful lightweight alternative to probabilistic UQ for recurrent and ODE system identification models, requiring only interval arithmetic and a standard optimizer. The paper provides concrete architectural extensions, an explicit training algorithm, open-source code, and honest out-of-sample reporting of PICP/PINAW with standard deviations over five seeds. The out-of-sample evaluation is appropriate and not circular. However, the main empirical support for the central claim is weakened by a load-bearing design choice: the interval recurrent states and outputs are reset to point-model values at every time step, so the reported coverage is conditional one-step-ahead coverage rather than closed-loop simulation coverage. As a result, the current evidence does not support the claim that the generated intervals quantify the uncertainty of the dynamical system in simulation mode. The contribution is plausibly salvageable by either propagating interval states in closed loop or explicitly reframing the claim as one-step-ahead parameter-interval uncertainty.

major comments (3)
  1. [Section IV-B, Eqs. (28)-(30); Section IV-C, Eq. (34)]
  2. [Section VI-A, paragraph after hyperparameter table]
  3. [Section VI-B, Eq. (42) and Fig. 4]
minor comments (6)
  1. [Section VI-A]
  2. [Algorithm 2 and Eq. (39)]
  3. [Eq. (37)]
  4. [Section II, Eqs. (4)-(6)]
  5. [Fig. 4 caption]
  6. [Table III]

Circularity Check

2 steps flagged · score 3.0 of 10

Elasticity re-labels fitted interval widths; state reset makes reported PI a conditional one-step measure rather than closed-loop SysID uncertainty.

  1. renaming known result [Section VI-B, Eq. (42)]
    "The term ”elasticity” is introduced to quantify the level of uncertainty in an INN parameter, defined as follows: ¯r =∥θ−θ∥ ∥θ∗∥ (42). Examining the elasticity of the parameters reveals which parts of the designed NN have a greater influence on the uncertainty of the output prediction."

    By construction θ = θ* − Δ and θ = θ* + Δ, so r̄ = 2‖Δ‖/‖θ*‖: the normalized magnitudes of the very interval-width parameters optimized in Eq. (35). The later 'finding' that the y(k−2) weights are most uncertain is a direct read-off of the fitted Δ values, not an independent empirical discovery about underlying uncertainty causes. This is a description of the optimization result renamed as elasticity; it does not affect the coverage claim but is not an independent explanation.

  2. self definitional [Section III vs. Section IV-C, Eqs. (28)-(30) and (34)]
    "During training and inference, the model operates in simulation mode, with x(k) updated after each prediction. ... As we have done in ILSTM, to generate a PI around the output of the pre-trained NODE, we update at each step INN output with: [y(k− 1),y(k− 1)] = [y(k− 1),y (k− 1)] (34)."

    Eq. (34) (and Eqs. (28)–(30) for ILSTM) resets the interval hidden/cell state and output to the pre-trained point model's crisp values at every step. Hence the regressor x(k) in Eq. (8) is always crisp, and no interval state is ever fed back into the recurrence. The reported PICP is therefore, by construction, a conditional one-step-ahead coverage computed around point-model states, not the coverage of an autonomously simulated dynamical system whose uncertainty propagates through feedback. The 'simulation mode' framing claims more than the inference equations actually implement.

full rationale

The central coverage results in Table III are evaluated on held-out test data, so the PICP values are not merely the training loss restated; the LRQR-W objective is optimized in-sample and coverage is measured out-of-sample, which is legitimate evidence for the trained intervals. The self-citations ([8], [13], [14]) are motivational and do not carry the technical derivation. The main circular/definitional content is twofold. First, the 'elasticity' analysis in Section VI-B defines r̄ as the normalized interval width and then reports which weights have large r̄ as if this were a discovered property of the system; it is exactly the fitted Δ output of the optimization, so it is a renaming rather than an independent finding. Second, the inference equations for ILSTM and INODE deliberately reset the interval states to point-model states, so the generated prediction interval is defined as a conditional interval around crisp state trajectories; claiming 'simulation mode' coverage for the dynamical system is therefore not supported by the computed PICP as a closed-loop measure. These issues do not nullify the empirical coverage results, but they lower the claimed scope: the intervals capture one-step parameter-uncertainty around point-model states, and the elasticity narrative is a description of the fit. Score 3 reflects this partial circularity without treating the central coverage evaluation as fitting-labeled-as-prediction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central method adds fitted interval widths around every weight of a pre-trained network, plus per-dataset initialization rates and a width penalty weight, plus a per-dataset constraint on recurrent weights for MR-Damper. The main assumptions are that interval arithmetic semantics are valid for this purpose, that activations can be extended monotonically, that symmetric weight widening covers the relevant uncertainty, and that the calibration loss transfers to test coverage. The only invented concept is elasticity, which is a descriptive statistic over the fitted quantities.

free parameters (4)
  • Interval widths Delta and Delta-bar for every learnable parameter of the pre-trained network = not listed individually; inferred per dataset and seed by minimizing Eq. (35)
    These are the actual learned quantities; each weight and bias interval is theta* +/- Delta. They are fit on training data against the LRQR-W loss.
  • Uncertainty rates r_h (hidden) and r_o (output) = 0.75/0.75, 1/1, or 1/0.2 depending on dataset and architecture (Table I)
    Initialization scale for the Delta values, chosen per dataset and presumably tuned on validation; the paper gives no selection procedure.
  • Width penalty weight lambda = not reported
    Hyperparameter balancing coverage loss and width penalty in Eq. (35); its value is not disclosed in the paper.
  • Recurrent-weight intervals for MR-Damper = 0 for all recurrent weights U in ILSTM-1 and ILSTM-2
    Hand-set per-dataset constraint: the Delta terms for recurrent weights were set to zero, meaning no gradient flow, only for the MR-Damper dataset. This post-hoc adjustment affects that dataset's reported coverage.
assumptions (5)
  • standard math Interval arithmetic operations as defined in Section II provide a correct semantics for propagating parameter uncertainty through the network.
    Sections II and IV define interval addition, multiplication, and the interval dot product, and the paper relies on these operations throughout the forward pass, including the dependency problem and its overestimation.
  • domain assumption The sigmoid and tanh activations are applied to intervals as [sigma(lo), sigma(hi)] because they are monotonic.
    Eqs. (24)-(25) apply sigma_sig and sigma_tanh to interval inputs without stating the interval extension. Monotonicity makes this valid, but it is never explicitly justified.
  • domain assumption A pre-trained point model theta* is a good representation of the system, and all predictive uncertainty can be captured by widening its parameters symmetrically.
    Eq. (21) transforms theta* into [theta* - Delta, theta* + Delta]; the method assumes this parameterization can express the uncertainty relevant for the SysID task.
  • domain assumption Minimizing the LRQR-W loss (Eqs. 35-38) on training data yields prediction intervals with the desired coverage on test data.
    The paper relies on the relaxed quantile regression result [25], but provides no calibration guarantee for the interval arithmetic forward pass with its specific parameterization tricks.
  • domain assumption Training and running the model in simulation mode, feeding the model's own outputs back as lagged inputs, is a valid protocol for these benchmarks.
    Section III and Algorithm 2 define training in simulation mode with feedback of predicted outputs; no citation or analysis supports that this yields calibrated intervals.
invented entities (1)
  • Elasticity bar-r = ||theta-bar - theta-underline|| / ||theta*||
    purpose: To quantify per-parameter uncertainty and explain which inputs or weights cause output uncertainty.
    An invented scalar statistic defined in Eq. (42) from the fitted Delta values. Its 'insights' are post-hoc readings of the optimization result, with no independent falsifiable prediction attached.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Introducing Interval Neural Networks for Uncertainty-Aware System Identification." pith.science (2026). https://pith.science/paper/GS4XF7GK

@misc{pith2026250418845,
  author       = {Pith},
  title        = {Pith review of: Introducing Interval Neural Networks for Uncertainty-Aware System Identification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GS4XF7GK}},
  note         = {Machine review of arXiv:2504.18845}
}
read the original abstract

System Identification (SysID) is crucial for modeling and understanding dynamical systems using experimental data. While traditional SysID methods emphasize linear models, their inability to fully capture nonlinear dynamics has driven the adoption of Deep Learning (DL) as a more powerful alternative. However, the lack of uncertainty quantification (UQ) in DL-based models poses challenges for reliability and safety, highlighting the necessity of incorporating UQ. This paper introduces a systematic framework for constructing and learning Interval Neural Networks (INNs) to perform UQ in SysID tasks. INNs are derived by transforming the learnable parameters (LPs) of pre-trained neural networks into interval-valued LPs without relying on probabilistic assumptions. By employing interval arithmetic throughout the network, INNs can generate Prediction Intervals (PIs) that capture target coverage effectively. We extend Long Short-Term Memory (LSTM) and Neural Ordinary Differential Equations (Neural ODEs) into Interval LSTM (ILSTM) and Interval NODE (INODE) architectures, providing the mathematical foundations for their application in SysID. To train INNs, we propose a DL framework that integrates a UQ loss function and parameterization tricks to handle constraints arising from interval LPs. We introduce novel concept "elasticity" for underlying uncertainty causes and validate ILSTM and INODE in SysID experiments, demonstrating their effectiveness.

Figures

Figures reproduced from arXiv: 2504.18845 by the authors.

Figure 2
Figure 2. Hair Dryer Dataset: UQ Performance Comparison [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. MR-Damper Dataset: UQ Performance Comparison [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 1
Figure 1. Heat Exchanger Dataset: UQ Performance Comparison [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Heatmaps of INN LPs from a SysID experiment on the Exchanger Dataset: (a) INDOE-1 and (b) INODE-2 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 20 canonical work pages

  1. [1]

    Ljung, System Identification: Theory for the User

    L. Ljung, System Identification: Theory for the User . Prentice Hall PTR, 1999

  2. [2]

    Identification of dynamic models in complex networks with prediction error methods: Predictor input selection,

    A. Dankers, P. M. J. Van den Hof, X. Bombois, and P. S. C. Heuberger, “Identification of dynamic models in complex networks with prediction error methods: Predictor input selection,” IEEE Transactions on Auto- matic Control, vol. 61, no. 4, pp. 937–952, 2016

  3. [3]

    Initial results in power system identification from injected probing signals using a subspace method,

    N. Zhou, J. Pierre, and J. Hauer, “Initial results in power system identification from injected probing signals using a subspace method,” IEEE Transactions on Power Systems , vol. 21, no. 3, pp. 1296–1302, 2006

  4. [4]

    Deep networks for system identification: A survey,

    G. Pillonetto, A. Aravkin, D. Gedon, L. Ljung, A. H. Ribeiro, and T. B. Sch¨on, “Deep networks for system identification: A survey,”Automatica, vol. 171, p. 111907, 2025

  5. [5]

    A novel federated learning framework for system identification,

    E. Kec ¸eci, M. G¨uzelkaya, and T. Kumbasar, “A novel federated learning framework for system identification,” in 2024 8th International Artificial Intelligence and Data Processing Symposium (IDAP) . IEEE, 2024, pp. 1–6

  6. [6]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, p. 1735–1780, 1997

  7. [7]

    Neural ordinary differential equations,

    R. T. Chen, Y . Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information pro- cessing systems, vol. 31, 2018

  8. [8]

    Deep learning frameworks to learn prediction and simulation focused control system models,

    T. Tuna, A. Beke, and T. Kumbasar, “Deep learning frameworks to learn prediction and simulation focused control system models,” Applied Intelligence, vol. 52, no. 1, pp. 662–679, 2022

Show all 28 references
  1. [9]

    Neural ordinary differential equations for nonlinear system identification,

    A. Rahman, J. Drgo ˇna, A. Tuor, and J. Strube, “Neural ordinary differential equations for nonlinear system identification,” in American Control Conference, 2022, pp. 3979–3984

  2. [10]

    Enhancing continuous time series modelling with a latent ode-lstm approach,

    C. Coelho, M. F. P. Costa, and L. L. Ferr ´as, “Enhancing continuous time series modelling with a latent ode-lstm approach,” Applied Mathematics and Computation, vol. 475, p. 128727, 2024

  3. [11]

    A comprehensive framework for verification, validation, and uncertainty quantification in scientific com- puting,

    C. J. Roy and W. L. Oberkampf, “A comprehensive framework for verification, validation, and uncertainty quantification in scientific com- puting,” Computer methods in applied mechanics and engineering , vol. 200, no. 25-28, pp. 2131–2144, 2011

  4. [12]

    A review of uncertainty quantification in deep learning: Techniques, applications and challenges,

    M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya et al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information fusion, vol. 76, pp. 243–297, 2021

  5. [13]

    Exploring zadeh’s general type- 2 fuzzy logic systems for uncertainty quantification,

    Y . G¨uven, A. K¨okl¨u, and T. Kumbasar, “Exploring zadeh’s general type- 2 fuzzy logic systems for uncertainty quantification,” IEEE Transactions on Fuzzy Systems , 2024

  6. [14]

    Odyssey of interval type-2 fuzzy logic systems: Learning strategies for uncertainty quantification,

    A. K ¨okl¨u, Y . G¨uven, and T. Kumbasar, “Odyssey of interval type-2 fuzzy logic systems: Learning strategies for uncertainty quantification,” IEEE Transactions on Fuzzy Systems , 2024

  7. [15]

    High-quality prediction intervals for deep learning: A distribution-free, ensembled approach,

    T. Pearce, A. Brintrup, M. Zaki, and A. Neely, “High-quality prediction intervals for deep learning: A distribution-free, ensembled approach,” in Proceedings of the International Conference on Machine Learning , vol. 80, 2018, pp. 4075–4084

  8. [16]

    Compre- hensive review of neural network-based prediction intervals and new advances,

    A. Khosravi, S. Nahavandi, D. Creighton, and A. F. Atiya, “Compre- hensive review of neural network-based prediction intervals and new advances,” IEEE Transactions on Neural Networks , vol. 22, no. 9, pp. 1341–1356, 2011

  9. [17]

    Assessments of epistemic uncertainty using gaussian stochastic weight averaging for fluid-flow regression,

    M. Morimoto, K. Fukami, R. Maulik, R. Vinuesa, and K. Fukagata, “Assessments of epistemic uncertainty using gaussian stochastic weight averaging for fluid-flow regression,” Physica D: Nonlinear Phenomena , vol. 440, p. 133454, 2022

  10. [18]

    Universal approximation theorem for interval neural networks,

    M. R. Baker and R. B. Patil, “Universal approximation theorem for interval neural networks,” Reliable Computing , vol. 4, pp. 235–239, 1998

  11. [19]

    Understand- ing certified training with interval bound propagation,

    Y . Mao, M. N. M ¨uller, M. Fischer, and M. Vechev, “Understand- ing certified training with interval bound propagation,” arXiv preprint arXiv:2306.10426, 2023

  12. [20]

    Neural network model for imprecise regression with interval dependent variables,

    K. Tretiak, G. Schollmeyer, and S. Ferson, “Neural network model for imprecise regression with interval dependent variables,” Neural Networks, vol. 161, pp. 550–564, 2023

  13. [21]

    Efficient training of interval neural networks for imprecise training data,

    J. Sadeghi, M. de Angelis, and E. Patelli, “Efficient training of interval neural networks for imprecise training data,” Neural Networks, vol. 118, pp. 338–351, 2019

  14. [22]

    Interval neural networks: Uncertainty scores,

    L. Oala, C. Heiß, J. Macdonald, M. M ¨arz, W. Samek, and G. Ku- tyniok, “Interval neural networks: Uncertainty scores,” arXiv preprint arXiv:2003.11566, 2020

  15. [23]

    Interval arithmetic: From principles to implementation,

    T. Hickey, Q. Ju, and M. H. Van Emden, “Interval arithmetic: From principles to implementation,” J. ACM , vol. 48, no. 5, p. 1038–1068, 2001

  16. [24]

    Anodev2: A coupled neural ode framework,

    T. Zhang, Z. Yao, A. Gholami, J. E. Gonzalez, K. Keutzer, M. W. Mahoney, and G. Biros, “Anodev2: A coupled neural ode framework,” Advances in Neural Information Processing Systems , vol. 32, 2019

  17. [25]

    Relaxed quantile regression: Prediction intervals for asymmetric noise,

    T. Pouplin, A. Jeffares, N. Seedat, and M. van der Schaar, “Relaxed quantile regression: Prediction intervals for asymmetric noise,” arXiv preprint arXiv:2406.03258, 2024

  18. [26]

    Identification of hammerstein systems without explicit parameterisation of non-linearity,

    J. Wang, A. Sano, T. Chen, and B. Huang, “Identification of hammerstein systems without explicit parameterisation of non-linearity,”International Journal of Control , vol. 82, no. 5, pp. 937–952, 2009

  19. [27]

    Daisy: Database for the identification of systems,

    B. De Moor, R. D. Bie, I. Lemahieu, and J. Renders, “Daisy: Database for the identification of systems,” http://homes.esat.kuleuven.be/ smc/daisy/, 1997, accessed 13 Jan 2025

  20. [28]

    Short-term load and wind power forecasting using neural network-based prediction intervals,

    H. Quan, D. Srinivasan, and A. Khosravi, “Short-term load and wind power forecasting using neural network-based prediction intervals,” IEEE Transactions on Neural Networks and Learning Systems , vol. 25, no. 2, pp. 303–315, 2014

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.