{"id":"0193d382-fc1e-42fe-a2b9-c3ebe791b9b8","arxiv_id":"2504.18845","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Interval LSTM and Interval Neural ODE models, trained with a quantile-style loss around pre-trained point models, produce prediction intervals that reach near-target coverage on three system identification benchmarks.","lead":"This paper converts already-trained neural networks into interval-valued versions that output ranges instead of single predictions, and tests them on three benchmark system identification tasks. It shows the interval versions can hit target coverage rates on the test data, and analyzes which weights carry the uncertainty.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"State-reset in Eqs. (28)-(30) and (34) makes the reported PICP a conditional, one-step-ahead measure; closed-loop simulation coverage is not demonstrated and may be much worse.","rationale":"The reader's weakest-assumption analysis identified exactly the state-reset issue in Eqs. (28)-(30) and (34), and my reading agrees. This is the single most load-bearing concern because it determines what the reported PICP actually measures. If the intervals are conditioned on the point model's crisp state at each time step, they do not capture the compounding uncertainty of the dynamical system in closed-loop simulation, which is the stated use case. The paper explicitly presents these resets, so the issue is not a hidden consistency flaw but an unacknowledged scope limitation: the method, as described, provides teacher-forced or one-step-ahead conditional intervals, not full simulation-mode prediction intervals. The empirical coverage in Table III cannot resolve this because the evaluation uses the same crisp-state construction. A concrete simulation-mode rerun would settle whether the concern actually degrades coverage. Secondary issues, such as the post-hoc zeroing of recurrent-weight intervals for MR-Damper, the incorrect 'strictly positive' claim about ReLU, and the absence of external baselines, are real but less central; they do not change the verdict. The conditional verdict remains appropriate: the core idea is plausible and potentially useful for one-step-ahead UQ, but the claims must be reframed or the method extended to propagate state uncertainty before the SysID simulation-mode claim is accepted.","tokens_in":10096,"tokens_out":7605,"duration_ms":82040,"concrete_test":"Re-evaluate the trained INODE-2 and ILSTM-2 models on the same test splits in true simulation mode: remove the resets in Eqs. (28)-(30) and (34), feed the interval output y~(k-1) and the interval hidden state back into x(k) via Eq. (8), freeze the fitted Delta parameters, and recompute PICP for the 90% and 95% targets on all three datasets. If the mean PICP drops by more than 5 percentage points below the target on any dataset (e.g., below 85% for the 90% target), the reported coverage depends on the crisp-state reset and the simulation-mode claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the generated prediction intervals capture the uncertainty of the dynamical system in SysID. This requires the intervals to reflect uncertainty that propagates through state feedback. However, the inference equations deliberately reset the interval recurrent state to the pre-trained point model's state at every step: Eqs. (28)-(30) set [h(k-1), h(k-1)], [c(k-1), c(k-1)], and [y(k-1), y(k-1)] to crisp point values for ILSTM, and Eq. (34) resets the INODE output to the point prediction. Consequently, the regressor x(k) in Eq. (8) is always crisp, and no interval state is ever fed back into the recurrence. The PI at time k is computed from the point-model state plus interval parameters, so it represents only parameter-interval uncertainty in the instantaneous map, not the state uncertainty that would accumulate in closed-loop simulation. The paper states in Section III that models operate in simulation mode, so the relevant target is the closed-loop prediction interval. With the state reset, the reported PICP in Table III is at best a conditional, teacher-forced coverage measure. If the system's uncertainty grows through feedback, true closed-loop coverage will degrade, potentially far below the targets. This is the load-bearing weakness in the empirical support for the central claim. The per-dataset zeroing of recurrent-weight intervals for MR-Damper is a secondary confound, but the state-reset issue is more fundamental.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a framework for constructing and learning interval neural networks (INNs) for uncertainty quantification in nonlinear system identification. The authors convert the learned parameters of pre-trained LSTM and Neural ODE models into interval-valued parameters, propagate intervals through the network using interval arithmetic, and train the interval widths with a quantile-style loss (LRQR-W) plus a width penalty. The resulting Interval LSTM (ILSTM) and Interval Neural ODE (INODE) are evaluated on three benchmark datasets (Heat Exchanger, MR-Damper, Hair Dryer), with Prediction Interval Coverage Probability (PICP) and Prediction Interval Normalized Average Width (PINAW) reported over five seeds. The paper also introduces an 'elasticity' metric that summarizes interval width per parameter, intended to identify which parameters contribute most to output uncertainty. The central claim is that ILSTM and INODE generate calibrated prediction intervals for recurrent and ODE-based system identification models without probabilistic assumptions, with INODE-2 showing the best coverage.","tokens_in":10327,"tokens_out":5677,"duration_ms":60515,"significance":"If the central claim held, the framework would be a useful lightweight alternative to probabilistic UQ for recurrent and ODE system identification models, requiring only interval arithmetic and a standard optimizer. The paper provides concrete architectural extensions, an explicit training algorithm, open-source code, and honest out-of-sample reporting of PICP/PINAW with standard deviations over five seeds. The out-of-sample evaluation is appropriate and not circular. However, the main empirical support for the central claim is weakened by a load-bearing design choice: the interval recurrent states and outputs are reset to point-model values at every time step, so the reported coverage is conditional one-step-ahead coverage rather than closed-loop simulation coverage. As a result, the current evidence does not support the claim that the generated intervals quantify the uncertainty of the dynamical system in simulation mode. The contribution is plausibly salvageable by either propagating interval states in closed loop or explicitly reframing the claim as one-step-ahead parameter-interval uncertainty.","major_comments":[{"comment":"","section":"Section IV-B, Eqs. (28)-(30); Section IV-C, Eq. (34)"},{"comment":"","section":"Section VI-A, paragraph after hyperparameter table"},{"comment":"","section":"Section VI-B, Eq. (42) and Fig. 4"}],"minor_comments":[{"comment":"","section":"Section VI-A"},{"comment":"","section":"Algorithm 2 and Eq. (39)"},{"comment":"","section":"Eq. (37)"},{"comment":"","section":"Section II, Eqs. (4)-(6)"},{"comment":"","section":"Fig. 4 caption"},{"comment":"","section":"Table III"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the venue and the core idea is potentially publishable, but the state-reset issue in Section IV is fundamental and must be resolved before the central claim can be accepted. The per-dataset zeroing of recurrent-weight intervals is an additional empirical confound that should be addressed head-on. The GitHub link is a valuable reproducibility asset and should be verified by the editor. The manuscript also needs careful proofreading; there are several typographical errors and at least one reference to a non-existent section."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Ferah and Kumbasar turn pre-trained LSTM and NODE SysID models into interval-parameter networks that output prediction intervals, trained with a relaxed quantile loss plus a width penalty. The extension is systematic: they derive interval arithmetic for the recurrent equations, propose a reparameterization (abs vs ReLU) to keep interval bounds valid, and introduce an 'elasticity' diagnostic that shows which weights carry the most interval width. That is genuinely new in the SysID setting, and the code is public. The empirical work is honest in the narrow sense: PICP and PINAW are reported over five seeds, and the results show INODE-2 often lands near the 90/95 targets.\n\nBut the central claim overreaches. The inference equations (28)-(30) and (34) reset the interval hidden, cell, and output states to the pre-trained point model's crisp values at every time step. So the interval around y(k) reflects only parameter-interval uncertainty in the instantaneous map, not state uncertainty accumulated through feedback. Since the point models are trained and evaluated in simulation mode, the natural question is closed-loop coverage; this paper reports one-step-ahead conditional coverage. If the system's uncertainty grows through feedback, closed-loop coverage could be much worse. That is load-bearing, not a nit. The per-dataset zeroing of recurrent-weight intervals for MR-Damper is a further confound; it is a post-hoc model choice that should have been justified or ablated.\n\nThe paper also misses external UQ baselines — no comparison with MC dropout, deep ensembles, or quantile regression. 'Effective' is only established against the targets, not against existing tools. The elasticity analysis reads the fitted deltas back as a discovery about the system; it is a description of the optimized solution, useful but not a finding. Minor issues: the ReLU parameterization is called 'strictly positive' when ReLU is only non-negative (which is actually sufficient for valid intervals), and Table III labels INODE-1 as 'ILSTM trained with ReLU' (copy-paste). The math is otherwise coherent.\n\nI would send this to peer review, not desk reject. A serious referee should push for (a) either propagating interval states or re-labeling the claim as conditional, (b) an MR-Damper ablation on the recurrent-weight zeroing, and (c) one or two standard UQ baselines. With those, it would be a solid methodology paper. I wouldn't cite it in its current form for the closed-loop claim, but I'd cite it for the interval-parameter construction.","headline":"Useful interval-extension of LSTM/NODE for SysID, but the reported coverage is teacher-forced and the state-reset design undercuts the closed-loop claim.","tokens_in":10957,"tokens_out":2638,"would_cite":false,"duration_ms":27234,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By widening a trained model's weights into intervals, a system-identification network produces prediction intervals that hit target coverage without Bayesian assumptions.","keywords":["interval neural networks","uncertainty quantification","system identification","prediction intervals","long short-term memory","neural ordinary differential equations","interval arithmetic","relaxed quantile regression"],"falsifier":"Run the interval model on a dynamical system with known feedback-amplified noise, such as an unstable or strongly recurrent linear system whose true output variance grows with horizon, and measure PICP at increasing horizons: if coverage falls as the horizon grows while the point model's RMSE stays flat, the state-reset mechanism is the cause.","tokens_in":9787,"feed_emoji":"🎯","tokens_out":5211,"duration_ms":53487,"temperature":0.7,"pith_summary":"The paper proposes that uncertainty in a learned dynamical model can be represented not by probability distributions over weights but by intervals around each pretrained weight, and that propagating these intervals through the network with interval arithmetic produces prediction intervals. It extends two standard system-identification architectures, LSTM and Neural ODE, into Interval LSTM and Interval Neural ODE, and trains the interval widths with a coverage-aware loss plus a width penalty. On three benchmark datasets, MR-Damper, Heat Exchanger, and Hair Dryer, the intervals reach about 89–93 percent empirical coverage for a 90 percent target and 94–98 percent for a 95 percent target, with the absolute-value parameterization giving the most consistent results. A sympathetic reader would care because this offers distribution-free, calibration-controlled uncertainty estimates for safety-critical control models.","feed_headline":"Interval weights hit 90–95% coverage in learned system models","feed_subtitle":"No priors, no ensembles: widening pretrained LSTM and Neural ODE weights gives prediction intervals with controlled width.","key_machinery":"The load-bearing mechanism is interval arithmetic over interval-valued learnable parameters: for a pre-trained parameter $\\theta^*$, the network learns an interval $[\\theta^* - \\Delta, \\theta^* + \\Delta]$, and every layer maps interval inputs to interval outputs using interval addition, subtraction, and multiplication. For recurrent and ODE architectures, the state and feedback intervals are collapsed back to the pre-trained point values at each time step, so the generated prediction interval reflects parameter-interval uncertainty rather than accumulated state uncertainty. Training minimizes the LRQR-W loss, a relaxed quantile regression loss with a width penalty, while the elasticity measure $\\bar{r} = \\lVert \\theta - \\underline{\\theta} \\rVert / \\lVert \\theta^* \\rVert$ identifies which parameters carry the most uncertainty.","core_discovery":"The paper's central claim is that interval-valued parameters, trained around a pre-trained point model, are enough to generate prediction intervals for recurrent and neural-ODE system identifiers without probabilistic assumptions. Each pretrained parameter is replaced by an interval centered on its point value, and interval arithmetic propagates these intervals through the network to produce an interval output. The interval widths are learned by minimizing a relaxed quantile regression loss that rewards coverage and penalizes width, with the nonnegativity of widths enforced by parameterizing them with either an absolute-value or ReLU function. The authors report that the resulting intervals track target coverage on three system-identification benchmarks, and they introduce an 'elasticity' measure that attributes uncertainty to specific parameters and lagged inputs.","pith_inferences":["Because the recurrent state and feedback output are pinned to pretrained point values at every step, the method captures parameter-interval uncertainty, not state-uncertainty propagation; a natural extension would let the state interval evolve and compare coverage on long-horizon predictions.","Elasticity could serve as an input-lag selection heuristic for system identification, since it highlights which delayed outputs carry the most uncertainty.","The same interval-parameterization trick could transfer to other recurrent or ODE-based models, such as latent ODEs or RNN controllers, and be tested for distribution-free uncertainty quantification in closed-loop settings."],"forward_implications":["Users of LSTM or Neural-ODE system identifiers can attach prediction intervals to existing point models by training interval widths, without deriving a Bayesian posterior or running an ensemble.","The loss hyperparameters $\\alpha$ and $\\lambda$ provide a direct dial between coverage probability and interval width, letting practitioners tune the model to a required miscoverage rate.","The elasticity measure maps which weights or lagged inputs dominate the uncertainty, offering a diagnostic for where a dynamical model lacks information.","The absolute-value parameterization is the recommended configuration for coverage, since it outperforms the ReLU-based variant in the reported experiments."],"supporting_citations":[{"why":"Supplies the LSTM architecture that the paper extends into Interval LSTM.","marker":"[6]"},{"why":"Supplies the Neural ODE formulation that the paper extends into Interval NODE.","marker":"[7]"},{"why":"Supplies the interval arithmetic operations used to define interval neural network forward passes.","marker":"[23]"},{"why":"Supplies the LRQR-W loss used to train interval widths for coverage.","marker":"[25]"},{"why":"Provides the MR-Damper benchmark dataset used in the system identification comparison.","marker":"[26]"},{"why":"Provides the Heat Exchanger benchmark dataset used in the system identification comparison.","marker":"[27]"},{"why":"Provides the Hair Dryer benchmark dataset and the general system-identification framing.","marker":"[1]"},{"why":"Defines the PICP and PINAW metrics used to evaluate prediction interval quality.","marker":"[28]"}],"fun_headline_variants":["Interval neural nets quantify uncertainty without priors or ensembles","Interval LSTM and Neural ODEs hit target coverage in system ID","From point to interval weights: UQ for recurrent and ODE models","Interval weights give prediction intervals without probabilistic assumptions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that re-centering the interval network's internal state and feedback to the point model's values at every step still lets the output interval capture the real system's uncertainty; if the system's uncertainty grows through feedback, coverage can fail.","fun_headline_variants_meta":{"raw":{"variants":["Interval neural nets quantify uncertainty without priors or ensembles","Interval LSTM and Neural ODEs hit target coverage in system ID","From point to interval weights: UQ for recurrent and ODE models","Interval weights give prediction intervals without probabilistic assumptions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001402,"raw_usage":{"total_tokens":5659,"prompt_tokens":930,"completion_tokens":4729,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":4660}},"tokens_in":546,"tokens_out":4729,"duration_ms":28740,"temperature":1.0,"reasoning_tokens":4660,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:08:21.826214+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the interval model on a dynamical system with known feedback-amplified noise, such as an unstable or strongly recurrent linear system whose true output variance grows with horizon, and measure PICP at increasing horizons: if coverage falls as the horizon grows while the point model's RMSE stays flat, the state-reset mechanism is the cause.","supporting_citations":[{"cited_title":"Long short-term memory,","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM architecture that the paper extends into Interval LSTM."},{"cited_title":"Interval arithmetic: From principles to implementation,","cited_arxiv_id":null,"evidence_quote":"Supplies the interval arithmetic operations used to define interval neural network forward passes."},{"cited_title":"Identification of hammerstein systems without explicit parameterisation of non-linearity,","cited_arxiv_id":null,"evidence_quote":"Provides the MR-Damper benchmark dataset used in the system identification comparison."},{"cited_title":"Daisy: Database for the identification of systems,","cited_arxiv_id":null,"evidence_quote":"Provides the Heat Exchanger benchmark dataset used in the system identification comparison."},{"cited_title":"Short-term load and wind power forecasting using neural network-based prediction intervals,","cited_arxiv_id":null,"evidence_quote":"Defines the PICP and PINAW metrics used to evaluate prediction interval quality."}],"review_version":1}