Pith. sign in

REVIEW 3 major objections 6 minor 52 references

IRNN: Innovation-driven Recurrent Neural Network for Time-Series Data Modeling and Prediction

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Giving RNNs access to their own past prediction errors, the way Kalman filters use innovations, improves multi-step time-series forecasting accuracy on standard benchmarks while adding only 0.73 percent more parameters.

desk verdict The stress-test note is right: for one-step prediction IRNN/IGRU/ILSTM are reparameterizations of the vanilla models, so the large reported gains are likely optimization effects from IU-BPTT, not evidence for a richer function class; still, the paper is worth refereeing because the training scheme and multi-step behavior are non-trivial. read the letter →

arxiv 2505.05916 v1 pith:WUBVUXW2 submitted 2025-05-09 cs.LG cs.AI

Yifan Zhou , Yibo Wang , Chao Shang This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0762M20
keywords recurrentneuralnetworkstimeseriesforecastingKalmanfilterinnovationbackpropagationthroughLSTMGRUmulti-stepprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that a recurrent neural network should be told about its own recent prediction mistakes, not just about the raw input history. Borrowing the Kalman filter's notion of innovation, the one-step-ahead error between the true output and the predicted output, the authors feed that error back into the hidden-state update, and into the gates and memory cells of LSTM and GRU. Because these innovation inputs depend on the network parameters, ordinary backpropagation through time cannot train the model directly, so the paper introduces input-updating BPTT (IU-BPTT), which alternates between refreshing the innovation signals and taking gradient steps. On four electricity transformer temperature datasets, the innovation-driven variants consistently beat their plain counterparts, with ILSTM reducing test MSE by roughly 19 to 31 percent while increasing parameter count by less than one percent. If this holds, it means a small architectural change, error feedback rather than a larger model, is enough to lift recurrent forecasting accuracy.

What carries the argument

The load-bearing object is the innovation signal $e_t = y_t - \hat y_t$, the immediate prediction error, fed back into the recurrence through a learnable gain matrix, together with the IU-BPTT training loop that alternates between refreshing innovations and updating parameters. In the LSTM and GRU versions, the same error enters the forget, input, output, reset, and update gates as well as the cell-state or candidate-state computation. For multi-step prediction the paper sets future innovations to zero, so the recursive predictor runs exactly like the base RNN beyond the first step; the mechanism is therefore most active in one-step-ahead and teacher-forced regimes.

What would settle it

A direct test is to train ILSTM under IU-BPTT with the innovation input $e_{t-1}$ replaced by a time-shifted or randomly permuted version of the same prediction error. If test MSE stays as low as with the true $e_{t-1}$, the gain is not the specific Kalman-style feedback; if MSE rises back toward LSTM levels, the immediate one-step error signal is the active component.

Watch

Extended reading notes

Core claim

The central claim is that injecting past prediction errors $e_{t-1}=y_{t-1}-\hat y_{t-1}$ as additional inputs into RNN, GRU, and LSTM recurrences turns them into self-correcting predictors, analogous to the Kalman filter's use of innovations in linear state-space models. In single-layer IRNN the learnable matrix $W_{xe}$ multiplying $e_{t-1}$ plays the Kalman-gain role. The same innovation signal is added to the gates and cell candidate of GRU and LSTM, yielding IGRU and ILSTM. Because innovations are functions of the current parameters, the paper trains with IU-BPTT, alternating parameter-gradient updates with recomputation of innovations. Experiments on ETTh1, ETTh2, ETTm1, and ETTm2 show that the innovation-driven variants improve multi-step MSE across the board, that ILSTM gives the largest gains with roughly 31 percent average reduction on ETTh1 and ETTh2, and that the architecture only adds 0.73 percent parameters.

Load-bearing premise

The reported multi-step gains rely on the approximation that future prediction errors are zero when the network rolls forward, even though those errors are unknown and typically not zero in practice.

Editorial extensions

If this is right

  • ILSTM reduces test MSE by roughly 31 percent on ETTh1 and ETTh2, 19 percent on ETTm1, and 27 percent on ETTm2 relative to LSTM, with only 0.73 percent more parameters.
  • One-step predictions improve by 40 to 60 percent; at the five-step horizon the improvement shrinks to about 10 percent, consistent with the error-correction signal fading as future innovations are zeroed.
  • IU-BPTT makes the augmentation trainable with off-the-shelf BPTT, at a per-epoch cost of roughly 15 to 20 percent that can be reduced by updating innovations every N epochs with little loss in training MSE.
  • IRNNs can beat the one-step performance of the naive predictor on datasets where plain RNN, GRU, or LSTM cannot, indicating that innovation feedback helps the model capture dynamics rather than merely interpolating.
  • The innovation input is a generic extension: applying it to any RNN variant, including gated architectures, yields consistent accuracy gains without changing the training routine qualitatively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to replace the zero-future-innovation assumption with a learned or autoregressive model of future innovations; the paper's own decay of gains from one-step to five-step horizons suggests such a model would be the main lever for longer-horizon gains.
  • The architecture is effectively an output-error feedback controller wrapped around a neural state-space model, so control-theoretic tools for closed-loop stability and disturbance rejection could be applied to predict when the gain matrix will help or hurt.
  • Because the innovation only requires the target variable to be observed at the current time, the idea should transfer to attention-based sequence models and state-space models such as Mamba, provided they are used in a forecasting setting rather than a pure sequence-generation setting.
  • A minimal test of the mechanism's contribution would be to train the same IRNN with the innovation input randomly shuffled in time; if gains persist, the effect is not specifically error correction, while if they vanish, the temporal alignment of the error signal is what matters.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an 'innovation-driven' recurrent neural network, where past prediction errors e_{t-1}=y_{t-1}-ŷ_{t-1} are fed back as additional inputs into the hidden-state updates of RNN, GRU, and LSTM, motivated by Kalman-filter innovation feedback. To train these networks, the authors introduce IU-BPTT, which alternates between standard BPTT parameter updates with the innovation trajectory frozen and recomputing innovations from the updated parameters. Experiments on the four ETT datasets report large one-step MSE reductions (40-60%) and smaller multi-step reductions (around 10% at five steps) over vanilla RNN/GRU/LSTM, with only a 0.73% parameter increase, together with an ablation study of which gates and cell modules benefit from innovations. The paper is clearly written and the training algorithm is precisely specified.

Significance. If the stated causal mechanism were established, the work would offer a simple and broadly applicable modification for recurrent time-series models. The IU-BPTT algorithm is practical and well described, the ablation study is thoughtful, and the ETT benchmark is standard. However, the central interpretation is weakened by an exact reparameterization equivalence between IRNN and the vanilla RNN at one-step horizons, and the empirical claims lack statistical support. The contribution remains potentially useful as an alternative training and data-augmentation scheme, but the paper must be reframed and supplemented with control experiments and significance testing before its claims are credible.

major comments (3)
  1. [Section III-B, Eq. (7)] For one-step prediction, the innovation term is exactly absorbable into the vanilla RNN update. Substituting e_{t-1}=y_{t-1}-(W_yx x_{t-1}+b_y) into (7) gives x_t = σ((W_xx - W_xe W_yx)x_{t-1} + W_xu u_t + (W_xy + W_xe)y_{t-1} + (b_x - W_xe b_y)), which is the vanilla RNN recurrence (2) with reparameterized weights. The same substitution applies gate-by-gate to IGRU and ILSTM in Table I. Consequently, the 40-60% one-step MSE reductions in Table IV cannot be attributed to a larger function class or to innovation information; they must arise from the altered parameterization or from the IU-BPTT optimization trajectory. The authors should acknowledge this equivalence explicitly and add control experiments that separate architectural capacity from optimization effects, for example by training a vanilla LSTM under a corresponding reparameterized loss or by evaluating whether an equivalent vanilla network with different initialization achieves similar one-step accuracy.
  2. [Table IV and Section IV-A2] The text states that all networks are trained 20 times, yet Table IV reports only point estimates of test MSE with no standard deviations, confidence intervals, or significance tests. Given the typical run-to-run variance of recurrent networks and the fact that several reported gains (especially at five steps) are only around 10%, these differences may not be statistically significant. The authors should report mean and standard deviation (or confidence intervals) over the 20 runs for each cell of Table IV and apply paired significance tests for the reported RNN-versus-IRNN comparisons.
  3. [Section III-B, Eq. (9) and Section IV-B2] Equation (9) and Algorithm 1 (lines 9-11) set future innovations to zero in both training and evaluation, so the reported multi-step gains cannot come from online error correction during the prediction horizon; they must come from the way the initial context is processed using past innovations. The narrative in Section IV-B2 and Figure 4 that describes 'error-correcting capability' during multi-step prediction is therefore misleading. The authors should clarify that, in the multi-step setting, innovations act as a form of context preprocessing rather than closed-loop feedback, and the visualization should be labeled accordingly.
minor comments (6)
  1. [Introduction, first paragraph] "attentions" should be "attention" in the phrase "have attracted immense attentions."
  2. [Table VII caption] The caption contains the typo "Nertwork"; it should read "Network."
  3. [Figure 8 caption] The caption "Each The legend indicates..." should be corrected to "The legend indicates..."
  4. [Throughout] The name "Naïve" is written with a combining diaeresis in several places; use a consistent plain-text spelling such as "Naive" for uniformity.
  5. [Algorithm 1] The input line "Training setDT" is missing a space; it should read "Training set DT."
  6. [Abstract and Section IV-C] The abstract's claim that improvements come "without increasing the training cost substantially" is contradicted by Table VI for the default N=1, where per-epoch time increases by 15-20%; the claim should be qualified to the case of a large innovation-updating interval, as supported by Table VII.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: IRNN is an empirical architecture with external benchmarks; the innovation term is absorbable into vanilla RNN weights, which is a correctness concern rather than a circular derivation.

full rationale

The paper's derivation chain is self-contained in the sense relevant to circularity: IRNN is motivated by the Kalman-filter innovation concept, defines e_t = y_t - ŷ_t, injects W_xe e_{t-1} into the state update, extends the construction to LSTM and GRU (Table I), and trains with IU-BPTT. The reported accuracy gains are measured on the external ETT benchmark; no parameter is fitted to the reported test metrics, so the predictions in Table IV do not reduce by construction to the model's inputs. The only self-citation, Ref. [24] by two of the same authors, is used as motivational background ('Incorporating e_t is a prerequisite to ensure optimality of KF, which is also the case for data-driven implementation of KF [24]') and is not load-bearing for the main architecture or the empirical claims. The paper also honestly discloses its key limitation in Eq. (9): 'it suffices to set future innovations to zero, inherently assuming that there is no output prediction error in future propagation.' One substantive technical caveat, which is not circularity, is that because ŷ_{t-1}=W_yx x_{t-1}+b_y, the innovation e_{t-1}=y_{t-1}-ŷ_{t-1} is an affine function of x_{t-1} and y_{t-1}; substituting into Eq. (7) yields the vanilla RNN update (2) with reparameterized weights (W_xx-W_xe W_yx, W_xy+W_xe, b_x-W_xe b_y). Similarly, the zero-future-innovation predictor (9) coincides with vanilla RNN (8). Thus the innovation term does not enlarge the function class for one-step or multi-step prediction; any observed gains must arise from IU-BPTT's frozen-innovation gradient updates or parameterization, not from added representational capacity. This undermines the causal interpretation of the central claim but does not make the paper circular: the results are externally benchmarked, the derivation is transparent, and the zero-innovation assumption is explicitly acknowledged. Score 2 reflects only the minor motivational self-citation, which is not load-bearing.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on learned innovation weight matrices, the zero-future-innovation approximation, the alternating training scheme, and the ETT benchmark family. No new physical entity is introduced. The paper contributes an architecture and a training heuristic rather than a parameter-free derivation.

free parameters (2)
  • Innovation weight matrices Wxe (IRNN), Wre/Wue/Wxe (IGRU), Wfe/Wie/Woe/Wce (ILSTM) = learned from data; exact values not reported
    These matrices map past prediction errors into state and gate updates and are the mechanism behind the reported MSE gains. They are unconstrained and fit to each dataset, playing the role of a learned Kalman gain.
  • Hyperparameters (hidden size 128, learning rates, batch size 64, Tp=24, Tf=5, N=1) = given in Table III
    Chosen by hand rather than by a systematic search. The reported gains could shift under different settings, although the comparisons keep these fixed between paired models.
assumptions (4)
  • ad hoc to paper Past prediction errors carry information about hidden-state errors and can improve future predictions when added as inputs.
    Section III-B motivates this by analogy to the Kalman filter; no formal statement or proof is given for nonlinear RNNs.
  • ad hoc to paper Future innovations can be set to zero in multi-step prediction.
    Equation (9) sets Wxe times zero; the paper explicitly says this assumes no future output prediction error, which is usually false and limits long-horizon gains.
  • ad hoc to paper IU-BPTT alternation between parameter updates and innovation updates optimizes the intended objective.
    Section III-D and Algorithm 1: the gradient treats e_t as constant even though e_t depends on parameters; no convergence proof is provided.
  • domain assumption The ETT datasets are representative of real-world time-series prediction tasks.
    Section IV-A uses only the ETT family to support general claims about RNN variants; no other benchmark families are tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IRNN: Innovation-driven Recurrent Neural Network for Time-Series Data Modeling and Prediction." pith.science (2026). https://pith.science/paper/WUBVUXW2

@misc{pith2026250505916,
  author       = {Pith},
  title        = {Pith review of: IRNN: Innovation-driven Recurrent Neural Network for Time-Series Data Modeling and Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WUBVUXW2}},
  note         = {Machine review of arXiv:2505.05916}
}
read the original abstract

Many real-world datasets are time series that are sequentially collected and contain rich temporal information. Thus, a common interest in practice is to capture dynamics of time series and predict their future evolutions. To this end, the recurrent neural network (RNN) has been a prevalent and effective machine learning option, which admits a nonlinear state-space model representation. Motivated by the resemblance between RNN and Kalman filter (KF) for linear state-space models, we propose in this paper Innovation-driven RNN (IRNN), a novel RNN architecture tailored to time-series data modeling and prediction tasks. By adapting the concept of "innovation" from KF to RNN, past prediction errors are adopted as additional input signals to update hidden states of RNN and boost prediction performance. Since innovation data depend on network parameters, existing training algorithms for RNN do not apply to IRNN straightforwardly. Thus, a tailored training algorithm dubbed input updating-based back-propagation through time (IU-BPTT) is further proposed, which alternates between updating innovations and optimizing network parameters via gradient descent. Experiments on real-world benchmark datasets show that the integration of innovations into various forms of RNN leads to remarkably improved prediction accuracy of IRNN without increasing the training cost substantially.

Figures

Figures reproduced from arXiv: 2505.05916 by the authors.

Figure 1
Figure 1. Illustration of single-layer RNN (1a) and single-layer IRNN (1b) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. IU-BPTT for training single-layer IRNN. The updating of network [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Dataset segmentation and augmentation. The raw time-series trajectory [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Visualization of prediction results. We display the prediction results of LSTM and ILSTM on continuous data with length of 80 in two different time [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Training and validation loss curves on ETTh1. MSE losses are presented on a logarithmic scale. The minimal MSE in each loss curve is marked with [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: MSE loss curves of Single-layer IRNN with innovation updating [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Testing MSE of modified ILSTMs and IGRUs on ETTh1. Each The [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Testing MSE of modified LSTMs and GRUs on ETTh1. The legend [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 39 canonical work pages

  1. [1]

    Short-term residential load forecasting based on LSTM recurrent neural network,

    W. Kong, Z. Y . Dong, Y . Jia, D. J. Hill, Y . Xu, and Y . Zhang, “Short-term residential load forecasting based on LSTM recurrent neural network,” IEEE Transactions on Smart Grid , vol. 10, no. 1, pp. 841–851, 2019

    work page 2019

  2. [2]

    Accurate medium-range global weather forecasting with 3D neural networks,

    K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Accurate medium-range global weather forecasting with 3D neural networks,” Nature, vol. 619, no. 7970, pp. 533–538, 2023

    2023

  3. [3]

    Time-series forecasting with deep learning: A survey,

    B. Lim and S. Zohren, “Time-series forecasting with deep learning: A survey,” Philosophical Transactions of the Royal Society A , vol. 379, no. 2194, p. 20200209, 2021

    work page 2021

  4. [4]

    Summary and reflections on pedestrian trajectory prediction in the field of autonomous driving,

    Z. Fu, K. Jiang, C. Xie, Y . Xu, J. Huang, and D. Yang, “Summary and reflections on pedestrian trajectory prediction in the field of autonomous driving,” IEEE Transactions on Intelligent Vehicles , 2024

    work page 2024

  5. [5]

    MPM: Multi patterns memory model for short-term time series forecasting,

    D. Wang, R. Liu, C. Chen, and S. Li, “MPM: Multi patterns memory model for short-term time series forecasting,” IEEE Transactions on Knowledge and Data Engineering , 2024

    work page 2024

  6. [6]

    The importance of short lag-time in the runoff forecast- ing model based on long short-term memory,

    X. Chen, J. Huang, Z. Han, H. Gao, M. Liu, Z. Li, X. Liu, Q. Li, H. Qi, and Y . Huang, “The importance of short lag-time in the runoff forecast- ing model based on long short-term memory,” Journal of Hydrology , vol. 589, p. 125359, 2020

    work page 2020

  7. [7]

    Forecasting of noisy chaotic systems with deep neural networks,

    M. Sangiorgio, F. Dercole, and G. Guariso, “Forecasting of noisy chaotic systems with deep neural networks,” Chaos, Solitons & Fractals , vol. 153, p. 111570, 2021

    work page 2021

  8. [8]

    Time series analysis using autoregressive integrated moving average (ARIMA) models,

    B. K. Nelson, “Time series analysis using autoregressive integrated moving average (ARIMA) models,” Academic Emergency Medicine , vol. 5, no. 7, pp. 739–744, 1998

    work page 1998

Show all 52 references
  1. [9]

    Time series forecasting using artificial neural networks methodologies: A systematic review,

    A. Tealab, “Time series forecasting using artificial neural networks methodologies: A systematic review,”Future Computing and Informatics Journal, vol. 3, no. 2, pp. 334–340, 2018

    2018

  2. [10]

    Autoregressive models in environ- mental forecasting time series: A theoretical and application review,

    J. Kaur, K. S. Parmar, and S. Singh, “Autoregressive models in environ- mental forecasting time series: A theoretical and application review,” Environmental Science and Pollution Research , vol. 30, no. 8, pp. 19 617–19 641, 2023

    2023

  3. [11]

    Long Short-Term Memory,

    S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  4. [12]

    On the properties of neural machine translation: Encoder- decoder approaches,

    K. Cho, “On the properties of neural machine translation: Encoder- decoder approaches,” arXiv:1409.1259, 2014

    2014 arXiv

  5. [13]

    WITRAN: Water- wave information transmission and recurrent acceleration network for long-range time series forecasting,

    Y . Jia, Y . Lin, X. Hao, Y . Lin, S. Guo, and H. Wan, “WITRAN: Water- wave information transmission and recurrent acceleration network for long-range time series forecasting,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  6. [14]

    Temporal pattern attention for multivariate time series forecasting,

    S.-Y . Shih, F.-K. Sun, and H.-Y . Lee, “Temporal pattern attention for multivariate time series forecasting,” Machine Learning , vol. 108, pp. 1421–1441, 2019

    2019

  7. [15]

    A dual-stage attention-based recurrent neural network for time series prediction,

    Y . Qin, D. Song, H. Chen, W. Cheng, G. Jiang, and G. Cottrell, “A dual-stage attention-based recurrent neural network for time series prediction,” arXiv:1704.02971, 2017

    2017 arXiv

  8. [16]

    NGCU: A new RNN model for time-series data prediction,

    J. Wang, X. Li, J. Li, Q. Sun, and H. Wang, “NGCU: A new RNN model for time-series data prediction,” Big Data Research , vol. 27, p. 100296, 2022

    2022

  9. [17]

    Hierarchically gated recurrent neu- ral network for sequence modeling,

    Z. Qin, S. Yang, and Y . Zhong, “Hierarchically gated recurrent neu- ral network for sequence modeling,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  10. [18]

    DeepAR: Probabilistic forecasting with autoregressive recurrent networks,

    D. Salinas, V . Flunkert, J. Gasthaus, and T. Januschowski, “DeepAR: Probabilistic forecasting with autoregressive recurrent networks,” Inter- national Journal of Forecasting , vol. 36, no. 3, pp. 1181–1191, 2020

    2020

  11. [19]

    Learning generative RNN-ODE for collaborative time-series and event sequence forecasting,

    L. Li, J. Yan, Y . Zhang, J. Zhang, J. Bao, Y . Jin, and X. Yang, “Learning generative RNN-ODE for collaborative time-series and event sequence forecasting,” IEEE Transactions on Knowledge and Data Engineering , vol. 35, no. 7, pp. 7118–7137, 2022

    2022

  12. [20]

    Independently Recurrent Neural Network (IndRNN): Building a longer and deeper RNN,

    S. Li, W. Li, C. Cook, C. Zhu, and Y . Gao, “Independently Recurrent Neural Network (IndRNN): Building a longer and deeper RNN,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    2018

  13. [21]

    New approach to information fusion steady-state Kalman filtering,

    Z.-L. Deng, Y . Gao, L. Mao, Y . Li, and G. Hao, “New approach to information fusion steady-state Kalman filtering,” Automatica, vol. 41, no. 10, pp. 1695–1707, 2005

    2005

  14. [22]

    N. R. Draper and H. Smith, Applied Regression Analysis. John Wiley & Sons, 1998, vol. 326

    1998

  15. [23]

    A predictive-reactive method for improving the robustness of real-time data services,

    J. Oh and K.-D. Kang, “A predictive-reactive method for improving the robustness of real-time data services,” IEEE Transactions on Knowledge and Data Engineering , vol. 25, no. 5, pp. 974–986, 2012

    2012

  16. [24]

    Data-driven output predic- tion and control of stochastic systems: An innovation-based approach,

    Y . Wang, K. You, D. Huang, and C. Shang, “Data-driven output predic- tion and control of stochastic systems: An innovation-based approach,” Automatica, vol. 171, p. 111897, 2025

    2025

  17. [25]

    Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis,

    Z. Shao, F. Wang, Y . Xu, W. Wei, C. Yu, Z. Zhang, D. Yao, T. Sun, G. Jin, X. Cao et al. , “Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis,” IEEE Transactions on Knowledge and Data Engineering , 2024

    2024

  18. [26]

    Design of neural network-based estimator for tool wear modeling in hard turning,

    X. Wang, W. Wang, Y . Huang, N. Nguyen, and K. Krishnakumar, “Design of neural network-based estimator for tool wear modeling in hard turning,” Journal of Intelligent Manufacturing , vol. 19, pp. 383– 396, 2008

    2008

  19. [27]

    Model predictive control of unknown nonlinear dynamical systems based on recurrent neural networks,

    Y . Pan and J. Wang, “Model predictive control of unknown nonlinear dynamical systems based on recurrent neural networks,” IEEE Transac- tions on Industrial Electronics , vol. 59, no. 8, pp. 3089–3101, 2012

    2012

  20. [28]

    Machine learning- based predictive control of nonlinear processes. Part I: Theory,

    Z. Wu, A. Tran, D. Rincon, and P. D. Christofides, “Machine learning- based predictive control of nonlinear processes. Part I: Theory,” AIChE Journal, vol. 65, no. 11, p. e16729, 2019

    2019

  21. [29]

    Machine-learning-based predictive control of nonlinear pro- cesses. Part II: Computational implementation,

    ——, “Machine-learning-based predictive control of nonlinear pro- cesses. Part II: Computational implementation,” AIChE Journal, vol. 65, no. 11, p. e16734, 2019

    2019

  22. [30]

    Process structure-based recurrent neural network modeling for model predictive control of nonlinear processes,

    Z. Wu, D. Rincon, and P. D. Christofides, “Process structure-based recurrent neural network modeling for model predictive control of nonlinear processes,” Journal of Process Control , vol. 89, pp. 74–84, 2020

    2020

  23. [31]

    A new concept using LSTM neural networks for dynamic system identification,

    Y . Wang, “A new concept using LSTM neural networks for dynamic system identification,” in 2017 American Control Conference (ACC) . IEEE, 2017, pp. 5324–5329

    2017

  24. [32]

    A recurrent neural network- based identification of complex nonlinear dynamical systems: A novel structure, stability analysis and a comparative study,

    R. Shobana, R. Kumar, and B. Jaint, “A recurrent neural network- based identification of complex nonlinear dynamical systems: A novel structure, stability analysis and a comparative study,” Soft Computing , pp. 1–17, 2023

    2023

  25. [33]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. Re, “Efficiently modeling long sequences with structured state spaces,” in International Conference on Learning Representations, 2022

    2022

  26. [34]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  27. [35]

    Is Mamba effective for time series forecasting?

    Z. Wang, F. Kong, S. Feng, M. Wang, X. Yang, H. Zhao, D. Wang, and Y . Zhang, “Is Mamba effective for time series forecasting?” Neurocom- puting, vol. 619, p. 129178, 2025

    2025

  28. [36]

    Convergence study in extended Kalman filter- based training of recurrent neural networks,

    X. Wang and Y . Huang, “Convergence study in extended Kalman filter- based training of recurrent neural networks,” IEEE Transactions on Neural Networks, vol. 22, no. 4, pp. 588–600, 2011. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 12

    2011

  29. [37]

    Recurrent neural network training with convex loss and regularization functions by extended Kalman filtering,

    A. Bemporad, “Recurrent neural network training with convex loss and regularization functions by extended Kalman filtering,” IEEE Transac- tions on Automatic Control , vol. 68, no. 9, pp. 5661–5668, 2022

    2022

  30. [38]

    Deep state space models for time series forecasting,

    S. S. Rangapuram, M. W. Seeger, J. Gasthaus, L. Stella, Y . Wang, and T. Januschowski, “Deep state space models for time series forecasting,” Advances in Neural Information Processing systems , vol. 31, 2018

    2018

  31. [39]

    E. W. Kamen and J. K. Su, Introduction to Optimal Estimation. Springer Science & Business Media, 2012

    2012

  32. [40]

    An empirical exploration of recurrent network architectures,

    R. Jozefowicz, W. Zaremba, and I. Sutskever, “An empirical exploration of recurrent network architectures,” in International Conference on Machine Learning. PMLR, 2015, pp. 2342–2350

    2015

  33. [41]

    Survey on research of RNN-based spatio-temporal sequence prediction algorithms,

    W. Fang, Y . Chen, and Q. Xue, “Survey on research of RNN-based spatio-temporal sequence prediction algorithms,” Journal on Big Data , vol. 3, no. 3, p. 97, 2021

    2021

  34. [42]

    Backpropagation through time: What it does and how to do it,

    P. J. Werbos, “Backpropagation through time: What it does and how to do it,” Proceedings of the IEEE , vol. 78, no. 10, pp. 1550–1560, 1990

    1990

  35. [43]

    Chauvin and D

    Y . Chauvin and D. E. Rumelhart, Backpropagation: Theory, Architec- tures, and Applications . Psychology press, 2013

    2013

  36. [44]

    On-line learning algorithms for locally recurrent neural networks,

    P. Campolucci, A. Uncini, F. Piazza, and B. Rao, “On-line learning algorithms for locally recurrent neural networks,” IEEE Transactions on Neural Networks, vol. 10, no. 2, pp. 253–271, 1999

    1999

  37. [45]

    SegRNN: Segment recurrent neural network for long-term time series forecasting,

    S. Lin, W. Lin, W. Wu, F. Zhao, R. Mo, and H. Zhang, “SegRNN: Segment recurrent neural network for long-term time series forecasting,” arXiv:2308.11200, 2023

    2023 arXiv

  38. [46]

    Informer: Beyond efficient transformer for long sequence time-series forecasting,

    H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, 2021, pp. 11 106–11 115

    2021

  39. [47]

    Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting,

    T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting,” in International Conference on Machine Learning . PMLR, 2022, pp. 27 268–27 286

    2022

  40. [48]

    Deep time series models: A comprehensive survey and benchmark,

    Y . Wang, H. Wu, J. Dong, Y . Liu, M. Long, and J. Wang, “Deep time series models: A comprehensive survey and benchmark,” arXiv:2407.13278, 2024

    2024 arXiv

  41. [49]

    Foundation models for time series analysis: A tutorial and survey,

    Y . Liang, H. Wen, Y . Nie, Y . Jiang, M. Jin, D. Song, S. Pan, and Q. Wen, “Foundation models for time series analysis: A tutorial and survey,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2024, pp. 6555–6565

    2024

  42. [50]

    Adam: A method for stochastic optimization,

    D. P. Kingma, “Adam: A method for stochastic optimization,” arXiv:1412.6980, 2014

    2014 arXiv

  43. [51]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  44. [52]

    G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time Series Analysis: Forecasting and Control. John Wiley & Sons, 2015. Yifan Zhou received the B.S. degree in theoretical and applied mechanics from Tsinghua University, Beijing, China, in 2024, where he is currently p...

    2015

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.