Pith. sign in

REVIEW 5 minor 74 references

Unsupervised Learning in Echo State Networks for Input Reconstruction

T0 review · 0 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Input reconstruction in echo state networks is an unsupervised task, not a supervised one, whenever the network's parameters are known and its activation function is invertible.

desk verdict A correct, modest algebraic reformulation that turns input reconstruction in ESNs into an unsupervised task; the math holds up and the paper deserves peer review. read the letter →

arxiv 2501.11409 v4 pith:75RVVWNH submitted 2025-01-20 cs.LG cs.AIeess.SPnlin.CDq-bio.NC

Taiki Yamada , Yuichi Katori , Kantaro Fujiwara This is my paper · ORCID
classification cs.LGcs.AIeess.SPnlin.CDq-bio.NC
keywords echostatenetworksreservoircomputinginputreconstructionunsupervisedlearningsignaltheoreticalneurosciencedynamicalsystemreplicationnoisefiltering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that input reconstruction in an echo state network — a recurrent network whose fixed reservoir projects inputs into high-dimensional states while only the output layer is trained — can be done without ever using the input as a supervised target. The key observation is that under the state update $r_{t+1} = \sigma(A d_t + B r_t)$, an invertible activation $\sigma$ and a full-column-rank input matrix $A$ let the input be recovered from two consecutive reservoir states alone: $d_t = A^+[\sigma^{-1}(r_{t+1}) - B r_t]$. Substituting this identity into the standard least-squares readout makes the readout for input reconstruction a closed-form function of reservoir states and the known parameters $A$, $B$, and $\sigma$, so the task becomes unsupervised. The same identity is then applied to reformulate two downstream tasks — replicating the dynamical system that generated the input, and filtering noise out of contaminated inputs — as unsupervised procedures, with numerical experiments on synthetic inputs, the Lorenz attractor, and noisy observations. If correct, the work establishes a working principle for reservoir computing: fixed parameters are not just constraints to respect but specific values worth exploiting, and it draws a formal link between echo state networks and predictive-coding models of perception.

What carries the argument

The carrying object is an inversion identity for the ESN update (Lemma 1): $d_t = A^+[\sigma^{-1}(r_{t+1}) - B r_t]$, which expresses the unobserved input at time $t$ as an explicit function of two consecutive reservoir states and the known matrices $A$, $B$, and $\sigma$. Substituting this identity into the least-squares readout formula converts the input-reconstruction readout from a function of labeled target samples into a function of reservoir states alone, and this substitution is the algebraic heart of the paper. The second piece of machinery is the auxiliary least-squares problem $L_{\sigma,T}(\hat B) = \sum_{t=1}^{T-1} \|\sigma^{-1}(r_{t+1}) - \hat B r_t\|^2$, whose minimizer $\hat B$ appears in the simplified readout $W_R = A^+(\hat B - B)$; because this loss depends only on $\sigma$ and on the states, it supports the sequential recursive-least-squares algorithm, and because it does not involve $A$ or $B$, it is the basis for the paper's claim that dynamical-system replication is a strictly simpler task than input reconstruction in terms of internal knowledge requirements.

What would settle it

A reader can verify the main theorem in a few lines of computation: with known $(A, B, \sigma)$ satisfying the conditions, generate an input, record the reservoir states, and compare the state-only readout $W_R = A^+[\sigma^{-1}(R_{2,T+1}) - B R_{1,T}]R^+_{1,T}$ with the supervised readout $W_D = D_{1,T}R^+_{1,T}$; in exact arithmetic they must coincide, so any discrepancy beyond floating-point error refutes Theorem 1. For the replication claim, on the paper's own Lorenz setup, evaluate the residual $(I - A A^+)(\hat B - B)$ and check whether the divergence between the unsupervised system $\hat f_R$ and the supervised system $\hat f_D$ tracks its size as the reservoir dimension is varied; if it does, the simplification in the replication step is approximate rather than exact.

Watch

Extended reading notes

Core claim

The paper's central claim is that input reconstruction in an echo state network is an unsupervised task. For the state update $r_{t+1} = \sigma(A d_t + B r_t)$, the authors prove (Theorem 1) that whenever the activation $\sigma$ is invertible and the input matrix $A$ has full column rank, the input is recoverable from consecutive reservoir states by $d_t = A^+[\sigma^{-1}(r_{t+1}) - B r_t]$, so the least-squares readout $W_D = D_{1,T}R^+_{1,T}$, which conventionally needs the original input as a target, equals $W_R = A^+[\sigma^{-1}(R_{2,T+1}) - B R_{1,T}]R^+_{1,T}$, which uses only reservoir states and the known parameters. When the reservoir state matrix has full row rank (Theorem 2), this simplifies to $W_R = A^+(\hat B - B)$, where $\hat B$ minimizes the state-only loss $L_{\sigma,T}(\hat B) = \sum_{t=1}^{T-1} \|\sigma^{-1}(r_{t+1}) - \hat B r_t\|^2$, which in turn admits an online recursive-least-squares update (Algorithm 1). The authors then show that the same machinery reformulates two applications: replicating the dynamical system that generated the input reduces to the autonomous map $\hat f_R(r) = \sigma[\hat B r]$, whose training needs only the activation function $\sigma$; and noise filtering proceeds by using this replicated map as the evolution law in an ensemble Kalman filter with adaptively estimated observation covariance. Numerical experiments demonstrate reconstruction of a synthetic input, replication of the Lorenz-63 attractor whose projected orbits match the true attractor's shape, and a reduction of reconstruction error from an RRMSE of about 1.03 to 0.50 under strong test-time noise. The authors draw the general principle: reservoir computing should exploit the specific values of fixed parameters, not merely keep them fixed.

Load-bearing premise

The load-bearing premise is the step $A W_R + B = \hat B$ in the dynamical-system application, which is exactly true only when the input dimension equals the reservoir dimension so that $A A^+ = I$; none of the paper's experiments satisfies this, since every reservoir is far larger than its input.

Editorial extensions

If this is right

  • Input reconstruction becomes a closed-form function of reservoir states and known parameters, so an ESN with known $A$, $B$, and $\sigma$ can train its readout online without any label channel, which suits autonomous or embedded processing.
  • Any task that reduces to input reconstruction inherits the unsupervised formulation: the paper demonstrates replication of the Lorenz-63 attractor from reservoir states alone, with the projected orbits of the replicated system matching the true attractor's shape.
  • Noise filtering becomes a two-stage unsupervised procedure — train the readout and the replicated dynamics at one noise level, then run an ensemble Kalman filter with adaptive observation-covariance estimation on noisier inputs — and the experiments show filtered reconstruction beats the unfiltered readout whenever test-time noise exceeds training-time noise.
  • Because the reconstruction readout depends only on the network and not on which input drove it, the upper limit of an ESN's information-processing capacity is independent of the input type, which separates network properties that help input reconstruction from input properties that hurt it.
  • The state-only loss $L_{\sigma,T}$ coincides with a predictive-coding objective that compares a stimulus-evoked state with an internally predicted one, giving a deductive rather than heuristic bridge from reservoir computing to models of perception in the brain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because all numerical experiments use reservoirs far larger than their inputs ($n_r > n_{in}$), the residual term $(I - A A^+)(\hat B - B)$ is nonzero in every trial, so the unsupervised replicated system matches the supervised one only approximately; re-running the Lorenz experiment at several reservoir sizes would show whether the divergence between the two systems scales with that residual.
  • A practical recipe follows for physical or analog reservoirs: whenever a trustworthy model of the reservoir's own dynamics is available, the same inversion identity could recover external driving signals from internally measured states alone, which is the system-identification direction the paper leaves open.
  • The surrogate-inverse experiments with the non-invertible ReLU activation show reconstruction degrades gracefully rather than failing outright, suggesting that invertibility is sufficient but not strictly necessary; quantifying how the error grows as the activation departs from invertibility would sharpen the theory.
  • Algorithm 1's update is a local, covariance-tracking rule, which makes it a candidate plasticity mechanism for neural circuits; testing whether a neuron-like implementation can carry the precision-matrix recursion would extend the paper's brain-modeling claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. Echo state networks (ESNs) conventionally train the readout layer to reconstruct the input using supervised targets. This paper shows that, when the activation function is invertible and the input matrix has full column rank, the least-squares readout for input reconstruction can be written in closed form using only the ESN parameters and observed reservoir states (Theorem 1, Eq. 2.4; Theorem 2, Eq. 2.5). The authors give an RLS algorithm for online readout computation and use the identity to reformulate dynamical system replication (Eq. 3.2) and a Kalman-filter-based noise filtering scheme (Algorithms 2 and 3) as tasks that do not require the original input sequence. Numerical experiments on a piecewise oscillatory signal, the Lorenz-63 system, and noisy reconstruction support the claims.

Significance. The central algebraic derivation is sound and I find no load-bearing error. Theorems 1 and 2 follow from Lemma 1 by direct substitution, and the numerical checks (Table 1) are consistent with the stated regularity conditions. I specifically examined the potential issue with Eq. (3.2): the equality A W_R + B = \hat B does not require A A^+ = I. Under condition (iii), \hat B - B = A D_{1,T} R^+_{1,T} lies in the range of A, so A A^+(\hat B - B) = \hat B - B. The paper states condition (iii) in Section 2.3, so this is a stated regularity assumption rather than an algebraic overreach. The contribution is conceptual: it identifies a new principle for reservoir computing and reinterprets two downstream tasks within the same framework. The paper is transparent about its limitations (invertibility of sigma, full parameter access, no benchmark comparisons), which are appropriate for a theory-oriented paper.

minor comments (5)
  1. [§3.1.2, Eq. (3.2)] The equality A W_R + B = \hat B in Eq. (3.2) is valid under the full-row-rank condition (iii) of Theorem 2 and does not require A A^+ = I, because \hat B - B lies in the range of A under that condition. A one-sentence cross-reference to condition (iii) at the point of use would prevent a reader from inferring an unintended square-matrix assumption.
  2. [Algorithm 1] Algorithm 1 initializes \hat B_1 = B, which appears to conflict with the later claim that \hat B can be computed without access to B. Please clarify that this initialization is a warm start and that the RLS recursion converges to the same minimizer for B-independent initializations, or modify the algorithm to use a B-independent initialization when the goal is to avoid using B.
  3. [§3.1.2] The phrase 'strictly simpler task' should be sharpened: it is the construction of the replicated system \hat f_R on the reservoir state space that avoids using A and B; verifying its equivalence to the true dynamical system still relies on the structural conditions from Section 2.
  4. [§3.2.3] The sentence 'Noise filtering without adaptation of covariance matrix \hat R_t also works when the input noise intensity during testing \sqrt{tr Σ^(2)} < 1 is 100 to 101' is garbled; the intended noise-intensity range should be restated clearly.
  5. [§4.4.1] There is a typo: 'full column rank dondition' should read 'full column rank condition'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central unsupervised IR theorem is an algebraic identity derived from the ESN state equation under stated invertibility assumptions; no fitted input is renamed as a prediction and the claim does not rest on self-citation.

full rationale

The central derivation is self-contained. Lemma 1 solves the ESN state update rt+1 = sigma(A dt + B rt) for dt under the stated assumptions that sigma is invertible and rank(A) = n_in, yielding dt = A^+[sigma^-1(rt+1) - B rt]. Corollary 1 concatenates this identity, and Theorem 1 substitutes it directly into the standard least-squares readout WD = D_{1,T} R^+_{1,T} to obtain WR. No parameter is fitted to a subset of data and then reported as a prediction: WR is an algebraic rewrite of the supervised readout using only known ESN parameters and reservoir states. Theorem 2 is likewise an algebraic simplification using rank(R_{1,T}) = n_r. The paper explicitly lists conditions (i), (ii), and (iii) as regularity assumptions and acknowledges in Section 4.4.1 that full knowledge of A, B, and sigma is required; these are stated limitations, not circular steps. The reviewer's concern about Eq. (3.2) requiring A A^+ = I does not amount to circularity, and the paper conditions the simplified replication formula on full row rank of R_{1,T}. Minor self-citations (Katori 2018; Yonemura and Katori 2021; Sato et al. 2024) appear only in the discussion of related error-feedback ESN designs and are not load-bearing for Theorem 1 or the unsupervised reformulation. Numerical evaluations compare the unsupervised readout with the supervised readout and the original input, but the learning formula itself remains self-contained. Therefore no circular step is identified.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The core theorems contain no fitted constants; A, B, and σ are given inputs. Numerical hyperparameters such as spectral radius scaling, input variance, ensemble size, and adaptation rate are experimental settings, not parameters fitted to make the central identity work. The key assumptions are invertibility of σ, full column rank of A, full row rank of R_{1,T}, and the standard least squares readout definition. The noise filtering section adds state-space assumptions for the ensemble Kalman filter.

assumptions (6)
  • domain assumption The activation function σ is invertible on the relevant domain.
    Invoked in Lemma 1 and Corollary 1 to recover A d_t from σ^{-1}(r_{t+1}). Fails for ReLU; the paper acknowledges degraded performance.
  • domain assumption The input weight matrix A has full column rank, rank(A)=n_in.
    Invoked in Lemma 1 to multiply by A^+ so that A^+ A d_t = d_t.
  • domain assumption The reservoir state matrix R_{1,T} has full row rank n_r.
    Needed in Theorem 2 to use R_{1,T} R^+_{1,T}=I. May fail for periodic inputs; the paper proposes noise injection.
  • standard math The standard least squares readout W_D = D_{1,T} R^+_{1,T} is the relevant supervised solution.
    Used to define the input reconstruction target and to identify W_R with W_D.
  • domain assumption Known theorems on reservoir computing dynamical system replication establish topological conjugacy for the supervised closed loop \hat f_D.
    The paper cites Pathak, Lu, Grigoryeva, and others for replication validity and does not reprove those results.
  • domain assumption In noise filtering, model error w_t and observation noise v_t are white Gaussian with covariances Q and R, and R can be adaptively estimated.
    Required for the ensemble Kalman filter formulation in Eq. (3.3) and Algorithm 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Learning in Echo State Networks for Input Reconstruction." pith.science (2026). https://pith.science/paper/75RVVWNH

@misc{pith2026250111409,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Learning in Echo State Networks for Input Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75RVVWNH}},
  note         = {Machine review of arXiv:2501.11409}
}
read the original abstract

Echo state networks (ESNs) are a class of recurrent neural networks in which only the readout layer is trainable, while the recurrent and input layers are fixed. This architectural constraint enables computationally efficient processing of time-series data. Traditionally, the readout layer in ESNs is trained using supervised learning with target outputs. In this study, we focus on input reconstruction (IR), where the readout layer is trained to reconstruct the input time series fed into the ESN. We show that IR can be achieved through unsupervised learning (UL), without access to supervised targets, provided that the ESN parameters are known a priori and satisfy invertibility conditions. This formulation allows applications relying on IR, such as dynamical system replication and noise filtering, to be reformulated within the UL framework via straightforward integration with existing algorithms. Our results suggest that prior knowledge of ESN parameters can reduce reliance on supervision, thereby establishing a new principle: not only by fixing part of the network parameters but also by exploiting their specific values. Furthermore, our UL-based algorithms for input reconstruction and related tasks are suitable for autonomous processing, offering insights into how analogous computational mechanisms might operate in the brain in principle. These findings contribute to a deeper understanding of the mathematical foundations of ESNs and their relevance to models in computational neuroscience.

Figures

Figures reproduced from arXiv: 2501.11409 by the authors.

Figure 1
Figure 1. Overview of the proposed input reconstruction formulation in ESNs. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Numerical example of Algorithm 1 used for input reconstruction based on [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Schematics of dynamical system replication, where an agent constructs a [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Typical orbits of the dynamical systems. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Schematics of noise filtering in ESNs. The ESN input time series is contami [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Typical example of input reconstruction with noise filtering. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Input reconstruction performance for different input noise intensities. The [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 70 canonical work pages

  1. [1]

    Akhlaghi, S., Zhou, N., and Huang, Z. (2017). Adaptive adjustment of noise covariance in kalman filter for dynamic state estimation. In 2017 IEEE power & energy society general meeting , pages 1--5

    work page 2017

  2. [2]

    and Cichocki, A

    Amari, S. and Cichocki, A. (1998). Adaptive blind signal processing-neural network approaches. Proceedings of the IEEE , 86(10):2026--2048

    work page 1998

  3. [3]

    Antonelo, E. A. and Schrauwen, B. (2009). Unsupervised Learning in Reservoir Computing : Modeling Hippocampal Place Cells for Small Mobile Robots . In Artificial Neural Networks – International Conference on Artificial Neural Networks , pages 747--756

    work page 2009

  4. [4]

    C., Van der Sande, G., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C

    Appeltant, L., Soriano, M. C., Van der Sande, G., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., and Fischer, I. (2011). Information processing using a single dynamical node as complex system. Nature Communications , 2(1):468

    work page 2011

  5. [5]

    and Clopath, C

    Asabuki, T. and Clopath, C. (2024). Embedding stochastic dynamics of the environment in spontaneous activity by prediction-based plasticity. eLife , 13

    work page 2024

  6. [6]

    and Fukai, T

    Asabuki, T. and Fukai, T. (2020). Somatodendritic consistency check for temporal feature segmentation. Nature Communications , 11(1):1554

    work page 2020

  7. [7]

    Bahroun, Y., Chklovskii, D., and Sengupta, A. (2021). A Normative and Biologically Plausible Algorithm for Independent Component Analysis . In Advances in Neural Information Processing Systems , volume 34, pages 7368--7384

    work page 2021

  8. [8]

    G., de Jong, T

    Baltussen, M. G., de Jong, T. J., Duez, Q., Robinson, W. E., and Huck, W. T. S. (2024). Chemical reservoir computation in a self-organizing reaction network. Nature , 631(8021):549--555

    work page 2024

Show all 74 references
  1. [9]

    Bell, A. J. and Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural computation , 7(6):1129--1159

    1995

  2. [10]

    J., Klaassen, C

    Bickel, P. J., Klaassen, C. A., Bickel, P. J., Ritov, Y., Klaassen, J., Wellner, J. A., and Ritov, Y. (1993). Efficient and adaptive estimation for semiparametric models , volume 4. Springer

    1993

  3. [11]

    Bollt, E. (2021). On explaining the surprising success of reservoir computing forecaster of chaos? the universal machine learning dynamical system with contrast to var and dmd. Chaos: An Interdisciplinary Journal of Nonlinear Science , 31(1)

    2021

  4. [12]

    Candy, J. V. (2005). Model-based signal processing . John Wiley & Sons

    2005

  5. [13]

    T., Rubanova, Y., Bettencourt, J., and Duvenaud, D

    Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. (2018). Neural ordinary differential equations. Advances in neural information processing systems , 31

    2018

  6. [14]

    Dambre, J., Verstraeten, D., Schrauwen, B., and Massar, S. (2012). Information processing capacity of dynamical systems. Scientific reports , 2(1):514

    2012

  7. [15]

    Daum, F. (2005). Nonlinear filters: beyond the kalman filter. IEEE Aerospace and Electronic Systems Magazine , 20(8):57--69

    2005

  8. [16]

    Deaves, R. (1999). Covariance bounds for augmented state kalman filter application. Electronics Letters , 35(23):2062--2064

    1999

  9. [17]

    Dinh, T. T. and Hinaut, X. (2020). Language Acquisition with Echo State Networks : Towards Unsupervised Learning . In 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics , pages 1--6

    2020

  10. [18]

    Dueben, P. D. and Bauer, P. (2018). Challenges and design choices for global weather and climate models based on machine learning. Geoscientific Model Development , 11(10):3999--4009

    2018

  11. [19]

    Evensen, G. (2003). The Ensemble Kalman Filter : theoretical formulation and practical implementation. Ocean Dynamics , 53(4):343--367

    2003

  12. [20]

    Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience , 11(2):127--138

    2010

  13. [21]

    Ghahramani, Z. (2003). Unsupervised learning. In Summer school on machine learning , pages 72--112. Springer

    2003

  14. [22]

    Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages 315--323. JMLR Workshop and Conference Proceedings

    2011

  15. [23]

    Goswami, D., Wolek, A., and Paley, D. A. (2021). Data-driven estimation using an Echo - State Neural Network equipped with an Ensemble Kalman Filter . In 2021 American Control Conference , pages 2549--2554

    2021

  16. [24]

    Grewal, M. S. and Andrews, A. P., editors (2001). Kalman filtering: theory and practice using MATLAB . A Wiley - Interscience publication. Wiley, New York, NY Weinheim

    2001

  17. [25]

    G., and Ortega, J.-P

    Grigoryeva, L., Hart, A. G., and Ortega, J.-P. (2021). Chaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem. Physical Review E , 103(6):062204

    2021

  18. [26]

    G., and Ortega, J.-P

    Grigoryeva, L., Hart, A. G., and Ortega, J.-P. (2023). Learning strange attractors with reservoir systems. Nonlinearity , 36(9):4674

    2023

  19. [27]

    Hamilton, F., Berry, T., and Sauer, T. (2016). Ensemble kalman filtering without a model. Physical Review X , 6(1):011021

    2016

  20. [28]

    and Wang, Y

    Han, M. and Wang, Y. (2009). Nonlinear time series online prediction using reservoir kalman filter. In 2009 International Joint Conference on Neural Networks , pages 1090--1094

    2009

  21. [29]

    and Kokubu, H

    Hara, M. and Kokubu, H. (2022). Learning Dynamics by Reservoir Computing ( In Memory of Prof . Pavol Brunovský ). Journal of Dynamics and Differential Equations

    2022

  22. [30]

    Hart, A. G. (2024). Generalised synchronisations, embeddings, and approximations for continuous time reservoir computers. Physica D: Nonlinear Phenomena , 458:133956

    2024

  23. [31]

    G., Hook, J., and Dawes, J

    Hart, A. G., Hook, J., and Dawes, J. (2020). Embedding and approximation theorems for echo state networks. Neural Networks , 128:234--247

    2020

  24. [32]

    G., Hook, J

    Hart, A. G., Hook, J. L., and Dawes, J. H. P. (2021). Echo State Networks trained by Tikhonov least squares are L2 ( ) approximators of ergodic dynamical systems. Physica D: Nonlinear Phenomena , 421:132882

    2021

  25. [33]

    Haykin, S. S. (2002). Adaptive filter theory . Pearson Education India

    2002

  26. [34]

    and Toyoizumi, T

    Isomura, T. and Toyoizumi, T. (2016). A Local Learning Rule for Independent Component Analysis . Scientific Reports , 6(1):28073

    2016

  27. [35]

    echo state

    Jaeger, H. (2001). The “echo state” approach to analysing and training recurrent neural networks. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report , 148(34):13

    2001

  28. [36]

    Jaeger, H. (2002). Short term memory in echo state networks. GMD-Report 152, GMD - German National Research Institute for Computer Science

    2002

  29. [37]

    and Herault, J

    Jutten, C. and Herault, J. (1991). Blind separation of sources, part I : An adaptive algorithm based on neuromimetic architecture. Signal Processing , 24(1):1--10

    1991

  30. [38]

    Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME--Journal of Basic Engineering , 82(Series D):35--45

    1960

  31. [39]

    and Hasselblatt, B

    Katok, A. and Hasselblatt, B. (1995). Introduction to the Modern Theory of Dynamical Systems . Number 54 in Encyclopedia of Mathematics and its Applications. Cambridge University Press

    1995

  32. [40]

    Katori, Y. (2018). Network Model for Dynamics of Perception with Reservoir Computing and Predictive Coding . In Advances in Cognitive Neurodynamics ( VI ) , Advances in Cognitive Neurodynamics , pages 89--95

    2018

  33. [41]

    Z., Lu, Z., Nozari, E., Pappas, G

    Kim, J. Z., Lu, Z., Nozari, E., Pappas, G. J., and Bassett, D. S. (2021). Teaching recurrent neural networks to infer global temporal structure from local examples. Nature Machine Intelligence , 3(4):316--323

    2021

  34. [42]

    Knill, D. C. and Pouget, A. (2004). The Bayesian brain: the role of uncertainty in neural coding and computation. Trends in Neurosciences , 27(12):712--719

    2004

  35. [43]

    and Parlitz, U

    Kocarev, L. and Parlitz, U. (1996). Generalized synchronization, predictability, and equivalence of unidirectionally coupled dynamical systems. Physical review letters , 76(11):1816

    1996

  36. [44]

    and Duan, N

    Li, K.-C. and Duan, N. (1989). Regression Analysis Under Link Violation . The Annals of Statistics , 17(3):1009--1052

    1989

  37. [45]

    and Bassett, D

    Lu, Z. and Bassett, D. S. (2020). Invertible generalized synchronization: A putative mechanism for implicit learning in neural systems. Chaos: An Interdisciplinary Journal of Nonlinear Science , 30(6):063133

    2020

  38. [46]

    R., and Ott, E

    Lu, Z., Hunt, B. R., and Ott, E. (2018). Attractor reconstruction by machine learning. Chaos: An Interdisciplinary Journal of Nonlinear Science , 28(6):061104

    2018

  39. [47]

    Lu, Z., Pathak, J., Hunt, B., Girvan, M., Brockett, R., and Ott, E. (2017). Reservoir observers: Model -free inference of unmeasured variables in chaotic systems. Chaos: An Interdisciplinary Journal of Nonlinear Science , 27(4):041102

    2017

  40. [48]

    Lukoševičius, M., Jaeger, H., and Schrauwen, B. (2012). Reservoir Computing Trends . KI - Künstliche Intelligenz , 26(4):365--371

    2012

  41. [49]

    Maass, W., Natschläger, T., and Markram, H. (2002). Real- Time Computing Without Stable States : A New Framework for Neural Computation Based on Perturbations . Neural Computation , 14(11):2531--2560

    2002

  42. [50]

    Mehra, R. K. (1972). Approaches to adaptive filtering. IEEE Transactions on Automatic Control , 17(5):693--698

    1972

  43. [51]

    Mohamed, A. H. and Schwarz, K. P. (1999). Adaptive Kalman Filtering for INS / GPS . Journal of Geodesy , 73(4):193--203

    1999

  44. [52]

    Murphy, K. P. (1998). Switching kalman filters. Technical Report 98-10, Compaq Cambridge Research Laboratory, Cambridge, MA

    1998

  45. [53]

    Nakajima, K. (2020). Physical reservoir computing—an introductory perspective. Japanese Journal of Applied Physics , 59(6):060501

    2020

  46. [54]

    Nakajima, K., Hauser, H., Li, T., and Pfeifer, R. (2015). Information processing via physical soft body. Scientific Reports , 5(1):10487

    2015

  47. [55]

    Pathak, J., Hunt, B., Girvan, M., Lu, Z., and Ott, E. (2018). Model- Free Prediction of Large Spatiotemporally Chaotic Systems from Data : A Reservoir Computing Approach . Physical Review Letters , 120(2):024102

    2018

  48. [56]

    R., Girvan, M., and Ott, E

    Pathak, J., Lu, Z., Hunt, B. R., Girvan, M., and Ott, E. (2017). Using machine learning to replicate chaotic attractors and calculate lyapunov exponents from data. Chaos: An Interdisciplinary Journal of Nonlinear Science , 27(12):121102

    2017

  49. [57]

    Pearson, K. (1901). Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science , 2(11):559--572

    1901

  50. [58]

    Rao, R. P. N. and Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience , 2(1):79--87

    1999

  51. [59]

    J., and Fischer, I

    R \"o hm, A., Gauthier, D. J., and Fischer, I. (2021). Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing. Chaos: An Interdisciplinary Journal of Nonlinear Science , 31(10):103127

    2021

  52. [60]

    Sato, Y., Yamamoto, H., Ishikawa, Y., Sumi, T., Sono, Y., Sato, S., Katori, Y., and Hirano-Iwata, A. (2024). In silico modeling of reservoir-based predictive coding in biological neuronal networks on microelectrode arrays. Japanese Journal of Applied Physics , 63(10):108001

    2024

  53. [61]

    A., and Casdagli, M

    Sauer, T., Yorke, J. A., and Casdagli, M. (1991). Embedology. Journal of Statistical Physics , 65(3):579--616

    1991

  54. [62]

    and Messori, G

    Scher, S. and Messori, G. (2019). Generalization properties of feed-forward neural networks trained on lorenz systems. Nonlinear processes in geophysics , 26(4):381--399

    2019

  55. [63]

    Schrauwen, B., Verstraeten, D., and Van Campenhout, J. (2007). An overview of reservoir computing: theory, applications and implementations. Proceedings of the 15th European Symposium on Artificial Neural Networks. p. 471-482 2007 , pages 471--482

    2007

  56. [64]

    Steiner, P., Jalalvand, A., and Birkholz, P. (2023). Exploring unsupervised pre-training for echo state networks. Neural Computing and Applications , 35(34):24225--24242

    2023

  57. [65]

    B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., and Hirose, A

    Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., and Hirose, A. (2019). Recent advances in physical reservoir computing: A review. Neural Networks , 115:100--123

    2019

  58. [66]

    and Sawada, Y

    Tomizawa, F. and Sawada, Y. (2021). Combining ensemble Kalman filter and reservoir computing to predict spatiotemporal chaotic systems from imperfect observations and models. Geoscientific Model Development , 14(9):5623--5635

    2021

  59. [67]

    Tsai, C.-Y., Dutoit, X., Song, K.-T., Van Brussel, H., and Nuttin, M. (2010). Robust face tracking control of a mobile robot using self-tuning Kalman filter and echo state network. Asian Journal of Control , 12(4):488--509

    2010

  60. [68]

    Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A. (2008). Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning , pages 1096--1103

    2008

  61. [69]

    R., Sapsis, T

    Vlachas, P.-R., Pathak, J., Hunt, B. R., Sapsis, T. P., Girvan, M., Ott, E., and Koumoutsakos, P. (2020). Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks , 126:191--217

    2020

  62. [70]

    R., Pomerance, A., Antonsen, T., and Ott, E

    Wikner, A., Harvey, J., Girvan, M., Hunt, B. R., Pomerance, A., Antonsen, T., and Ott, E. (2024). Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing. Neural Networks , 170:94--110

    2024

  63. [71]

    and Sejnowski, T

    Wiskott, L. and Sejnowski, T. J. (2002). Slow Feature Analysis : Unsupervised Learning of Invariances . Neural Computation , 14(4):715--770

    2002

  64. [72]

    Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., and Sun, J. (2024). Emerging opportunities and challenges for the future of reservoir computing. Nature Communications , 15(1):2056

    2024

  65. [73]

    and Katori, Y

    Yonemura, Y. and Katori, Y. (2021). Network model of predictive coding based on reservoir computing for multi-modal processing of visual and auditory signals. Nonlinear Theory and Its Applications, IEICE , 12(2):143--156

    2021

  66. [74]

    and Vargas, D

    Zhang, H. and Vargas, D. V. (2023). A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning . IEEE Access , 11:81033--81070

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.